The Project “සංකල්ප” for Hearing Impaired Students

• Research Project
• Hackathon and Industry Project

Why this Project ?
Hearing Impaired children are less likely to go to school or complete their education due to the limitations and differentiation they face from the society. They are often hidden away and excluded from family and community life which will impede their development due the lack of knowledge in how to communicate with others in the society and limitations in understanding basic concepts in their learning curve.
Sign language comes into the picture in this scenario, but still there are some gaps to fill. Silent communication with hand signs is less capable of generating mind maps, concepts and theories to students. That is the pain point we are going to address and unlike any other learning aid, this project solution can enhance deaf students’ learning experience by leveraging the absorb rate of educational facts. Project “සංකල්ප” will be the next evolution in learning for deaf children in worldwide who are trying to win the world.



Approach to Implement Project “සංකල්ප”
During my 2nd year at University, I published a research paper titled "A Mobile Application with Augmented Reality to Enhance Sinhala Learning Experience for Children".

In that, I designed a mobile application, which is for Android OS. In that project, the app was designed to render AR models of Sinhala Alphabet and display according to the Image Markers that we used. That research artifact was taken as the starting point for Project “සංකල්ප”. I had a passionate team behind this. Without Kolitha, Thilina(Mentor), Shakya, Gayathree and Chamod, this project will not be a reality. We started from sketches, then flow charts, real personal analysis and field visits and many more paper work. And we came up with following high level module architecture.
To make it easy to design and maintain, 3 separate modules were grouped out of this entire product design.

1. Gesture Capturing & Recognizing Module
This module is equipped with a Kinect Device to capture hand gestures and a software layer to process it and identify those captured signals from the device.


2. Student Tracking Application
This is a mobile application to track hearing impaired students. GPS trackers and mobile phones are used to capture location details. And a special app view is given to teachers to get all student location details at a glance.


3. Augmented Reality Mobile Application Module
This is an innovative AR functionality which is embedded to Project “සංකල්ප” to render interactive AR content right at the mobile screen. We used a book contains Alphabet and use the existing pages of that as Image Markers. So when a student points the mobile phone camera towards a letter, the letter and a cartoon 3D object will get rendered with interactive way, including a virtual video playback with sound explaining the relevant content.

To make this product much more user friendly, we used gamification techniques. And as per the real user test with the project, this concept got extremely well feedback from students and teachers.
As a team, we love to expand this project as a community work and make is a national level project. We love to hear your suggestions and thoughts on this.