Creating an augmented reality application that will enable users to communicate more effectively using their surroundings.
Sponsored by: PSU RERC on AAC
Team Members
Joel Hodges Chen-Kuan Liao Ashley Amendola Amber Singh Ashwin Subramanian Sreemannarayan Bojja Haneesh Bommireddy
Instructor: Steven Shaffer
Project Poster
Click on any image to enlarge.
Project Video
Project Summary
Overview
The Research and Engineering Center (RERC) on Augmentative and Alternative Communication (AAC) works towards offering innovative solutions for individuals with communication challenges. Therefore, our team’s goal was to improve upon RERC’s pre-existing application by making it a more user-friendly, efficient, and personalized tool that facilitates communication for non-verbal individuals by leveraging image recognition.
Objectives
– Create a personalized model that allows users to upload their own photos and videos for image recognition.
– Modify the user interface (UI) of the existing model to enable user-uploaded media.
– Demonstrate a successful test case using a broader range of comparable objects.
Approach
– Modified UI objects in Unity and C# files in VS Code to improve the application’s UI elements and add button functionality in the main menu page.
– Researched various solutions for giving users the capability of adding their personal images and videos and saving it within the application.
– Worked on training the new model to give it the ability to distinguish between different objects from user-uploaded media.
Outcomes
Our team was able to improve upon the pre-existing model in which users can now upload their personal photos and videos of objects to be trained and later recognized by the application. These new features are now functional due to our enhanced UI. Overall, the application runs successfully in recognizing new objects identified by media that the user uploads.