Project Team
Students
Yuqi Zhang
Computer Science
Penn State Harrisburg
Faculty Mentors
Truong X. Tran
Harrisburg
Computer Science
Von Lockette, Paris
University Park
Mechanical Engineering
Project
Project Video
Project Abstract
Based on artificial intelligence (AI) and machine learning (ML), smart devices such as smartphones and tablets are able to detect and track the motion of objects in three-dimension. For example, the development of AI/ML models such as ML Kit Pose Detection API (Google) and Augmented Reality Kit (Apple) has enable the camera of mobile devices to capture and quantify human body motion. With this technology, mobile devices are able to accurately determine multidimensional kinematics across joints, collecting reliable and repeatable data for functional tasks. Mainstream Motion Capture Systems (such as Vicon, Qualisys, etc.) that are used more at present commercial scenarios have the disadvantages of high cost, complicated use and low portability. Relatively, the motion capture application that developed based on mobile devices have the advantages of low cost and high portability. At same time, it can still accurately detect and capture motion in the three-dimension. This application was designed to run on Apple (Apple Inc., USA) iOS mobile devices (iPhone and iPad). To capture joints motion, Apple ARKit-3 is powered by machine learning models running on Apple’s Neural Engine chip, using the Xcode 11 IDE and the Swift programming language. The application aim to capture 3D motion and calculate 3D joint angle for the upper limb joints of human body. The proposed applications include industrial and sports scenarios. The development of accurate motion tracking and detection can help to avoid incorrect/wrong movements, which can help to reduce the potential risky scenario. Accurately correcting wrong or bad moving patterns can significantly shorten the training time of specific motion for workers/athletes and effectively reduce the risk of injury of workers/athletes. With machine learning and commercial cameras (like iPhones or iPads), the difficulty of use can also be significantly reduced while maintaining effectiveness and accuracy.
Evaluate this Project
Use this form link to provide feedback to the presenters, and add your project evaluation for award(s) consideration.