Christopher G. Burford , Sam Prasanna James, Blake A. Mathie
ABSTRACT
Through our association with the Active Reach and Manipulation (ARM) Clinic, run by the Occupational Therapy Program, Wayne State University, we were presented with a challenge to develop a video game that could improve the functional outcomes of stroke rehabilitation therapy. Our approach focused on creating an immersive and engaging experience that would incentivize maximal, autonomous patient effort and control during rehabilitation exercises while generating useful performance data for the therapist. The resulting product utilizes a 3-axis wireless sensor, strapped to the person’s hand, to them to fly a simulated airplane through series’ of rings using the pronation and supination movements of their hand. This paper details the development, expectations, and future development plans for this rehabilitation gaming system.
BACKGROUND
In 2013 there were over 795,000 people in the U.S. that suffered a stroke costing nearly $39 billion in health care and work related expenses (CDC, 2013)(1). In light of such significant loading of the healthcare system, both financially and physically, efforts must be made to reduce the cost of individual rehabilitation efforts while improving the functional outcomes for those individuals.
Upper extremity stroke rehabilitation research has demonstrated the effectiveness of high repetition therapeutic activities(Butefisch, Hummelsheim, Denzler, & Mauritz, 1995(2); G. Kwakkel, 2006(3); G. Kwakkel, Kollen, & Lindeman, 2004(4)), motivating and engaging games and tasks (Bach-y-Rita et al., 2002(5); Lotze, Braun, Birbaumer, Anders, & Cohen, 2003(6); Wood et al., 2003(7)) , and use of just-right challenge strategies (Cameirao, Bermúdez i Badia, Oller, & Verschure, 2010(8)) to facilitate effective neural plasticity and improved function. The therapeutic exercise game developed for this project is designed to monitor performance and provide “just-right challenges”, within a motivating gaming context, with the objective of yielding the high repetition rates necessary for improved arm – hand function.
Virtual reality in the form of games have been shown to facilitate lengthy engagements resulting from a highly motivated person (G. Burdea, 2003(9); G. C. Burdea, Jain, Rabin, Pellosie, & Golomb, 2011(10)). Furthermore, research and experience have shown that such engagement coupled with feedback of results and even simple rewards built into the game for good performance facilitate compliance and extended periods of interventions (Dreeben-Irimia, 2010(11)). Off-the-shelf games can be beneficial, but present real obstacles for many persons with severe hand, wrist and fine motor control (G. Burdea, 2003(9); G. C. Burdea et al., 2011(10)). This last point speaks to the need for customized game and user interfaces for people involved in rehabilitation therapy. Finally, virtual reality and gaming are not just for the young, there is a growing body of evidence showing that the elderly also respond to games, but that the game design and format generally need to be less complex than off-the-shelf games (Lange et al., 2010(12)).
METHODOLOGY
Game Design
For this project we used Unity game engine (Unity Technologies, San Francisco CA.) to build the game. The game was organized into levels in order to address technical and experiential issues pertaining to the establishment and maintenance of patient information, game configuration, and actual game play. The user interface is a small wireless sensor strapped to the user’s wrist. This provides a very simple game controller that does not require gripping or the pushing of buttons. This interface allows specific and focused attention to pronation and supination. Different games and user interfaces will be used to address grip and other fine motor hand functions.
Level one is dedicated to user identification, sensor calibration, base line patient performance measurement, and game configurations. Input begins with uer identification and game settings which focus the ensuing game play on control, speed, or range of motion with the specific needs of the user. Maximum pronation and supination, reversal time, and neutral position of the user’s hand are measured, recorded, and marked with a graphical representation. This level requires a minute or less to complete and should be accomplished with the assistance of a therapist.
Figure 1: Level one, calibration. Player is directed to enter name and game settings followed by a series of baseline calibrations and performance measurements.
Figure 2: Level two gameplay. Player is offered three chances to reach the rings before game play is concluded.
Level two is actual game play. The user controls an airplane flying over the ocean. The objective is to fly through every ring as they appear. Each ring rewards the user with points and a subtle audible signal indicating that they successfully reached the target. Position, interval, and frequency of the ring appearance can be set during the level one. The rings are grouped in sets, between which a rest period is provided to the user allowing time for recovery between sets.
If a ring is missed, a different audible cue informs the user of the missing event. The missed ring is then moved further back to provide the user with another chance . If three rings are missed it is assumed that the user has fatigued beyond their ability to continue with the game and the level ends. Each successive set challenges the user to maintain or increase Range of Motion (ROM) while maintaining control. Such increases are constrained by the user’s performance during the previous set.
The gaming environment, comprised by both visual and audible elements, was designed to provide an immersive and tranquil experience for the player, with a level of complexity befitting an individual experiencing a low to moderate level of cognitive impairment. To achieve this goal, we limited the visual complexity of the aircraft and background and used high-contrast but muted colors to enhance the player’s ability to visually track the action. Additionally, the drone of the jet and background music was selected to elicit a sense of calm and to limit distractions. Audible and visual performance cues are in place to give the user real-time feedback and encouragement without creating events which remove the user from the immersive gaming experience.
Data Collection and Analysis
The game uses 3 space sensors from Yost engineering to wirelessly transmit the data to a USB dongle. The dongle when connected to the computer will show itself as a serial port device. A C# library was written to acquire data from the wireless sensors that data is collected for subsequent analysis and is also used to control the game object. The collected data is also written to a file in comma separated values. The data file contains a timestamp (the interval between two timestamps gives the time interval at which the data is captured), X axis data, Y axis data and Z axis. X, Y and Z values are normalized to maximum pronation and supination user performance between 70° pronation and 90° supination, with neutral hand position represented as zero. The post processing was done in Microsoft Excel application. Due to time and patient availability constraints we collected the movement data from a healthy subject exhibiting no abnormalities in wrist and elbow ROM. The ROM for supination and pronation for the subject was measured using a goniometer and determined to be 90° and 72°respectively. This manual measurement is done to validate our sensor data. The displacement data is plotted using the XY scatter plot. The velocity, acceleration and jerk were then derived and plotted using first, second and third derivatives to the displacement data accordingly.
Figure 3: This is ROM chart for supination and pronation of hand
EVALUATION
For this project our client was, Dr. Gerry Conti of the Wayne State University, director – ARM Clinic. Prior to entering into clinical evaluation using persons recovering from stroke, our client expected that the game would meet certain criteria when used by a fully functional individual. During game play all intended functionality was stable, easily accessed, and created the engaging and immersive environment as originally intended. Dr. Conti has since agreed to move our gaming system into her lab for use with persons under IRB protocols in anticipation of future publications. Furthermore, based on the recommendation of our client, we planned to file a provisional patent and the project team has resolved to continue development with the intent of establishing a new company to commercialize this concept.
DISCUSSION
The current state of the art in the use of video games in rehabilitation medicine leaves a great deal of space for further development and has not yet been exhaustively studied. While they are able to improve performance outcomes, they are only intended to apply to a single device or family of devices and do not use active data to adapt game play to promote maximum effort within the constantly diminishing capabilities of the player. Furthermore, there are not adjustments which address varying levels of cognitive capabilities.
To be successful our solution had to address the following criteria:
- Create an engaging and immersive gaming experience
- Provide a therapeutically suitable user interface for game control
- Generate timely and positive in-game reinforcement for successful performance of rehabilitation exercises
- Ensure that the level of detail and complexity accommodates users with reduced cognitive function
- Establish a baseline performance evaluation
- Provide high resolution, real-time data mapping the performance of a specific rehabilitation related function
- Actively adapt the difficulty of game play to ensure the user is encouraged to and able to achieve the rehabilitative goals
- Generate a detailed and clinically relevant analysis of the user’s performance
- Empower the user to properly and repeatedly perform a clinically relevant exercise with minimum therapist intervention
Our solution met all criteria and resulted in the creation of a gaming system that was task rather than device centric. With further development and additional sensors, it will be capable of promoting improved functional outcomes on any exercise device and will ultimately make it possible for therapists to monitor the performance of activities specific to standard functional assessment protocols as a telemedicine application.
ACKNOWLEDGEMENTS:
We would like to recognize and thank Dr. Robert Erlandson, Dr. Gerry Conti, Dr. Donna Case and Prem Kumar Sivakumar, our GTA, for their generosity and commitment as mentors and for their thoughtful advice throughout.
REFERENCES:
1. CDC. (2013). Stroke Facts. http://www.cdc.gov/stroke/facts.htm. 2013
2. Butefisch, C., Hummelsheim, H., Denzler, P., & Mauritz, K. H. (1995). Repetitive training of isolated movements improves the outcome of motor rehabilitation of the centrally paretic hand. . Journal of Neurologic Sciences, 130, 59-68.
3. Kwakkel, G. . (2006). Impact of intensity of practice after stroke: Issues for consideration. Disability and Rehabilitation, 28 (13-14), 823-830.
4. Kwakkel, G., Kollen, B. J., & Lindeman, E. . (2004). Understanding the pattern of functional recovery after stroke: Facts and theories. . Restorative Neurology and Neuroscience, 22, 281-299.
5. Bach-y-Rita, P., Wood, S. , Leder, R. , Paredes, O., Wicab Bach-y-Rita, E., & Murillo, N. (2002). Computer-assisted motivating rehabilitation (CAMR) for institutional, home, and educational late stroke programs. Topics in Stroke Rehabilitation, 8(4), 1-10.
6. Lotze, M., Braun, C., Birbaumer, N., Anders, S., & Cohen, L. G. . (2003). Motor learning elicited by voluntary drive. Brain, 126(866-872).
7. Wood, S. R., Murillo, N., Bach-y-Rita, P., Leder, R. S., Marks, J. T., & Page, S. J. (2003). Motivating, game-based stroke rehabilitation: a brief report. Topics in Stroke Rehabilitation, 10(2), 134-140.
8. Cameirao, M. S., Bermúdez i Badia, S., Oller, E. D., & Verschure, P. F. M. J. (2010). Neurorehabilitation using the virtual reality based Rehabilitation Gaming System: Methodology, design, psychometrics, usability and validation. Journal of NeuroEngineering and Rehabilitation, 7, 48-.
9. Burdea, GC. (2003). Virtual rehabilitation-benefits and challenges. Methods of Information in Medicine-Methodik der Information in der Medizin, 42(5), 519-523.
10. Burdea, Grigore C, Jain, Abhishek, Rabin, Bryan, Pellosie, Richard, & Golomb, Meredith. (2011). Long-term hand tele-rehabilitation on the playstation 3: Benefits and challenges. Paper presented at the Engineering in Medicine and Biology Society, EMBC, 2011 Annual International Conference of the IEEE.
11. Dreeben-Irimia, Olga. (2010). Patient education in rehabilitation: Jones & Bartlett Learning.
12. Lange, BS, Requejo, P, Flynn, SM, Rizzo, AA, Valero-Cuevas, FJ, Baker, Lisa, & Winstein, C. (2010). The potential of virtual reality and gaming to assist successful aging with disability. Physical medicine and rehabilitation clinics of North America, 21(2), 339.
Primary Author:
Christopher Burford
1847 Trailwood ct.
Windsor, Ontario, CA
N8W5G7
fb7374@wayne.edu