Assistance app for bedridden patients
HelpWave enables bedridden patients with limited mobility to communicate with caregivers using only their eye winks and tongue movements, detected in real time by the phone's front camera using on-device machine learning - no wearables or specialized hardware required.
A non-profit organization dedicated to creating accessible communication technologies for patients with neurodegenerative diseases and severe mobility impairments.
Patients with severe mobility impairments - such as those with ALS, advanced Parkinson's, or locked-in syndrome - are often unable to call for help using conventional nurse call systems. Even voice-activated systems fail when patients cannot speak clearly. Existing assistive technology is expensive, bulky, and requires specialist setup.
We built a mobile app using on-device machine learning with Apple's Vision framework and ARKit. A facial landmark model tracks the patient's face continuously, detecting intentional wink patterns and tongue protrusion events. These gestures are mapped to configurable commands - such as "Call nurse," "Need pain medication," or custom messages. When triggered, the app sends push notifications to caregivers' devices instantly.
The app uses the phone's front TrueDepth camera and ARKit's face tracking to monitor 52 facial landmarks in real time at 60fps. A trained classifier distinguishes intentional winks and tongue movements from natural blinking and random facial activity, achieving greater than 97% precision in controlled testing.
Each detected gesture triggers a push notification that is delivered to all registered caregiver devices within under one second. Notifications include the patient's name, their room, and the specific request type. Caregivers can acknowledge the request from their phone, which clears the alert and logs the response time for care quality auditing.
Beyond emergency requests, HelpWave includes a simple communication board - activated by gestures - that allows patients to select from common phrases or pre-configured personal messages. This gives patients a voice in routine interactions, reducing feelings of isolation and enabling more meaningful communication with family and nursing staff.
The on-device CoreML model was trained on a dataset of over 50,000 facial gesture events, captured from participants across a range of ages, skin tones, and lighting conditions. A dwell-time threshold prevents false positives from accidental expressions, with the sensitivity level adjustable by caregivers to suit each patient's capabilities.
HelpWave was piloted in a palliative care ward, where 8 patients with advanced ALS used the system. Response times to patient requests improved by 45% compared to the existing call bell system. All 8 patients reported feeling more independent and less anxious about their ability to communicate needs.
Computer Vision Engineer
Expert in facial tracking and gesture detection
iOS Developer
Builds native high-performance Apple applications
UX/UI Designer
Key contributor to project success
QA Engineer
Ensures technical integrity and bug-free delivery
This project can be extended to integrate with smart room environments (IoT), allowing patients to control lighting, blinds, and television using the same facial gestures. Custom voice synthesis support can also be added, enabling patients who use an eye-tracking keyboard to communicate using a personalized synthetic voice.
Tell us what you're building - we'll help you ship it.
Contact Us
2915 Ogletown Road, Newark, DE 19713, USA
213, G-Block, V.V.C.E, Gokulam, Mysore 570002, India
Connect with us
contact@neuroflares.com