Problem Statement

People with severe physical disabilities, speech difficulties, or limited hand movement often face significant challenges in expressing their basic needs, discomfort, requests, and emergency situations. Traditional communication methods may depend on speaking, writing, typing, or physical buttons, which can be difficult or impossible for some individuals to use independently. This can lead to delays in communication, increased dependence on caregivers, and difficulty receiving timely assistance. In particular, communicating simple needs such as food, water, restroom assistance, or urgent help can become challenging in everyday situations. Therefore, there is a need for a simple, accessible, and reliable communication method that allows individuals with limited physical or speech abilities to express their needs more easily and receive appropriate assistance when required.

Solution

Our AI-powered application converts a non-speaking patient's sign language into text, analyzes their symptoms, and provides a quick health summary with possible condition matches, helping doctors understand patients faster and reduce consultation time.

In Detail

Our solution is an AI-powered healthcare application designed to improve communication between non-speaking patients and doctors. The patient communicates their symptoms using sign language through the camera. The system uses hand and pose detection to recognize the signs and convert them into meaningful text. It then organizes the symptoms, including details such as duration and severity, into a structured medical summary. AI analyzes the collected symptoms and provides possible condition matches and an urgency indication to support the doctor's initial assessment. Instead of making the doctor spend valuable consultation time collecting basic information, the application provides the key symptoms in a clear and organized format before or at the beginning of the consultation. This allows doctors to understand the patient's condition more efficiently while ensuring that non-speaking patients can communicate their health concerns clearly and independently. The final medical assessment and diagnosis remain with the qualified healthcare professional.