TimeCapsules | Devpost

landing page
setup page
picture page! uses computer vision and AI to interpret, understand, and create custom reminders!
success!

Inspiration

There are over 800 million people around the world over the age of 65, and as healthcare advances, the elderly population will increase even faster. The one thing they all have in common is that they frequently take some medication or pills. In fact, almost 90% of older adults regularly take at least one prescription drug, and almost 80% regularly take at least two prescription drugs. Whether it be for lowering cholesterol or just fulfilling your daily Vitamin C intake, they are essential for maintaining the health of people of all ages.

However, the text on the medical bottles is often small and complex, making it hard to understand, especially for the elderly or visually impaired.

About 1 in 9 people age 65 and older has some form of Alzheimer's. Especially when people usually take more than one type of medication, it becomes hard to manage when and how often to take each one.

That's where TimeCapsules comes in.

What it does

TimeCapsules uses Computer Vision, AI, and SMS to innovate how we use medication. Our application will leverage OpenCV to take a picture of your pill bottle, parse the text, and then utilize OpenAI to interpret, and create a custom SMS reminder to your phone. Twilio will then send text reminders to take your pills depending on the instructions listed on the bottle.

Start by filling in some basic information, and then simply face the bottle to your camera. It will snap a photo and interpret the text. It will now be added to a dashboard where you can manage all your medications. The app will also send a text to your phone when you wake up, reminding you to take your pills! Depending on the instructions, it will continue to text you throughout the day.

How we built it

Front-end: React Backend: Python, Flask APIs: OpenCV, OpenAI, Twillio

Challenges we ran into

Integrating front-end with back-end. Clashing dependencies. Passing parameters between different functions and making sure they fit the required type. Test Data: 75% accurate (15/20 attempts), the image must be as clear as possible for OpenCV to accurately read the instructions. We would love to make this higher.

Accomplishments that we're proud of

Completing the project! Working with a variety of different technologies and languages, such as Twilio, OpenAi, OpenCV, Flask, React, etc.

What we learned

How to do full-stack development. Working with APIs. Using computer vision.

What's next for TimeCapsules

Convert to mobile app, to increase image quality, test results and increase accessibility. Add more features, such as a "reply to confirm pill taken" to notify the guardian of successful intake. Increase the scope of things it can remind you about, still using our computer vision technology to input (to take in food, tasks etc) Integrate live video reading, for a more streamline process.