Third Eye

What it does

Third Eye combines voice audio input with a portable camera to aid the blind and severely visually impaired with understanding and navigating their surroundings. It is also helpful for those with vision, since it can be configured to expand the user's visual field or be used as a security option that doesn't require visual attention.

How we built it

We created Third Eye by combining various libraries, frameworks, and tools such as React Speech Recognition, GPT-4, and Web Text to Speech within a carefully designed application. We researched how to use each one and interfaced them with each other, as well as our own laptops and phones. We tested on multiple devices to ensure functionality and a future compact physical product for Third Eye.

Challenges we ran into

We struggled with several bugs pertaining to the behaviour of Third Eye while it was under construction; we overcomplicated our back end in the beginning and library usage. However, we managed to succeed in critically thinking about our design and cutting out unnecessary components. In addition, we initially had unreliable speech detection. Determined, we turned to different API options until we found one that was suited to the needs of Third Eye.

Built With

• css
• next.js
• react
• typescript
• web-speech
• whisper