Guardian AIngel

• 

  The warning notification system for users within 1km of a recently-reported incident.

• 

  The incident overview screen on the dashboard for emergency responders.

• 

  The incident view on the dashboard for emergency responders with live GPS tracking.

• 

  The incident view on the dashboard with key details/transcription from the audio.

• 

  The emergency contact alert system.

• 

  Our tech stack.

Inspiration

The inspiration behind Guardian AIngel came from both our personal and family experiences and the rise in violent hate crimes that we've seen on the news, particularly crimes that affect the most vulnerable populations. We hoped to create a solution that could be easily accessible and effective in mitigating these crimes by providing prompt help, but also to work toward the ultimate issue of inequity and inaccessibility in life-or-death situations.

What it does

Guardian AIngel is an AI-powered safety app providing users with fast response to emergency services, especially in precarious situations. Traditional safety apps rely on manual actions (calling 911, sending location), but in real emergencies, victims often can’t react fast enough. AI automates threat detection, escalation, and response before it’s too late. To detect threats, we use a mixture of audio, phone accelerometer data, and user input to automatically predict in advance.

How we built it

Using phone accelerometer data, our custom CNN can accurately detect if the user is experiencing irregular movements and automatically call emergency services. In addition, threats can be detected in advance using audio transcription

Ethical Considerations

Human interaction is crucial to Guardian AIngel's functionality, detecting threats based on the user's environment and allowing them to manually confirm their safety.

Without AI, real-time threat detection and automatic responses wouldn't be possible. The AI analyses both audio and motion data, something that human intervention alone cannot manage in a timely or efficient way.

We considered bias both in the systemic issues we aimed to address and in the AI models we used. We believe that both the physical and online safety of our users is important. So, we prioritized data privacy by encrypting and anonymizing user data and allowing only authenticated access to emergency personnel. In this way, we also hoped to prevent biases related to race, gender, and other factors, ultimately to address the issue of inequitable and disproportionate emergency response and care.

Challenges we ran into

Our group was quite new to React Native and mobile development as a whole. We spent a lot of time debugging the connection between our phones and the development server. We also had to learn various mobile development principles and concepts specific to react native.

One critical issue we had was detecting attacks accurately. At first, our logic was too lenient, and it eventually became too strict. For that reason, our group opted to create a custom convolutional neural network using TensorFlow to handle various cases that would be seen in real-life scenarios. We trained the neural network using the data from our phones’ accelerometers.

Another major issue we had was with the recording of audio from the user. We had to configure our database on the dashboard side to store the sequential audio segments, and we used OpenAI’s Whisper on the app’s side to transcribe and analyze the audio. However, we ran into many networking errors when attempting to upload the audio file from the mobile app to the database. We eventually fixed this issue by debugging and reconfiguring our API routes.

Accomplishments that we're proud of

Multiple times, we considered abandoning our mobile app due to a multitude of issues. We're proud of being able to stick through with mobile development to the end. In addition, we're also proud of our TensorFlow model, especially since the react-native TF library was deprecated, and there were tremendous difficulty getting it implemented in an app. Finally, we're so proud of being able to deliver a finished product, and showing that all our hard work was not for waste.

What we learned

Through NSBEHacks, we learned the value of the friends we made along the way (and the value of sleep, which we definitely didn't get).

What's next for Guardian AIngel

We plan to implement a route safety feature that uses local crime data and user reports to suggest safer routes to a destination and display a heatmap of the crimes. We also plan to implement more detailed analytics in our dashboard to better help emergency responders combat crime on a broader scale.

Built With

• expo.io
• google-colab
• leaflet.js
• next.js
• react-native
• sqlc
• supabase
• tailwindcss
• tensorflow