RapidRecall | Devpost

Inspiration

When thinking of pressing social issues, we thought about issues that impact a large number of people. One of these issues that stood out to us was education. As students, we know that learning isn't always easy, and we've tried many ways to make it more enjoyable so it becomes more of a passion and less of a chore.

One effective way we've been able to make learning more engaging is through gamification. As students who were interested in exploring the abilities of VR, seeing a category open for "Best Gaming Hack" inspired us to create a way to use VR to make education more engaging and immersive.

What it does

RapidRecall is a web app + VR game that allows you take a photo of your class notes or lecture material and we use AI to automatically generate a multiple choice quiz within an immersive VR shooting game where you have to shoot down the wrong answers before running out of time.

How we built it

We built RapidRecall using Unity and a couple of the Meta Quest 3 headsets. We also integrated with the StrikerVR physical gun to add a physical aspect to the mixed reality experience. For the web app, we used Next.js and OpenAI's GPT-4o automatically extract information from the photo and generate the quiz questions. The Next.js app was hosted on Vercel and we used Neon for the PostgreSQL database.

Challenges we ran into

One of the challenges that we ran into was integrating the StrikerVR gun with our game. Since the StrikerVR gun is still very early in development, we couldn't find any public documentation for developers who wanted to build experiences with the VR gun. Instead, we had to read through hundreds of lines of sample code to understand how to use the SDK, which we then had to adapt to our use case. This took much longer than expected as we kept running into random connection issues and spent hours inserting debugging code to figure out why the gun wasn't shooting, but was finally able to get a working gun that shot projectiles.

Accomplishments that we're proud of

The biggest accomplishment that we're proud of was building a complete working mixed reality game from scratch despite our team having minimal prior experience with Unity (in fact, two of us had never used Unity before!) We were initially afraid that we wouldn't be able to build a working game in such a short amount of time, but thanks to various YouTube tutorials and ChatGPT, we were able to quickly figure things along the way and build a game that was far beyond any of our expectations.

What we learned

We learned how to build a complete VR game from scratch using Unity. We also learned how to integrate the StrikerVR gun, which we're hoping to continue to use in future experiences we build. We also learned how to use ChatGPT to learn and guide us as we learn on the fly despite having minimal/no prior experience.

What's next for RapidRecall

We thought of many ideas for how to make our VR education game more immersive, including effects like automatically fading out from virtual reality as you get more questions right or wrong. However, due to the time limit, we decided to keep our featureset fairly minimal and focused on completing the basic elements of a VR shooting game. However, we do hope to eventually experiment with adding more immersive aspects of VR into our game.