Inspiration
SyntheSpace was born from a simple observation: the "mental translation" required when shopping online is exhausting. Whether it's a new sofa or a desk lamp, users often struggle to visualize how a 2D thumbnail will actually look in their 3D living room. Inspired by early virtual try-on apps for fashion, we wanted to bring that same "try-before-you-buy" confidence to the entire home furnishing and decor ecosystem. We envisioned a world where any product photo on the web could instantly become a physical presence in your room.
Additionally, we were inspired by the idea of the "Mind Palace" from Sherlock Holmes, the idea that you can envision a space in your mind and fully visualize it in reality. We wanted to create a tool that helps people do exactly that: take a vision for a space they have in their mind and then see exactly how it would look in a real environment.
What it does
SyntheShape is a sandbox that lets you create and design rooms/spaces and see how different objects appear in that space using AR. You can completely design a space from scratch, or base it off of an existing room you want to decorate or furnish. This allows you to visualize products before buying them, increasing customer satisfaction and lowering return rates. Additionally, it is also a fun tool users can use to exercise their creativity and design essentially any space they can imagine, and visualize in real space with AR.
SyntheShape works in 5 mains steps:
1. Scan Scan your environment to create a digital representation of your space as an empty room
2. Upload Upload a photo of any product (furniture, decorations, equipment, etc.)
3. Generate Our ML model creates a 3D asset that can be used instantly
4. Place Position the asset in your room however you like in our easy-to-use editor
5. Visualize See it in real-world context with WebXR - visualize the space you created in your own environment
Furthermore, we incorporated an AI agent that can help you build and design custom spaces. You can prompt it to create a specific type of space, or an idea you have for a room, and it will generate 3D assets for it and organize them in the room to fit the user specifications. Allows for users to quickly generate different types of rooms, or get ideas for a space they want to decorate.
How we built it
The project is built on a modern, distributed architecture:
- Frontend: A high-performance React 19 application built with Vite and TypeScript.
- 3D/AR Engine: We utilized
@react-three/fiberand@react-three/xrto build a declarative 3D scene. Hit-testing and plane detection allow for precise object placement. - Backend: A FastAPI (Python) server handles the heavy lifting of AI inference.
- AI Pipeline:
- Hunyuan3D (v2.1): Our core 2D-to-3D generation model.
- Gemini 2.0 Flash: Used for object detection and semantic categorization.
- Rembg: For pre-processing images to remove backgrounds, which significantly improves mesh quality.
- Math & Physics: To ensure accurate placement, we perform hit-testing where the intersection of a ray $R(t) = O + tD$ with a detected plane $P$ determines the world coordinates $P_{world}$.
The 3D generation also relies on octree resolutions $R = 2^n$, where we typically set $n=8$ for a resolution of 256 to balance detail and performance: $$ \text{Resolution} = 2^8 = 256 $$
Challenges we ran into
The journey wasn't linear; we pivoted a few times as we discovered the limitations of current WebXR implementations. One of the biggest technical hurdles was AR Object Placement. While detecting planes is standard, ensuring that a custom-generated AI model, which might have non-uniform bounds, sits naturally on a surface without "sinking" or "floating" required complex bounding box calculations and center-of-mass adjustments.
It was difficult to create the program that scans the room and generates a 3D model of the room within our editor. We had many design challenges on how we wanted it to work, and had to ultimately decide on the approach we found to be the most modular.
It was difficult to make the 3D editor seem seamless and easy to use. Oftentimes it would be frustrating to move assets in space, and it took a lot of fine tuning and reworking to get it to feel smooth and user friendly.
Accomplishments that we're proud of
We are proud we were able to put a full product together in the limited time we had. This was a very technical and challenging project, and we are proud of what we were able to deliver.
We are proud of the web app and how it looks. We like the aesthetic and theme, and think it’s a very marketable design.
We are proud of our effective use of ML and GenAI in the project. The 2D-3D image generation model is really cool, and the agent that searches the web for objects to add to the room is really awesome as well. It has a lot of business potential and we are excited about all the possible applications.
What we learned
AR is quite difficult to work with, and is very finicky. A lot of expertise and experience is required to make it work/look good, so we will definitely be looking into it more and hope to polish our skills and product.
We learned how to take a grand idea and orchestrate it into deliverables and doable tasks, making a big project like this one doable within 36 hours.
We learned a lot about using the Hunyuan3D-2 ML model for 3D asset generation, as well as using the Railtracks Agentic framework. It was a great experience for getting more familiar with Gen AI, and we all feel stronger working with this ever growing and important technology.
Lastly, we learned that it is important to dream big. It’s great to have grand and exciting ideas, and they are necessary to stand out and win a competition of this scale. However, deliverability is most important, and thus we had to cut some features and simplify some ideas to be able to have a fully deliverable project and working demo for the deadline. Functionality is key, and should be prioritized.
What's next for SyntheSpace
We are hoping to make a startup and market this product as a fully functional service, for both regular users and corporations. In particular, we believe home furnishing companies, such as Ikea, would greatly benefit from incorporating SyntheSpace. Giving users the ability to fully design rooms from scratch with exclusively Ikea products would be incredible, and allow people to achieve their visions while generating more sales and customer satisfaction for Ikea. This can be extended to online vendors in general, like Amazon, who sell decorations (posters, wall scrolls, lighting, etc.) and would benefit from users being able to test the products in their own homes before purchasing the product. Additionally, it’s just a fun tool people can use in their free time to design rooms, homes, and other spaces as a leisure activity. People enjoy designing homes and planning out their spaces, and we could see plenty of people finding enjoyment or entertainment in using SyntheSpace. All in all, we are very excited about our project, and look forward to making it a full-fledged product and starting a successful start-up around it.
Log in or sign up for Devpost to join the conversation.