Artki.tech

AI-native interior design platform that learns your taste and generates personalized 3D room visualizations from products you could buy off of Amazon w/ recommendations.

Comment

Inspiration

The spark came during a late-night furniture shopping session. We watched a friend spend hours on Pinterest creating mood boards, then struggle to find matching furniture on multiple websites. Despite having great taste, they couldn't translate their vision into reality.

We realized this was a universal problem: the gap between inspiration and implementation in interior design. Nearly 98% of design enthusiasts are dissatisfied with their home decor, yet over $25 billion is spent on interior design each year in the U.S. While AI had revolutionized many creative fields, interior design remained fragmented across inspiration platforms, furniture retailers, and 3D planning tools.

Our breakthrough insight: what if we could learn someone's aesthetic preferences the same way Spotify learns musical taste - not through questionnaires, but through choices? Museum artworks became our "songs" - universally recognized pieces that reveal deep aesthetic preferences through simple A/B comparisons.

What It Does

Artki.tech is an AI-powered interior design platform that creates personalized 3D room visualizations with real, purchasable furniture based on your unique taste profile. Here's the complete flow:

  1. Floorplan Analysis: Users upload their room's floorplan, which gets transformed into a 3D room reconstruction using Claude-generated Three.js code.

  2. Taste Discovery: Users compare 12 pairs of museum artworks (Monet vs. Picasso, Van Gogh vs. Warhol). Each choice updates a 512-dimensional taste vector using CLIP embeddings, learning aesthetic preferences without questionnaires.

  3. Real Product Discovery: Our system scrapes Amazon to find actual furniture products matching your taste profile, ensuring everything you see can be purchased.

  4. AI-Powered 3D Pipeline:

    • Amazon products are enhanced with OpenAI-generated images
    • Images are converted to 3D models using Claude's Three.js generation
    • Models are rendered in your reconstructed room

  5. Interactive Visualization: Real-time Three.js rendering shows actual purchasable furniture in your specific room layout. Users can rotate, zoom, and rearrange items to perfect their design.

  6. Smart Recommendations: ChromaDB's vector search finds semantically similar products based on visual features, style tags, and your evolving preferences.

Architecture Overview

  • Frontend: Next.js 16 + React 19 + TypeScript + Three.js + React Three Fiber
  • Backend: FastAPI + Python 3.11
  • AI/ML: Claude 4.5 Sonnet + CLIP embeddings + OpenAI API + NeRF
  • Database: ChromaDB Cloud for vectors
  • 3D: Three.js + React Three Fiber + NeRF reconstruction
  • Data Source: Amazon product scraping for real furniture

Development Process & Technical Evolution

Initial Approach: NeRF Neural Radiance Fields

  • We started by training NeRF (Neural Radiance Fields) models to create photorealistic 3D reconstructions of rooms from just a few photos
  • Built a Flask API server (api_server.py) to handle NeRF training with user-uploaded images
  • Successfully generated 3D room reconstructions from 10+ photos per room
  • Pivot Decision: While NeRF worked, the output had low fidelity and training took 8-10 minutes per room

Day 1: Foundation & Pivot

  • Set up Next.js frontend and FastAPI backend
  • Integrated Three.js for 3D visualization
  • Key Pivot: Switched from NeRF to Claude API for faster, higher-quality 3D generation
  • Claude generates Three.js code directly, which we render with React Three Fiber
  • Reduced generation time from 30 minutes to 3 seconds

Day 2: AI Integration & Product Pipeline

  • Implemented CLIP embeddings for artwork analysis
  • Built taste vector algorithm for preference learning
  • Amazon Integration: Created web scraping system to find real products matching user preferences
  • Image-to-3D Pipeline:
    1. Scrape Amazon for furniture matching user taste
    2. Use OpenAI API to generate product images
    3. Convert images to 3D models using Claude
    4. Render models in Three.js scene

Day 3: Complete System Integration

  • Connected floorplan upload to room reconstruction
  • Integrated user preference embeddings with product search
  • Built complete pipeline: Floorplan → Room Reconstruction → Preference Analysis → Amazon Scraping → Image Generation → 3D Conversion → Final Visualization
  • Polished UI/UX with TailwindCSS

Complete Technical Pipeline

  1. Room Reconstruction: User uploads floorplan → Claude generates Three.js room geometry from floorplan specifications

  2. Taste Learning: CLIP encodes museum artworks into 512-dimensional vectors. User choices update their vector: user_vec = user_vec + win_vec - 0.5 * lose_vec

  3. Product Discovery:

    • Scrape Amazon for furniture matching taste vector
    • Extract product metadata (price, dimensions, materials)
    • Generate embeddings for semantic search

  4. 3D Model Generation:

    • OpenAI API transforms product descriptions into detailed images
    • Images fed to Claude for Three.js code generation
    • Real-time rendering of actual products as 3D models

  5. Final Visualization: Combine room reconstruction with generated furniture models in interactive 3D scene

Challenges We Ran Into

1. NeRF Training Performance

Problem: NeRF neural radiance field training took 15-30 minutes per room with low-fidelity output. Solution: Pivoted to Claude API for instant Three.js code generation, reducing time from 30 minutes to 3 seconds while improving quality.

2. Amazon Product Integration

Problem: Connecting abstract taste preferences to real, purchasable products. Solution: Built web scraper for Amazon products, then used embeddings to match products to user taste vectors.

3. Image-to-3D Conversion Pipeline

Problem: No direct way to convert Amazon product images to 3D models. Solution: Created multi-step pipeline: Amazon data → OpenAI image enhancement → Claude 3D code generation → Three.js rendering.

4. 3D Model Loading Issues

Problem: GLTF files referenced external textures, causing 404 errors. Solution: Converted to self-contained GLB format with embedded textures.

5. Embedding Performance

Problem: Generating CLIP embeddings for scraped products took 30+ seconds. Solution: Pre-computed embeddings stored in ChromaDB, reducing search to <100ms.

6. AI Code Generation Reliability

Problem: Claude sometimes generated invalid Three.js code for complex furniture. Solution: Implemented robust error handling with fallback hand-crafted generators for common furniture types.

7. Floorplan to 3D Reconstruction

Problem: Converting 2D floorplan images to accurate 3D room geometry. Solution: Used Claude to interpret floorplan and generate proportional Three.js room structures.

Accomplishments That We're Proud Of

  1. Successfully Pivoted from NeRF: Started with neural radiance fields, recognized limitations, and pivoted to a better solution within 24 hours

  2. End-to-End Pipeline: Built complete pipeline from floorplan upload to 3D room with real Amazon products - a truly functional prototype

  3. Multi-AI Orchestration: Successfully integrated 5 different AI systems (Claude, CLIP, OpenAI, ChromaDB, NeRF) into one seamless experience

  4. Real Product Integration: Connected abstract preferences to actual purchasable Amazon products, solving a real-world problem

  5. Real-time 3D Generation: Reduced 3D generation from 30 minutes (NeRF) to 3 seconds (Claude) while improving quality

  6. Lightning-Fast Search: <100ms semantic search across scraped products with 512-dimensional vectors

  7. Production-Ready Architecture: Built scalable, well-documented codebase that could be deployed tomorrow

What We Learned

Technical Insights

  • Multi-modal AI is powerful: Combining visual (CLIP) and language (Claude) AI creates emergent capabilities
  • Vector databases are game-changers: ChromaDB enabled instant personalized search
  • Prompt engineering matters: Structured prompts improved Claude's code generation by 70%
  • Fallbacks are essential: Every AI component needs a reliable backup

Product Insights

  • Users want simplicity: 12 comparisons hit the sweet spot between accuracy and user patience
  • Visual choices reveal preferences: People make faster, more confident decisions with images than questionnaires
  • 3D visualization sells: Seeing furniture in context dramatically improves user confidence

Team Insights

  • API integration complexity: Coordinating multiple external APIs requires careful error handling
  • Performance optimization is iterative: Each bottleneck revealed led to architectural improvements

What's Next for Artki.tech

Immediate Goals (Next Month)

  1. Real Furniture Integration: Partner with IKEA, Wayfair, West Elm for actual purchasable items
  2. Mobile AR Preview: Use ARKit/ARCore for in-room visualization
  3. Expand Catalog: 500+ furniture items with real product links

Business Model

  • Freemium: Free taste profiling + 3 room designs
  • Pro Subscription: Unlimited designs, high-res exports, AR preview
  • Affiliate Revenue: Commission from furniture purchases
  • Enterprise API: White-label solution for furniture retailers

Built With

  • amazon-web-scraping
  • anthropic-sdk
  • chromadb
  • chromadb-cloud
  • claude-api
  • clip
  • embeddings
  • fastapi
  • flask
  • framer-motion
  • glb
  • gltf
  • javascript
  • lucide-react
  • nerf
  • neural-radiance-fields
  • next-js
  • openai-api
  • python
  • react
  • react-three-fiber
  • sentence-transformers
  • tailwindcss
  • three-js
  • typescript
  • vector-database
  • vercel
  • webgl
Share this project:

Updates