AgenticMD | Devpost

Inspiration

The inspiration for AgentMD stems from a sobering reality in modern healthcare: the "pajama time" epidemic. Internal medicine doctors often spend hours after work finishing charts, manually cross-referencing patient histories with new symptoms. We realized that while AI has become incredible at processing data, it often lacks the clinical workflow integration required to be useful at the bedside. We wanted to build a tool that restores the "sacred space" of the doctor-patient relationship, removing the laptop as a physical and mental barrier.

What it does

AgentMD acts as an intelligent bridge between a live clinical conversation and a finalized treatment plan.

Ambient Intelligence: Using ultra-low latency transcription, it listens to the consultation and generates a high-fidelity medical transcript in real-time.
Contextual DDX: By pulling from the patient's historical records, it uses reasoning-focused AI to generate a Differential Diagnosis (DDX) that isn't just a list of possibilities, but a ranked set of hypotheses tailored to that specific individual.
Interactive Refinement: The physician interacts with the AI's output through a "Verification Layer," marking findings as true/false and accepting/rejecting recommended tests.
Plan Generation: Once the doctor has curated the logic, AgentMD synthesizes a comprehensive care plan and a structured clinical note.

How we built it

We built AgentMD using a robust, modern tech stack designed for speed, security, and clinical accuracy:

Transcription (ElevenLabs ScribeV2): We integrated ScribeV2 for its industry-leading medical accuracy. Its low-latency streaming allows it to capture complex drug names and anatomical terms without skipping a beat.
The Reasoning Engine (Gemini 3): We chose Gemini 3 for the core diagnostic engine. Its massive context window allows us to feed in the entire patient history alongside the live transcript. We use its reasoning capabilities to compute diagnostic confidence scores C for a specific diagnosis d: $$C(d) = \underset{i=1}{\overset{n}{\sum}}(w_i \cdot s_i) + \delta H$$ Where $s_i$ represents the presence of symptom $i$, $w_i$ is its clinical weight, and $\delta H$ is the historical coefficient derived from the patient's longitudinal records.
Backend (FastAPI & Supabase): The application logic is handled by FastAPI, providing a high-performance asynchronous bridge. We used Supabase for our database and authentication, leveraging its PostgreSQL foundation for secure data storage.
Frontend (AngularJS): To ensure a stable and performant dashboard, we utilized AngularJS. Its data-binding capabilities allowed us to build an interactive "Verification Layer" where doctors can toggle diagnoses in real-time.

Challenges we ran into

Integration & Schema Alignment: Synchronizing complex, nested JSON objects from Gemini 3 between our FastAPI backend and AngularJS frontend was difficult. We had to enforce strict data typing to ensure the UI didn't break when the AI returned non-standardized clinical terms.
Deployment & Scaling: Orchestrating a stack with real-time audio streaming (ElevenLabs) and heavy AI compute (Gemini 3) presented hurdles in managing environment variables and maintaining stable database connections via Supabase under high-frequency requests.
Clinical UI Design: Finding the balance between "data density" and "clinical speed" was a major challenge. We iterated through multiple designs to create a "one-tap" system that lets doctors verify AI results without losing focus on the patient.

Accomplishments that we're proud of

Solving a Real-World Problem: We moved beyond "AI for AI's sake" to address the actual administrative burnout affecting internal medicine physicians.
Defined Functional Requirements: We successfully translated vague clinical needs into a precise technical roadmap, ensuring every feature directly serves a diagnostic purpose.
Human-in-the-Loop Integration: Our "Accept/Reject" interface ensures the physician remains the ultimate authority, transforming the AI from a "black box" into a transparent assistant.
Rapid Execution: We are incredibly proud of building a fully functional, end-to-end medical platform—from ambient voice capture to structured plan generation—in an extremely short timeframe.

What we learned

We learned that UX is just as important as the Model. A doctor doesn't need a chatbot; they need a structured dashboard. We also gained deep insights into the nuances of medical ethics and the necessity of data privacy when handling ambient audio in a clinical setting.

What's next for AgenticMD

The future of AgentMD involves deeper integration with EMR (Electronic Medical Record) systems like Epic and Cerner to automate the "one-click" filing of notes. We are also looking into Multi-modal Diagnostics, where Gemini 3 can analyze uploaded images of rashes or X-rays alongside the conversation to provide an even more robust diagnostic picture.