JETZT Voice Assistant
A production inbound voice AI agent for JETZT: Telnyx media streaming into Deepgram STT, a LangGraph/GPT-4o reasoning layer with live tool access, and Deepgram TTS streamed back to the caller sentence-by-sentence for low-latency conversation.
Caller | Telnyx (media stream) | Deepgram STT | LangGraph + GPT-4o |-- Knowledge lookup |-- CRM lead lookup / update |-- Calendly book / reschedule / cancel |-- Message taking | Deepgram TTS (streamed sentence-by-sentence) | Caller
Problem
JETZT needed a phone agent that could answer inbound calls, hold a natural conversation, and take real actions — book, reschedule, or cancel a Calendly meeting, look up or update a lead, or take a message — without callers noticing a full LLM round-trip on every turn.
Architecture
Telnyx streams call audio over WebSockets into a Deepgram STT session. Transcribed turns feed a LangGraph state machine wrapping GPT-4o, which holds conversation state and calls tools mid-conversation. Responses are streamed to Deepgram TTS sentence-by-sentence rather than waiting for the full completion, then piped back into the Telnyx media stream.
My Contribution
Designed and built the full pipeline solo — the media-streaming layer, the LangGraph tool-calling agent, and the sentence-level TTS streaming that keeps latency conversational. Diagnosed and fixed a string of production-only failures around Vercel serverless execution limits, WebSocket lifecycle, Deepgram SDK upgrades, recording auth, and STT cold-start.
Outcome
Runs in production at JETZT handling real inbound calls with conversational, low-latency responses.