A caller says “you charged me 10x.” What has to happen in the next second?
At Cresta’s Voice Mode On hackathon (SF Tech Week), Natali Wong & I built “Veronica,” the Anti-Boring Club’s concierge, on Cresta Conductor. Here’s one call, layer by layer, for builders & the voice-curious.

Inside Veronica: one call, ears → brain → hands → voice. Natali Wong & Robert Schwentker
📞 1. Line in
A Twilio number puts the agent on the phone network. When a call comes in, a webhook hands it to the agent, then the audio streams live both ways. - In a browser or app you’d use WebRTC, e.g. LiveKit.
🛠 Cresta says WebRTC can save up to 300ms vs a phone line. Handing off to a live human needs a SIP trunk into the contact center. (Today I learned there’s something called a CCaaS - Contact Center as a Service * Gerald insists it’s pronounced “kick-ass.”)
👂 2. Ears
Streaming speech-to-text (we picked Deepgram). The hard part isn’t the words. It’s knowing when you’ve stopped talking, and going quiet when you interrupt.
🛠 Endpointing + barge-in. Models like Deepgram Flux fold turn detection into transcription.
🧠 3. Brain
I described Veronica in plain English. Conductor (built on the Claude Agent SDK) asked questions, drafted a blueprint, then split the build into an umbrella agent with sub-agents: verify, find charges, refund, close.
🛠 It can test with simulated callers and evaluators before a human dials in.

Natali Wong’s unBoring Club · Prompt: Claude · Pixels: Gemini · Bell pressed by an unknown giant hand
🔧 4. Hands
Tools Conductor writes in Python. A rule from my agent-payments work: give the model verbs, not a pen.
🗣 5. Voice
ElevenLabs. Short turns: 3 sentences max, one question, then listen. (Cresta’s TTS blog says the same: 15 words a sentence, 3 sentences a turn.)

⏱ The whole loop
Per Cresta, pauses past ~300ms start to feel unnatural. So every layer streams into the next, overlapping instead of stacking.
🎭 Personality is a spec
Our first “Veronica” was polite and dull. We had to tell her it was a demo, not a bank. Then Gerald the llama accountant arrived. (Gerald approves every refund. The goat files them. Mostly.)

12 events in, one perfect night out. Jev by typesafe.ai, from Natali Wong’s app
What we didn’t finish: a pathUSD refund on Tempo testnet back to the buyer’s wallet (think Stripe Link), a text receipt, and an event picker.

Overcharged 10×? Refunded on the call. Planned Tempo testnet flow, by Robert Schwentker
Demos I loved:
🥇 Kiki (1st place): nail salon booking that switched English↔Spanish mid-call. Ears and voice both have to switch. (Turns out someone did expect the Spanish.)
🦷 Cresta Smile Dental (Most Creative): refills cancelled slots from a waitlist ranked by value, then texts.
🚗 WorthStop Concierge (Biggest Automation Impact): one call plans gas, dinner and parking on your route, matches card cashback, then texts the plan to Telegram.
🔁 Retainify: turns a return call into a retention offer.
🏠 Airbnb host agent: late-night check-in help, hands off to the host with context.
🎮 Austin & Thomas’s support agent: checks who you are before sharing order details.
🚰 Report It Now: a late-night maintenance line that logs the leak, notes the friendly dog for the technician, and won’t say who lives upstairs.

Team 3: Report It Now
Thank you Cresta, and Will Diamond, Tanya Batsenko, Lillian Zhao, Michael Kim for the highly unBoring hackathon!
(As “Veronica” would say: Gerald has reviewed this post and finds it suspiciously under budget.)
#VoiceAI #AIAgents #SFTechWeek
Sources:
Cresta blog: Crafting a Natural-Sounding AI Voice
bsky.app/profile/schwentker.sandboxlabs.ai/post/3mxfzfl5rwc2h
twitter.com/schwentker/status/2108397043488350523

photo near hackathon - as Carousel animals escape the Merry Go Round