Understand Scribe v2 Realtime in a phone audio pipeline

Place recognition at the correct boundary

Automatic speech recognition converts incoming audio into text that another component can use. In Lupin’s current cascading gateway, Scribe v2 Realtime performs that role; it does not authorize workspace tools or choose which private account a caller may access. ElevenLabs also offers a batch Scribe model, but batch and realtime recognition have different interaction contracts. Name the realtime path explicitly in operational notes. A transcript that looks accurate after a call does not by itself prove that partial recognition arrived quickly enough for good conversational turn-taking.

Match the incoming telephone audio format

The checked gateway creates a realtime recognition URL with model_id scribe_v2_realtime and audio_format ulaw_8000. That setting describes the telephone input stream, not a recommended format for every microphone. The URL also requests language detection, manual commitment and disabled logging. If a transcription stream becomes unintelligible, inspect encoding, sample rate and frame continuity before changing the voice or conversation prompt. Supplying linear PCM bytes under a mu-law label can make an otherwise healthy provider connection produce meaningless text and misleading downstream answers.

Distinguish partial text from committed text

Partial recognition helps the gateway decide whether a caller appears to be continuing. It remains provisional until the turn is committed. Lupin uses gateway voice activity and a partial-text endpointing policy to determine that boundary rather than blindly treating every partial event as a new user message. A tutorial fixture should include a correction within one utterance so the final turn contains the intended wording. Starting workspace work on every partial fragment risks both incorrect context and unnecessary approval requests for a task the caller has not finished describing.

Understand the prepared recognizer lifecycle

The current transcription implementation uses a fresh recognizer for each utterance and prepares a ready spare. Its code documents an observed tendency for a session to retain its first language after a commit; the fresh-utterance design addresses that behavior while avoiding a new handshake on the critical path. Treat this as Lupin’s implementation choice and test evidence, not a claim about all provider sessions forever. Conversation and synthesis connections have separate lifecycles. Recycling recognition should not reset the entire task or grant the new session broader workspace authority.

Validate recognition independently of calling

Use synthetic audio fixtures containing known text, pauses, a proper noun and a deliberate language transition. Record expected final text, actual committed text and whether a partial result caused an early endpoint. Keep the test local or in an approved provider sandbox; no real phone number is required to inspect framing and lifecycle logic. Report recognition accuracy separately from response correctness. If the transcript is right but the answer is wrong, continue into conversation context and tool evidence instead of repeatedly tuning the recognizer for a problem outside its responsibility.

Check framing before tuning language

Feed a short synthetic mu-law fixture with a known phrase and verify its committed text. Repeat with deliberately mislabeled PCM bytes and assert a detectable failure or mismatch. This comparison establishes why encoding checks precede language tuning and prevents a decoder defect from being hidden behind repeated prompt changes.

Sources & verification

Reviewed against current checked-in Lupin routes, shared contracts and official provider references. No live calls, OAuth authorizations, app actions or purchases were performed.

• Lupin phone, SMS and calls ↗Checked 2026-11-05
• ElevenLabs realtime speech recognition ↗Checked 2026-11-05
• ElevenLabs model reference ↗Checked 2026-11-05

Prepared with AI assistance by Lupin editorial. How we research and maintain these guides.