Stream

Use the VoxML Stream verb to connect an answered call to a WebSocket service. It is intended for real-time transcription, conversational AI, quality monitoring, and other applications that need access to the call audio.

TryVox begins the stream while executing the call's VoxML response. There is currently no separate REST API for creating, listing, retrieving, or stopping stream objects on an already-running call.

Basic example

{
  "voxml_version": "1.0",
  "instructions": [
    {
      "verb": "Stream",
      "url": "wss://voice.example.com/calls",
      "track": "inbound_track",
      "parameters": {
        "agent_id": "support-assistant",
        "language": "en-IN"
      }
    }
  ]
}

Fields

outbound_track is not currently supported. TryVox returns a stream execution error instead of silently substituting another track.

Choosing a track

Use inbound_track for speech recognition or an AI assistant that should listen to the caller:

{
  "verb": "Stream",
  "url": "wss://voice.example.com/listen",
  "track": "inbound_track"
}

Use both_tracks for analytics, compliance, or transcription that needs the complete conversation:

{
  "verb": "Stream",
  "url": "wss://voice.example.com/analyse",
  "track": "both_tracks"
}

With both_tracks, the two directions are mixed into one mono channel. They are not delivered as separately labelled tracks.

WebSocket connection

Your server must:

• Accept a WebSocket client connection from TryVox.
• Accept the audio.drachtio.org WebSocket subprotocol.
• Handle one initial UTF-8 text frame followed by binary audio frames.
• Keep reads non-blocking and process audio continuously.

The connection sequence is:

1. TryVox opens the WebSocket.
2. TryVox sends the parameters object as a JSON text frame.
3. TryVox sends raw PCM audio in binary frames.
4. The connection closes when the call channel or stream ends.

For example, these parameters:

{
  "parameters": {
    "account_ref": "acct_42",
    "conversation_ref": "conv_91"
  }
}

arrive as the first text frame:

{
  "account_ref": "acct_42",
  "conversation_ref": "conv_91"
}

Do not treat the first text frame as audio.

Audio format

Audio sent from TryVox to your server uses:

The audio frames are binary WebSocket messages. They are not JSON objects and are not Base64 encoded.

If your speech or model provider expects 16 kHz audio, resample the PCM stream after receiving it. Do not reinterpret 8 kHz bytes as 16 kHz audio.

Authentication

The url may contain a short-lived, call-scoped token:

{
  "verb": "Stream",
  "url": "wss://voice.example.com/calls?token=eyJhbGciOi...",
  "track": "inbound_track"
}

Prefer an expiring token over a permanent API key. Validate it during the WebSocket upgrade and bind it to the expected account or call. Avoid placing reusable account credentials in parameters, because metadata travels inside the established WebSocket rather than authenticating the initial upgrade request.

Custom WebSocket request headers are not currently configurable through the Stream verb.

Stream termination

The WebSocket close event is the stream-level termination signal. Release model sessions, buffers, and temporary resources when it arrives.

For the authoritative call result, also configure the call's hangup_url. A stream can close because of a call hangup, a network failure, or media-server shutdown, so the close event alone does not contain the final call disposition.

TryVox does not currently emit separate Stream status callbacks or expose a Stream ID.

Sending audio toward the call

The same WebSocket can send audio back to the caller. Send a UTF-8 JSON text frame with type: "playAudio":

{
  "type": "playAudio",
  "data": {
    "audioContentType": "raw",
    "sampleRate": 8000,
    "audioContent": "BASE64_ENCODED_PCM_AUDIO"
  }
}

audioContent must be Base64-encoded mono signed linear PCM. For raw audio, sampleRate must be 8000 or 16000. You may instead send a complete WAV file by setting audioContentType to wave and Base64-encoding the file.

Send messages in playback order, as soon as each is ready — do not pace them to real time.

Consecutive messages are joined into a single playback, so audio that arrives while an earlier message is still playing continues from it without a seam. You do not need to buffer a whole utterance before sending it: a reply still being generated can go out in as many pieces as it arrives in. Verified on a live call, twelve consecutive 250 ms chunks of a continuous tone play as one unbroken three-second tone.

Two practical limits remain. Audio that arrives after a batch has started playing waits for that batch to finish, so there is still a boundary between batches — it is not a single continuous buffer you can write into at an arbitrary rate. And messages are dispatched through a bounded internal queue, so a sustained flood arriving faster than the call can play it will still shed audio rather than queue indefinitely.

Interrupting the agent (barge-in)

When the caller starts speaking over the agent, send a killAudio text frame:

{ "type": "killAudio" }

This stops audio that is currently playing and discards anything you have already sent that has not started yet. Without it, queued audio would begin playing the moment the current chunk was cut short and the caller would still be talked over.

Detecting the interruption is your side's job — you have the caller's audio in real time, so run your own voice-activity or turn detection and send killAudio as soon as you decide the caller has the floor.

Because sent audio can be cancelled this way, it is safe to send a reply as soon as it is ready rather than waiting to be certain the caller has finished.

checkpoint and stop are not defined. Use the WebSocket close event for stream cleanup and the call API or VoxML for call control.

AI assistant example

{
  "voxml_version": "1.0",
  "instructions": [
    {
      "verb": "Say",
      "text": "Please wait while I connect the assistant."
    },
    {
      "verb": "Stream",
      "url": "wss://assistant.example.com/media?session=short-lived-token",
      "track": "inbound_track",
      "parameters": {
        "assistant": "customer-care",
        "locale": "en-IN"
      }
    }
  ]
}

Operational guidance

• Use wss:// with a publicly trusted certificate in production.
• Authorize the WebSocket during the upgrade, before accepting audio.
• Expect the first frame to be text and all subsequent audio frames to be binary.
• Apply backpressure and keep per-frame processing short.
• Resample explicitly when downstream services require another sample rate.
• Treat WebSocket disconnects as normal cleanup events and use hangup_url for the final call status.
• Do not log access tokens or raw call audio unless your retention and consent policies allow it.

Current limitations

• Streams can only be started through VoxML.
• The input format is fixed to 8 kHz L16.
• Only inbound_track and mono-mixed both_tracks are supported.
• Stream IDs, REST lifecycle operations, reconnect controls, status callbacks, and custom upgrade headers are not exposed.
• Bidirectional playback supports playAudio; checkpoint, clear, and stop control events are not exposed.