Receive captions via a WebSocket connection – CaptionHub

When a stream is running you can receive new captions via a WebSocket connection.

Workflow

Create a new project with SRT or RTMP source

Select the WebSocket API output

Under Latency configuration, set the Maximum target latency for your captions. A minimum of 5 seconds is recommended.

Use our SDK to subscribe to new captions

Maximum target latency

The Maximum target latency setting in a WebSocket project determines the maximum latency of your captions. If you’re delaying your broadcast by 10 seconds, your maximum latency should be 10 seconds. Setting a maximum latency doesn’t mean that captions always arrive after 10s - we send them once finished.

The primary trade-off here is between speed and quality: the lower the latency, the less time there is for Timbra to run transcription and natural captions.

Keeping sync

The WebSocket API messages contain the captions. Depending on the source, it contains 2 or 3 pairs of timestamps.

Transcript time (start_time and end_time) is based on the audio you’ve sent us. If you’ve sent us 10 seconds of audio, a caption that starts at 10.5 seconds will have a start_time of 10.5 seconds - even if 15 seconds have elapsed since the start.

Approximate wall-clock capture time (capture_timestamp, with its own start_time and end_time). This is the time of the caption based on when it was received. Useful to determine when something was said, and to understand the latency between you and Timbra.

SEI timestamp (sei_timestamp). If an SEI timestamp is present in the source stream, we’ll expose it as well. This will allow you to sync captions to your internal workflow.

For the full schema, check out CaptionsEvent in the SDK docs.

Message delivery

Timbra uses Pusher for highly available message delivery. A dedicated channel is created for your project, and you can subscribe to it using the CaptionHub SDK (NodeJS), any of Pusher’s client SDKs, or implement your own.

Example code

The following is an example of how to create a project, start the flow, and listen to WebSocket messages, using the captionhub-node-sdk (TypeScript):

import { CaptionHub } from '@captionhub/captionhub-node-sdk';

const captionhub = new CaptionHub("YOUR_API_KEY");

const flow = await captionhub.timbra.createFlow({
  name: `My first SRT project`,
  source: 'srt_push',
  output: 'websocket',
});

await captionhub.timbra.startFlow(flow.flow_id);

// Start streaming to `flow.srt_details.full_url`

const connection = await captionhub.timbra.subscribe({
  flowId: flow.flow_id,
  onCaption: (event) => {
    console.log("Received captions:", event);
  }
});