Add captions to a low latency video stream – CaptionHub

☝

Contact your account manager to enable this feature

The Timbra plugin normally reads the HLS timeline and shows each caption perfectly in sync with the audio. A WebRTC, MOQ or LL-HLS stream is only a second or two behind the speaker, so there is nothing to wait for and nothing to synchronise against.

A low latency flow captions those streams instead. Your encoder sends the same feed to Timbra and to your own delivery service (IVS, Dolby). Timbra transcribes and translates it as it arrives and publishes to the plugin, and the plugin displays the words into your player as they are spoken. This style of captions is sometimes called “roll-up” because the each line rolls up when filled.

How it works

Timbra sits beside your delivery rather than inside it, so it adds no latency. Words reach the player a few seconds after they are spoken. Translations need to wait for a sentence to finish, so they’re delayed a little more.

Captions roll up rather than arriving as finished blocks. Words fill a line, a full line pushes the earlier lines up, and lines beyond the third scroll off the top. The plugin spaces the words out at the rhythm they were spoken in, and closes the gap when a backlog builds.

What you need

An SRT or RTMP source. See Stream to Timbra using RTMP or SRT.

Video delivery of your own that is genuinely low latency: WebRTC, MOQ or LL-HLS.

A player the plugin supports, or a plain HTML5 video element.

Setting it up

Create a live project with an SRT or RTMP input and the "Player plugin" output.

Open the "Plugin integration" tab and copy the captions URL.

Add the plugin to the page with your video.

For a plain video element, use the HTML5 plugin and pass it the element your player draws into:

import { Timbra } from '@captionhub/timbra.js';

const video = document.getElementById('my-player');

const timbra = new Timbra.HTMLVideoPlugin({
  video: video,
  captionsURL: "<<captions URL>>"
});

timbra.startCaptions();

If your video runs through Video.js, HLS.js, OptiView or the Mux player, use that player's plugin instead. The captions roll up the same way.

What is different from a HLS flow

Captions follow the speaker rather than the picture. If your video runs more than a second or two behind live, the captions will be ahead of it.

Caption position and styling are fixed. Roll-up captions set their own layout of three lines.

Audio events, such as a note that music is playing, are not shown.