Signalling protocol

Overview

WebRTC uses RTCPeerConnection to communicate streaming data between
browsers (aka peers) but also needs a mechanism to coordinate
communication and to send control messages, a process known as
signalling.

We are using WebSocket signalling between WebRTC clients and the server.
WebSockets enables opening up a session from the client to the webserver
and then leaving it open for messages from both directions.

We are using the JSON messages format. To start signalling, clients need
to open a WebSocket connection with the server.

WebSocket URL format

wss://[hostname]/webrtc/[stream name]

Example

wss://ice.ceeblue.tv/webrtc/as+c94ca3ca-4f4b-415d-9467-7ae2fb9bf2d2

Requests

Common

Session Description Protocol (SDP offer)

Example:

{
  "type": "offer_sdp",
  "offer_sdp": "v=0\r\no=- 724438709080065200 2 IN IP4 127.0.0.1\r\n...a=rtpmap:123 ulpfec/90000\r\n"
}

Playback

Switch track

In the case of a multi quality stream, the user can select (switch) the
tracks (qualities) that he wants to receive.

Examples

{
  "type": "tracks",
  "video": 2
}

{
  "type": "tracks",
  "audio": 0
}

Streaming

Set video bitrate limit

It causes the server to send a WebRTC command telling the browser to
limit the max video bitrate to that value.

{
  "type": "video_bitrate",
  "video_bitrate": 1600000
}

Responses

Common

Session Description Protocol (SDP answer)

Example

{
  "type": "on_answer_sdp",
  "result": true,
  "answer_sdp": "v=0\r\no=- 1588582671271257 0 IN IP4 0.0.0.0\r\n...\r\na=end-of-candidates\r\n"
}

Streaming

Set video bitrate

Happens in case of successful setting of the video bitrate or change by
the server.

Example

{
  "result": true,
  "type": "on_video_bitrate",
  "video_bitrate": 2100000,
  "video_bitrate_constraint": 2100000
}

RTP props

On RTP settings change

Example

{
  "result": true,
  "type": "on_rtp_props",
  "drop": 30,
  "nack": 5
}

Media

Periodic report on the status of streaming quality stats.

Example

{
  "type": "on_media_receive",
  "millis": 13900,
  "stats": {
    "jitter_ms": 3.0552641779,
    "loss_num": 0,
    "loss_perc": 0.0000000000,
    "nack_num": 0
  },
  "tracks": [ "H264", "opus" ]
}

Playback

On time

Periodic report on the status of playing.

Example

{
  "type": "on_time",
  "begin": 2057477,
  "current": 2110997,
  "end": 2111317,
  "tracks": [
    1,
    6
  ]
}

On stop

This happens when a live stream ends

Example

{
  "type": "on_stop",
  "begin": 0,
  "current": 41084,
  "end": 0
}

Streaming overview

Server-side

Video bitrate

The server accepts a video bitrate video_bitrate (default value
6000000). The setting is used to notify a sender of the total estimated
available bit rate on the receiving side.

RTP properties

The server accepts drop and nack. These settings allow you to tune
how quickly the server will respond to network problems.

• drop - Amount of packet any track will wait for a packet to arrive
  before considering it loss (default 30)

• nack - Amount of packet any track will wait for a packet to arrive
  before NACKing it (default 5)

Error handling

In case of packet loss, the server reduces video bitrate
video_bitrate_constrain.

Once per second:

• > 50% loss, constrain previous value by 9%

• > 10% loss, constrain previous value by 5%

• > 5% loss, constrain previous value by 1%

• < 5% loss, do not constrain further

Client-side

The description of the code used on the demo push (streaming) page. It
consists of three modules that can be worked together.

Push page example: https://ice.ceeblue.tv/push.html

Signalling

The webrtc-streamer.js WebRTCStreamer class.

Responsible for the signalling protocol (communication with the server),
which is necessary for the initialization and control of the WebRTC
connection.

The constructor requires a server endpoint, a stream name, an access
token (optional).

Example

let signalling = new WebRTCStreamer('ice.ceeblue.tv:4433', 'as+083563ed-9bf9-4d5a-90b9-aee35afda3bf');

Events:

• started - on streaming started

• stopped - on streaming stopped

• error - on errors happened

• rtp_props - on RTP properties set

• stats - on server stats received

• video_bitrate - on video bitrate set/change

Example

signalling.addEventListener('started',   onStreamingStarted);
signalling.addEventListener('stopped',   onStreamingStopped);
signalling.addEventListener('error',     onStreamingError);
signalling.addEventListener('rtp_props', onRtpProps);

Methods:

• setMediaStream - set media stream for streaming

• setPeerConfiguration - set an RTCConfiguration dictionary
  providing options to configure the new connection

• start - start streaming

• stop - stop streaming

• setRTPProps(nack, drop) - set RTP properties

• setTargetVideoBitrate - set target video bitrate

• getStats(callback) - get peer connection statistics

Example

signalling.setPeerConfiguration({
  iceServers: [{
    urls: ['turn:....ceeblue.tv:3478?transport=tcp','turn:....ceeblue.tv:3478'],
    username: 'xxx', credential: 'yyy',
  }]
});
signalling.setMediaStream(stream); // Set media stream
signalling.start(); // Start streaming

Adaptive input manager

The webrtc-adaptive-input.js WebRTCAdaptiveInputManager class.

Responsible for video bitrate adapting to the variability of the
Internet bandwidth. The module is optional.

The constructor requires a signalling instance WebRTCStreamer.

Example

let adaptiveInput = new WebRTCAdaptiveInputManager(signalling);
adaptiveInput.setVideoBitrateLimit(3000000); // Maximum bitrate
adaptiveInput.setStartupVideoBitrate(3000000 * 0.7); // Let start from 2 Mbps

Default workflow

Manager collects server media stats, and monitors the value of lost
packets for the last 5 seconds. It also collects target video bitrate
for the last 60 seconds, when the stream was lossless.

When video bitrate less than the limit, then the manager checks the
possibility to recover bitrate to reach video bitrate limit. It happens
at intervals of 2,5 to 60 seconds, depending on the success of the
recovery attempt. The recovery interval is halved if the previous
attempt to increase bitrate was successful (min 2.5 seconds). The
recovery interval is doubled if the previous attempt to increase the
bitrate failed (maximum 60 seconds). The attempt is considered
unsuccessful if after increasing the bitrate there are packet losses.

On a streaming startup, the manager sets the target video bitrate value
to 2Mbit/s. (The limit is 3Mbit/s). If during the first 10 seconds of
streaming, there is a loss of packets, then the video bitrate is halved.

The manager increases the bitrate by 5% with each attempt, when target
bitrate is less than average and has not been packet loss in the last 5
seconds. It increases the target bitrate by 0.5% if the value is above
average bitrate (slow increase, to avoid bitrate change jumps).

If packet loss does not exceed 5%, the manager reduces the bitrate by
1%. (As we are approaching the bandwidth)

If the bitrate is less than 1.2Mbps, and the image size is about HD,
then the manager will halve the image size.

Metrics monitoring

The webrtc-monitoring.js WebRTCStatsMonitoring class.

esponsible for collecting peer statistics, etc, and sending it to the
Ceeblue server. The module is optional.

The constructor requires a signalling instance WebRTCStreamer.

Example

let monitoring = new WebRTCStatsMonitoring(signalling);

Playback overview

Client-side

The description of the code used on the demo playback page.

Play page example: https://ice.ceeblue.tv/play.html

Signalling

The webrtc-playback.js WebRTCPlayback class.

Responsible for the signalling protocol (communication with the server),
which is necessary for the initialization and control of the WebRTC
connection.

The constructor requires a server endpoint, a stream name, an access
token (optional).

Example

let signalling = new WebRTCPlayback('ice.ceeblue.tv:4433', 'as+083563ed-9bf9-4d5a-90b9-aee35afda3bf');

Events:

• connected - on signalling started

• started - on playback started

• stopped - on playback stopped

• time - on playback position update

• error - on errors happened

Example

signalling.addEventListener('started',   onPlaybackStarted);
signalling.addEventListener('stopped',   onPlaybackStopped);
signalling.addEventListener('error',     onPlaybackError);

Methods:

• setVideoElement - set video (DOM) element for image display

• setPeerConfiguration - set an RTCConfiguration dictionary
  providing options to configure the new connection

• start - start playback

• stop - stop playback

• audioTrack - set audio track index

• videoTrack - get video track index

Example

signalling.setPeerConfiguration({
  iceServers: [{
    urls: ['turn:....ceeblue.tv:3478?transport=tcp','turn:....ceeblue.tv:3478'],
    username: 'xxx', credential: 'yyy',
  }]
});
signalling.setVideoElement(stream); // Set video element
signalling.start(); // Start streaming