mirror of
https://github.com/pikvm/ustreamer.git
synced 2025-12-24 03:00:01 +00:00
238 lines
10 KiB
Markdown
238 lines
10 KiB
Markdown
# Streaming H.264 Video over WebRTC with Janus
|
||
|
||
µStreamer supports bandwidth-efficient streaming using [H.264 compression](https://en.wikipedia.org/wiki/Advanced_Video_Coding) and the Janus WebRTC server.
|
||
|
||
This guide explains how to configure µStreamer to provide an H.264 video stream and consume it within a web application using WebRTC.
|
||
|
||
## Components
|
||
|
||
In addition to µStreamer, H.264 streaming involves the following components:
|
||
|
||
- [**Janus**](https://janus.conf.meetecho.com/): A general-purpose WebRTC server.
|
||
- [**µStreamer Janus Plugin**](https://github.com/pikvm/ustreamer/tree/master/janus): A Janus plugin that allows Janus to consume a video stream from µStreamer via shared memory.
|
||
- [**Janus JavaScript Client**](https://janus.conf.meetecho.com/docs/JS.html): A frontend library that communicates with the Janus server and the µStreamer Janus plugin.
|
||
|
||
## Control Flow
|
||
|
||
To connect to a µStreamer video stream over WebRTC, an application must establish the following control flow:
|
||
|
||
1. The backend server starts the µStreamer service.
|
||
1. The backend server starts the Janus WebRTC server with the µStreamer Janus plugin.
|
||
1. The client-side JavaScript application establishes a connection to the Janus server.
|
||
1. The client-side JavaScript application instructs the Janus server to attach the µStreamer Janus plugin and start the video stream.
|
||
1. The client-side JavaScript application renders the video stream in the web browser.
|
||
|
||
## Server Setup
|
||
|
||
### Prerequisites
|
||
|
||
To integrate with Janus, µStreamer has the following prerequisites:
|
||
|
||
- The system packages that µStreamer depends on (see the [main build instructions](../README.md#building)).
|
||
- The Janus WebRTC server with WebSockets transport enabled (see the [Janus documentation](https://github.com/meetecho/janus-gateway)).
|
||
|
||
### Fixing Janus C Headers
|
||
|
||
To compile µStreamer with the Janus option, (see [“Installation”](#installation)), you need to make the Janus header files available within µStreamer's build context.
|
||
|
||
First, create a symbolic link from `/usr/include` to the Janus install directory. By default, Janus installs to `/opt/janus`.
|
||
|
||
```sh
|
||
ln -s /opt/janus/include/janus /usr/include/janus
|
||
```
|
||
|
||
Janus uses `#include` paths that make it difficult for third-party libraries to build against its headers. To fix this, modify the `#include` directive in `janus/plugins/plugin.h` to prepend a `../` to the included file name (`#include "refcount.h"` → `#include "../refcount.h"`).
|
||
|
||
```sh
|
||
sed \
|
||
--in-place \
|
||
--expression 's|^#include "refcount.h"$|#include "../refcount.h"|g' \
|
||
/usr/include/janus/plugins/plugin.h
|
||
```
|
||
|
||
### Installation
|
||
|
||
First, compile µStreamer with the Janus plugin option (`WITH_JANUS`).
|
||
|
||
```sh
|
||
git clone --depth=1 https://github.com/pikvm/ustreamer
|
||
cd ustreamer
|
||
make WITH_JANUS=1
|
||
```
|
||
|
||
The `WITH_JANUS` option compiles the µStreamer Janus plugin and outputs it to `janus/libjanus_ustreamer.so`. Move this file to the plugin directory of your Janus installation.
|
||
|
||
```sh
|
||
mv \
|
||
janus/libjanus_ustreamer.so \
|
||
/opt/janus/lib/janus/plugins/libjanus_ustreamer.so
|
||
```
|
||
|
||
Finally, specify a qualifier for the shared memory object so that the µStreamer Janus plugin can read µStreamer's video data.
|
||
|
||
```sh
|
||
cat > /opt/janus/lib/janus/configs/janus.plugin.ustreamer.jcfg <<EOF
|
||
memsink: {
|
||
object = "demo::ustreamer::h264"
|
||
}
|
||
EOF
|
||
```
|
||
|
||
If you're using a TC358743-based video capture device that supports audio capture, run the following command to enable audio streaming:
|
||
|
||
```sh
|
||
cat << EOF >> /opt/janus/lib/janus/configs/janus.plugin.ustreamer.jcfg
|
||
audio: {
|
||
device = "hw:1"
|
||
tc358743 = "/dev/video0"
|
||
}
|
||
EOF
|
||
```
|
||
|
||
### Start µStreamer and the Janus WebRTC Server
|
||
|
||
For µStreamer to share the video stream with the µStreamer Janus plugin, µStreamer must run with the following command-line flags:
|
||
|
||
- `--h264-sink` with the qualifier of the shared memory object you specified above (`demo::ustreamer::h264`)
|
||
- `--h264-sink-mode` with the permissions bitmask for the shared memory object (e.g., `660`)
|
||
- `--h264-sink-rm` to clean up the shared memory object when the µStreamer process exits
|
||
|
||
To load the µStreamer Janus plugin and configuration, the Janus WebRTC server must run with the following command-line flags:
|
||
|
||
- `--configs-folder` with the path to the Janus configuration directory (e.g., `/opt/janus/lib/janus/configs/`)
|
||
- `--plugins-folder` with the path to the Janus plugin directory (e.g., `/opt/janus/lib/janus/plugins/`)
|
||
|
||
## Client Setup
|
||
|
||
Once an application's backend server is running µStreamer and Janus, a browser-based JavaScript application can consume the server's video stream.
|
||
|
||
### Prerequisites
|
||
|
||
The client needs to load the following JavaScript libraries:
|
||
|
||
- [WebRTC Adapter library](https://webrtc.github.io/adapter/adapter-8.1.0.js) (`webrtc-adapter.js`, version `8.1.0`)
|
||
- [Janus Gateway JavaScript library](https://raw.githubusercontent.com/meetecho/janus-gateway/v1.0.0/html/janus.js) (`janus.js`, version `1.0.0`)
|
||
|
||
### Control Flow
|
||
|
||
The client-side JavaScript application uses the following control flow:
|
||
|
||
1. The client loads and initializes the Janus client library.
|
||
1. The client establishes a WebSockets connection to the Janus server.
|
||
1. The client instructs the Janus server to attach the µStreamer Janus plugin.
|
||
1. On success, the client obtains a plugin handle through which it can send requests directly to the µStreamer Janus plugin. The client processes responses via the `attach` callbacks:
|
||
- `onmessage` for general messages
|
||
- `onremotetrack` for the H.264 video stream and (optionally) an Opus audio stream
|
||
1. The client issues a `watch` request to the µStreamer Janus plugin, which initiates the H.264 stream in the plugin itself.
|
||
- It takes a few seconds for uStreamer's video stream to become available to Janus. The first `watch` request may fail, so the client must retry the `watch` request.
|
||
1. The client and server negotiate the underlying parameters of the WebRTC session. This procedure is called JavaScript Session Establishment Protocol (JSEP). The server makes a `jsepOffer` to the client, and the client responds with a `jsepAnswer`.
|
||
1. The client issues a `start` request to the µStreamer Janus plugin to indicate that the client wants to begin consuming the video stream.
|
||
1. The µStreamer Janus plugin delivers the H.264 video stream and (optionally) an Opus audio stream to the client via WebRTC.
|
||
1. The Janus client library invokes the `onremotetrack` callback with the video stream. The client attaches the video stream to the `<video>` element, rendering the video stream in the browser window.
|
||
1. (if an audio track is available) The Janus client library invokes the `onremotetrack` callback with the Opus audio stream. The client adds the audio stream to the `<video>` element, rendering the audio in the browser window.
|
||
|
||
### Sample Code
|
||
|
||
```html
|
||
<!DOCTYPE html>
|
||
<html>
|
||
<head>
|
||
<meta charset="utf-8">
|
||
<title>µStreamer H.264 demo</title>
|
||
<script src="https://webrtc.github.io/adapter/adapter-8.1.0.js"></script>
|
||
<!-- janus.js is the JavaScript client library of Janus, as specified above in
|
||
the prerequisites section of the client setup. You might need to change
|
||
the `src` path, depending on where you serve this file from. -->
|
||
<script src="janus.js"></script>
|
||
<style>
|
||
video {
|
||
height: 100%;
|
||
width: 100%;
|
||
object-fit: contain;
|
||
}
|
||
</style>
|
||
</head>
|
||
<body>
|
||
<video id="webrtc-output" autoplay playsinline muted></video>
|
||
<script type="text/javascript">
|
||
// Initialize Janus library.
|
||
Janus.init({
|
||
// Turn on debug logs in the browser console.
|
||
debug: true,
|
||
|
||
// Configure Janus to use standard browser APIs internally.
|
||
dependencies: Janus.useDefaultDependencies(),
|
||
});
|
||
|
||
// Establish a WebSockets connection to the server.
|
||
const janus = new Janus({
|
||
// Specify the URL of the Janus server’s WebSockets endpoint.
|
||
server: `ws://${window.location.hostname}:8188/`,
|
||
|
||
// Callback function if the client connects successfully.
|
||
success: attachUStreamerPlugin,
|
||
|
||
// Callback function if the client fails to connect.
|
||
error: console.error,
|
||
});
|
||
|
||
let uStreamerPluginHandle = null;
|
||
|
||
function attachUStreamerPlugin() {
|
||
// Instruct the server to attach the µStreamer Janus plugin.
|
||
janus.attach({
|
||
// Qualifier of the plugin.
|
||
plugin: "janus.plugin.ustreamer",
|
||
|
||
// Callback function, for when the server attached the plugin
|
||
// successfully.
|
||
success: function (pluginHandle) {
|
||
uStreamerPluginHandle = pluginHandle;
|
||
// Instruct the µStreamer Janus plugin to initiate streaming.
|
||
uStreamerPluginHandle.send({ message: { request: "watch", params: {audio: true} } });
|
||
},
|
||
|
||
// Callback function if the server fails to attach the plugin.
|
||
error: console.error,
|
||
|
||
// Callback function for processing messages from the Janus server.
|
||
onmessage: function (msg, jsepOffer) {
|
||
// If there is a JSEP offer, respond to it. This starts the WebRTC
|
||
// connection.
|
||
if (jsepOffer) {
|
||
uStreamerPluginHandle.createAnswer({
|
||
jsep: jsepOffer,
|
||
// Prevent the client from sending audio and video, as this would
|
||
// trigger a permission dialog in the browser.
|
||
media: { audioSend: false, videoSend: false },
|
||
success: function (jsepAnswer) {
|
||
uStreamerPluginHandle.send({
|
||
message: { request: "start" },
|
||
jsep: jsepAnswer,
|
||
});
|
||
},
|
||
error: console.error,
|
||
});
|
||
}
|
||
},
|
||
|
||
// Callback function, for when a media stream arrives.
|
||
onremotetrack: function (mediaStreamTrack, mediaId, isAdded) {
|
||
if (isAdded) {
|
||
// Attach the received media track to the video element. Cloning the
|
||
// mediaStreamTrack creates a new object with a distinct, globally
|
||
// unique stream identifier.
|
||
const videoElement = document.getElementById("webrtc-output");
|
||
if (videoElement.srcObject === null) {
|
||
videoElement.srcObject = new MediaStream();
|
||
}
|
||
videoElement.srcObject.addTrack(mediaStreamTrack.clone());
|
||
}
|
||
},
|
||
});
|
||
}
|
||
</script>
|
||
</body>
|
||
</html>
|
||
```
|