mirror of
https://github.com/pikvm/ustreamer.git
synced 2026-02-19 08:16:31 +00:00
Merge pull request #150 from tiny-pilot/h264-guide
Add documentation for integrating with Janus to stream H.264 video
This commit is contained in:
Maxim Devaev
GitHub
@@ -99,6 +99,9 @@ $ modprobe bcm2835-v4l2 max_video_width=2592 max_video_height=1944
|
||||
-----
|
||||
# Integrations
|
||||
|
||||
## Janus
|
||||
µStreamer supports bandwidth-efficient streaming using [H.264 compression](https://en.wikipedia.org/wiki/Advanced_Video_Coding) and the Janus WebRTC server. See the [Janus integration guide](docs/h264.md) for full details.
|
||||
|
||||
## Nginx
|
||||
When uStreamer is behind an Nginx proxy, it's buffering behavior introduces latency into the video stream. It's possible to disable Nginx's buffering to eliminate the additional latency:
|
||||
|
||||
|
||||
231
docs/h264.md
Normal file
231
docs/h264.md
Normal file
@@ -0,0 +1,231 @@
|
||||
# Streaming H.264 Video over WebRTC with Janus
|
||||
|
||||
µStreamer supports bandwidth-efficient streaming using [H.264 compression](https://en.wikipedia.org/wiki/Advanced_Video_Coding) and the Janus WebRTC server.
|
||||
|
||||
This guide explains how to configure µStreamer to provide an H.264 video stream and consume it within a web application using WebRTC.
|
||||
|
||||
## Components
|
||||
|
||||
In addition to µStreamer, H.264 streaming involves the following components:
|
||||
|
||||
- [**Janus**](https://janus.conf.meetecho.com/): A general-purpose WebRTC server.
|
||||
- [**µStreamer Janus Plugin**](https://github.com/pikvm/ustreamer/tree/master/janus): A Janus plugin that allows Janus to consume a video stream from µStreamer via shared memory.
|
||||
- [**Janus JavaScript Client**](https://janus.conf.meetecho.com/docs/JS.html): A frontend library that communicates with the Janus server and the µStreamer Janus plugin.
|
||||
|
||||
## Control Flow
|
||||
|
||||
To connect to a µStreamer video stream over WebRTC, an application must establish the following control flow:
|
||||
|
||||
1. The backend server starts the µStreamer service.
|
||||
1. The backend server starts the Janus WebRTC server with the µStreamer Janus plugin.
|
||||
1. The client-side JavaScript application establishes a connection to the Janus server.
|
||||
1. The client-side JavaScript application instructs the Janus server to attach the µStreamer Janus plugin and start the video stream.
|
||||
1. The client-side JavaScript application renders the video stream in the web browser.
|
||||
|
||||
## Server Setup
|
||||
|
||||
### Prerequisites
|
||||
|
||||
To integrate with Janus, µStreamer has the following prerequisites:
|
||||
|
||||
- The system packages that µStreamer depends on (see the [main build instructions](../README.md#building)).
|
||||
- The Janus WebRTC server with WebSockets transport enabled (see the [Janus documentation](https://github.com/meetecho/janus-gateway)).
|
||||
|
||||
### Fixing Janus C Headers
|
||||
|
||||
To compile µStreamer with the Janus option, (see [“Installation”](#installation)), you need to make the Janus header files available within µStreamer's build context.
|
||||
|
||||
First, create a symbolic link from `/usr/include` to the Janus install directory. By default, Janus installs to `/opt/janus`.
|
||||
|
||||
```sh
|
||||
ln -s /opt/janus/include/janus /usr/include/janus
|
||||
```
|
||||
|
||||
Janus uses `#include` paths that make it difficult for third-party libraries to build against its headers. To fix this, modify the `#include` directive in `janus/plugins/plugin.h` to prepend a `../` to the included file name (`#include "refcount.h"` → `#include "../refcount.h"`).
|
||||
|
||||
```sh
|
||||
sed \
|
||||
--in-place \
|
||||
--expression 's|^#include "refcount.h"$|#include "../refcount.h"|g' \
|
||||
/usr/include/janus/plugins/plugin.h
|
||||
```
|
||||
|
||||
### Installation
|
||||
|
||||
First, compile µStreamer with the Janus plugin option (`WITH_JANUS`).
|
||||
|
||||
```sh
|
||||
git clone --depth=1 https://github.com/pikvm/ustreamer
|
||||
cd ustreamer
|
||||
make WITH_JANUS=1
|
||||
```
|
||||
|
||||
The `WITH_JANUS` option compiles the µStreamer Janus plugin and outputs it to `janus/libjanus_ustreamer.so`. Move this file to the plugin directory of your Janus installation.
|
||||
|
||||
```sh
|
||||
mv \
|
||||
janus/libjanus_ustreamer.so \
|
||||
/opt/janus/lib/janus/plugins/libjanus_ustreamer.so
|
||||
```
|
||||
|
||||
Finally, specify a qualifier for the shared memory object so that the µStreamer Janus plugin can read µStreamer's video data.
|
||||
|
||||
```sh
|
||||
cat > /opt/janus/lib/janus/configs/janus.plugin.ustreamer.jcfg <<EOF
|
||||
memsink: {
|
||||
object = "demo::ustreamer::h264"
|
||||
}
|
||||
EOF
|
||||
```
|
||||
|
||||
### Start µStreamer and the Janus WebRTC Server
|
||||
|
||||
For µStreamer to share the video stream with the µStreamer Janus plugin, µStreamer must run with the following command-line flags:
|
||||
|
||||
- `--h264-sink` with the qualifier of the shared memory object you specified above (`demo::ustreamer::h264`)
|
||||
- `--h264-sink-mode` with the permissions bitmask for the shared memory object (e.g., `660`)
|
||||
- `--h264-sink-rm` to clean up the shared memory object when the µStreamer process exits
|
||||
|
||||
To load the µStreamer Janus plugin and configuration, the Janus WebRTC server must run with the following command-line flags:
|
||||
|
||||
- `--configs-folder` with the path to the Janus configuration directory (e.g., `/opt/janus/lib/janus/configs/`)
|
||||
- `--plugins-folder` with the path to the Janus plugin directory (e.g., `/opt/janus/lib/janus/plugins/`)
|
||||
|
||||
## Client Setup
|
||||
|
||||
Once an application's backend server is running µStreamer and Janus, a browser-based JavaScript application can consume the server's video stream.
|
||||
|
||||
### Prerequisites
|
||||
|
||||
The client needs to load the following JavaScript libraries:
|
||||
|
||||
- [WebRTC Adapter library](https://webrtc.github.io/adapter/adapter-8.1.0.js) (`webrtc-adapter.js`, version `8.1.0`)
|
||||
- [Janus Gateway JavaScript library](https://raw.githubusercontent.com/meetecho/janus-gateway/v1.0.0/html/janus.js) (`janus.js`, version `1.0.0`)
|
||||
|
||||
### Control Flow
|
||||
|
||||
The client-side JavaScript application uses the following control flow:
|
||||
|
||||
1. The client loads and initializes the Janus client library.
|
||||
1. The client establishes a WebSockets connection to the Janus server.
|
||||
1. The client instructs the Janus server to attach the µStreamer Janus plugin.
|
||||
1. On success, the client obtains a plugin handle through which it can send requests directly to the µStreamer Janus plugin. The client processes responses via the `attach` callbacks:
|
||||
- `onmessage` for general messages
|
||||
- `onremotetrack` for the H.264 video stream
|
||||
1. The client issues a `watch` request to the µStreamer Janus plugin, which initiates the H.264 stream in the plugin itself.
|
||||
- It takes a few seconds for uStreamer's video stream to become available to Janus. The first `watch` request may fail, so the client must retry the `watch` request.
|
||||
1. The client and server negotiate the underlying parameters of the WebRTC session. This procedure is called JavaScript Session Establishment Protocol (JSEP). The server makes a `jsepOffer` to the client, and the client responds with a `jsepAnswer`.
|
||||
1. The client issues a `start` request to the µStreamer Janus plugin to indicate that the client wants to begin consuming the video stream.
|
||||
1. The µStreamer Janus plugin delivers the H.264 video stream to the client via WebRTC.
|
||||
1. The Janus client library invokes the `onremotetrack` callback. The client attaches the video stream to the `<video>` element, rendering the video stream in the browser window.
|
||||
|
||||
### Sample Code
|
||||
|
||||
```html
|
||||
<!DOCTYPE html>
|
||||
<html>
|
||||
<head>
|
||||
<meta charset="utf-8">
|
||||
<title>µStreamer H.264 demo</title>
|
||||
<script src="https://webrtc.github.io/adapter/adapter-8.1.0.js"></script>
|
||||
<!-- janus.js is the JavaScript client library of Janus, as specified above in
|
||||
the prerequisites section of the client setup. You might need to change
|
||||
the `src` path, depending on where you serve this file from. -->
|
||||
<script src="janus.js"></script>
|
||||
<style>
|
||||
video {
|
||||
height: 100%;
|
||||
width: 100%;
|
||||
object-fit: contain;
|
||||
}
|
||||
</style>
|
||||
</head>
|
||||
<body>
|
||||
<video id="webrtc-output" autoplay playsinline muted></video>
|
||||
<script type="text/javascript">
|
||||
// Initialize Janus library.
|
||||
Janus.init({
|
||||
// Turn on debug logs in the browser console.
|
||||
debug: true,
|
||||
|
||||
// Configure Janus to use standard browser APIs internally.
|
||||
dependencies: Janus.useDefaultDependencies(),
|
||||
});
|
||||
|
||||
// Establish a WebSockets connection to the server.
|
||||
const janus = new Janus({
|
||||
// Specify the URL of the Janus server’s WebSockets endpoint.
|
||||
server: `ws://${window.location.hostname}:8188/`,
|
||||
|
||||
// Callback function if the client connects successfully.
|
||||
success: attachUStreamerPlugin,
|
||||
|
||||
// Callback function if the client fails to connect.
|
||||
error: console.error,
|
||||
});
|
||||
|
||||
let uStreamerPluginHandle = null;
|
||||
|
||||
function attachUStreamerPlugin() {
|
||||
// Instruct the server to attach the µStreamer Janus plugin.
|
||||
janus.attach({
|
||||
// Qualifier of the plugin.
|
||||
plugin: "janus.plugin.ustreamer",
|
||||
|
||||
// Callback function, for when the server attached the plugin
|
||||
// successfully.
|
||||
success: function (pluginHandle) {
|
||||
uStreamerPluginHandle = pluginHandle;
|
||||
// Instruct the µStreamer Janus plugin to initiate the video stream.
|
||||
uStreamerPluginHandle.send({ message: { request: "watch" } });
|
||||
},
|
||||
|
||||
// Callback function if the server fails to attach the plugin.
|
||||
error: console.error,
|
||||
|
||||
// Callback function for processing messages from the Janus server.
|
||||
onmessage: function (msg, jsepOffer) {
|
||||
// 503 indicates that the plugin is not ready to stream yet. Retry the
|
||||
// watch request until the video stream is available.
|
||||
if (msg.error_code === 503) {
|
||||
uStreamerPluginHandle.send({ message: { request: "watch" } });
|
||||
return;
|
||||
}
|
||||
|
||||
// If there is a JSEP offer, respond to it. This starts the WebRTC
|
||||
// connection.
|
||||
if (jsepOffer) {
|
||||
uStreamerPluginHandle.createAnswer({
|
||||
jsep: jsepOffer,
|
||||
// Prevent the client from sending audio and video, as this would
|
||||
// trigger a permission dialog in the browser.
|
||||
media: { audioSend: false, videoSend: false },
|
||||
success: function (jsepAnswer) {
|
||||
uStreamerPluginHandle.send({
|
||||
message: { request: "start" },
|
||||
jsep: jsepAnswer,
|
||||
});
|
||||
},
|
||||
error: console.error,
|
||||
});
|
||||
}
|
||||
},
|
||||
|
||||
// Callback function, for when the video stream arrives.
|
||||
onremotetrack: function (mediaStreamTrack, mediaId, isAdded) {
|
||||
if (isAdded) {
|
||||
// Attach the received media track to the video element. Cloning the
|
||||
// mediaStreamTrack creates a new object with a distinct, globally
|
||||
// unique stream identifier.
|
||||
const videoElement = document.getElementById("webrtc-output");
|
||||
const stream = new MediaStream();
|
||||
stream.addTrack(mediaStreamTrack.clone());
|
||||
videoElement.srcObject = stream;
|
||||
}
|
||||
},
|
||||
});
|
||||
}
|
||||
</script>
|
||||
</body>
|
||||
</html>
|
||||
```
|
||||
Reference in New Issue
Block a user