mirror of
https://github.com/pikvm/ustreamer.git
synced 2025-12-23 18:50:00 +00:00
Tweak language in H264 guide (#3)
* Adjust wording in uStreamer Janus guide * Small tweaks * Small changes * Updating based on review notes
This commit is contained in:
Michael Lynch
GitHub
parent
a8dfa96db0
commit
f0763e3865
@ -99,6 +99,9 @@ $ modprobe bcm2835-v4l2 max_video_width=2592 max_video_height=1944
|
||||
-----
|
||||
# Integrations
|
||||
|
||||
## Janus
|
||||
µStreamer supports bandwidth-efficient streaming using [H.264 compression](https://en.wikipedia.org/wiki/Advanced_Video_Coding) and the Janus WebRTC server. See the [Janus integration guide](docs/h264.md) for full details.
|
||||
|
||||
## Nginx
|
||||
When uStreamer is behind an Nginx proxy, it's buffering behavior introduces latency into the video stream. It's possible to disable Nginx's buffering to eliminate the additional latency:
|
||||
|
||||
|
||||
134
docs/h264.md
134
docs/h264.md
@ -1,42 +1,53 @@
|
||||
# Guide to H.264 streaming
|
||||
# Streaming H.264 Video over WebRTC with Janus
|
||||
|
||||
µStreamer supports bandwidth-efficient streaming using [H.264 compression](https://en.wikipedia.org/wiki/Advanced_Video_Coding) and the Janus WebRTC server. This guide explains how to configure the V4L2 device to provide a H.264 video stream via WebRTC, and how to consume that stream within a web application.
|
||||
µStreamer supports bandwidth-efficient streaming using [H.264 compression](https://en.wikipedia.org/wiki/Advanced_Video_Coding) and the Janus WebRTC server.
|
||||
|
||||
## Components and Control Flow
|
||||
This guide explains how to configure µStreamer to provide an H.264 video stream and consume it within a web application using WebRTC.
|
||||
|
||||
In addition to µStreamer itself, the following components are involved:
|
||||
## Components
|
||||
|
||||
- [**Janus**](https://janus.conf.meetecho.com/): A general-purpose WebRTC server. Its purpose is to facilitate the WebRTC connection and communication, but it doesn’t know anything about µStreamer or other media suppliers.
|
||||
- [**µStreamer Janus Plugin**](https://github.com/pikvm/ustreamer/tree/master/janus): A Janus plugin to link µStreamer and Janus. It provides the video data via shared memory from µStreamer.
|
||||
- [**Janus JavaScript-Client**](https://janus.conf.meetecho.com/docs/JS.html): A frontend library for connecting to the Janus server, and to exchange commands and data with the µStreamer Janus plugin.
|
||||
In addition to µStreamer, H.264 streaming involves the following components:
|
||||
|
||||
This is a high-level overview of the control flow:
|
||||
- [**Janus**](https://janus.conf.meetecho.com/): A general-purpose WebRTC server.
|
||||
- [**µStreamer Janus Plugin**](https://github.com/pikvm/ustreamer/tree/master/janus): A Janus plugin that allows Janus to consume a video stream from µStreamer via shared memory.
|
||||
- [**Janus JavaScript Client**](https://janus.conf.meetecho.com/docs/JS.html): A frontend library that communicates with the Janus server and the µStreamer Janus plugin.
|
||||
|
||||
1. The V4L2 device starts the µStreamer service.
|
||||
1. The V4L2 device starts the Janus WebRTC server with the µStreamer Janus plugin.
|
||||
## Control Flow
|
||||
|
||||
To connect to a µStreamer video stream over WebRTC, an application must establish the following control flow:
|
||||
|
||||
1. The backend server starts the µStreamer service.
|
||||
1. The backend server starts the Janus WebRTC server with the µStreamer Janus plugin.
|
||||
1. The client-side JavaScript application establishes a connection to the Janus server.
|
||||
1. The client-side JavaScript application instructs the Janus WebRTC server to attach the µStreamer Janus plugin and to start the video stream.
|
||||
1. The client-side JavaScript application instructs the Janus server to attach the µStreamer Janus plugin and start the video stream.
|
||||
1. The client-side JavaScript application renders the video stream in the web browser.
|
||||
|
||||
## Server Setup
|
||||
|
||||
### Prerequisites
|
||||
|
||||
The following prerequisites need to be installed on the server:
|
||||
To integrate with Janus, µStreamer has the following prerequisites:
|
||||
|
||||
- The system packages that µStreamer depends on; see [µStreamer documentation](https://github.com/pikvm/ustreamer/blob/master/README.md#building).
|
||||
- The Janus WebRTC server, with WebSocket transport enabled; see [Janus documentation](https://github.com/meetecho/janus-gateway).
|
||||
- The system packages that µStreamer depends on (see the [main build instructions](../README.md#building)).
|
||||
- The Janus WebRTC server with WebSockets transport enabled (see the [Janus documentation](https://github.com/meetecho/janus-gateway)).
|
||||
|
||||
For compiling µStreamer (see section “Installation”), the Janus header files have to be available to the C compiler. By default, the base path of Janus is `/opt/janus`.
|
||||
### Fixing Janus C Headers
|
||||
|
||||
To compile µStreamer with the Janus option, (see [“Installation”](#installation)), you need to make the Janus header files available within µStreamer's build context.
|
||||
|
||||
First, create a symbolic link from `/usr/include` to the Janus install directory. By default, Janus installs to `/opt/janus`.
|
||||
|
||||
```sh
|
||||
ln -s /opt/janus/include/janus /usr/include/janus
|
||||
```
|
||||
|
||||
You might encounter a compiler error raised in `janus/plugins/plugin.h` that reports a missing `refcount.h` header file. To fix this, modify the respective `#include` directive in the `janus/plugins/plugin.h` file and prepend a `../` to the included file name (`#include "refcount.h"` → `#include "../refcount.h"`).
|
||||
Janus uses `#include` paths that make it difficult for third-party libraries to build against its headers. To fix this, modify the `#include` directive in `janus/plugins/plugin.h` to prepend a `../` to the included file name (`#include "refcount.h"` → `#include "../refcount.h"`).
|
||||
|
||||
```sh
|
||||
cd /usr/include/janus/plugins
|
||||
sed -i -e 's|^#include "refcount.h"$|#include "../refcount.h"|g' plugin.h
|
||||
sed \
|
||||
--in-place \
|
||||
--expression 's|^#include "refcount.h"$|#include "../refcount.h"|g' \
|
||||
/usr/include/janus/plugins/plugin.h
|
||||
```
|
||||
|
||||
### Installation
|
||||
@ -49,13 +60,15 @@ cd ustreamer
|
||||
make WITH_JANUS=1
|
||||
```
|
||||
|
||||
Next, move the compiled `janus/libjanus_ustreamer.so` shared library file to the plugin directory of your Janus installation.
|
||||
The `WITH_JANUS` option compiles the µStreamer Janus plugin and outputs it to `janus/libjanus_ustreamer.so`. Move this file to the plugin directory of your Janus installation.
|
||||
|
||||
```sh
|
||||
mv janus/libjanus_ustreamer.so /opt/janus/lib/janus/plugins/libjanus_ustreamer.so
|
||||
mv \
|
||||
janus/libjanus_ustreamer.so \
|
||||
/opt/janus/lib/janus/plugins/libjanus_ustreamer.so
|
||||
```
|
||||
|
||||
Finally, specify a qualifier for the shared memory object, so that the µStreamer Janus plugin can read the video data of µStreamer.
|
||||
Finally, specify a qualifier for the shared memory object so that the µStreamer Janus plugin can read µStreamer's video data.
|
||||
|
||||
```sh
|
||||
cat > /opt/janus/lib/janus/configs/janus.plugin.ustreamer.jcfg <<EOF
|
||||
@ -65,43 +78,46 @@ memsink: {
|
||||
EOF
|
||||
```
|
||||
|
||||
### Start Up µStreamer and the Janus WebRTC Server
|
||||
### Start µStreamer and the Janus WebRTC Server
|
||||
|
||||
µStreamer needs to be started with the following additional options, so that it can share memory with the µStreamer Janus plugin:
|
||||
For µStreamer to share the video stream with the µStreamer Janus plugin, µStreamer must run with the following command-line flags:
|
||||
|
||||
- `--h264-sink` with the qualifier of the shared memory object, in our case: `demo::ustreamer::h264`
|
||||
- `--h264-sink-mode` with the permissions bitmask for the shared memory object, e.g. `660`.
|
||||
- `--h264-sink-rm` for cleaning up the shared memory object on stop.
|
||||
- `--h264-sink` with the qualifier of the shared memory object you specified above (`demo::ustreamer::h264`)
|
||||
- `--h264-sink-mode` with the permissions bitmask for the shared memory object (e.g., `660`)
|
||||
- `--h264-sink-rm` to clean up the shared memory object when the µStreamer process exits
|
||||
|
||||
The Janus WebRTC server needs to be started with the following additional options, so that it loads the µStreamer Janus plugin and configuration:
|
||||
To load the µStreamer Janus plugin and configuration, the Janus WebRTC server must run with the following command-line flags:
|
||||
|
||||
- `--configs-folder` with the path to the Janus configuration directory, for example: `/opt/janus/lib/janus/configs/`
|
||||
- `--plugins-folder` with the path to the Janus plugin directory, for example: `/opt/janus/lib/janus/plugins/`
|
||||
- `--configs-folder` with the path to the Janus configuration directory (e.g., `/opt/janus/lib/janus/configs/`)
|
||||
- `--plugins-folder` with the path to the Janus plugin directory (e.g., `/opt/janus/lib/janus/plugins/`)
|
||||
|
||||
## Client Setup
|
||||
|
||||
Once an application's backend server is running µStreamer and Janus, a browser-based JavaScript application can consume the server's video stream.
|
||||
|
||||
### Prerequisites
|
||||
|
||||
The client needs to load the following JavaScript libraries:
|
||||
|
||||
- [The WebRTC Adapter library](https://webrtc.github.io/adapter/adapter-8.1.0.js) (`webrtc-adapter.js`, version `8.1.0`)
|
||||
- [The JavaScript library of Janus Gateway](https://raw.githubusercontent.com/meetecho/janus-gateway/v1.0.0/html/janus.js) (`janus.js`, version `1.0.0`)
|
||||
- [WebRTC Adapter library](https://webrtc.github.io/adapter/adapter-8.1.0.js) (`webrtc-adapter.js`, version `8.1.0`)
|
||||
- [Janus Gateway JavaScript library](https://raw.githubusercontent.com/meetecho/janus-gateway/v1.0.0/html/janus.js) (`janus.js`, version `1.0.0`)
|
||||
|
||||
### Control Flow
|
||||
|
||||
The only HTML we need is a `<video>` element for rendering the H.264 video stream.
|
||||
The client-side JavaScript application uses the following control flow:
|
||||
|
||||
The control flow inside the client-side JavaScript application is this:
|
||||
|
||||
1. The client loads and initiates the Janus client library, and establishes a network connection to the Janus server.
|
||||
1. The client instructs the server to attach the µStreamer Janus plugin.
|
||||
1. On success, the client obtains a plugin handle through which it can send requests to the µStreamer Janus plugin directly. The responses are processed via the `attach` callbacks, in this case:
|
||||
- `onmessage` callback for general messages
|
||||
1. The client loads and initializes the Janus client library.
|
||||
1. The client establishes a WebSockets connection to the Janus server.
|
||||
1. The client instructs the Janus server to attach the µStreamer Janus plugin.
|
||||
1. On success, the client obtains a plugin handle through which it can send requests directly to the µStreamer Janus plugin. The client processes responses via the `attach` callbacks:
|
||||
- `onmessage` for general messages
|
||||
- `onremotetrack` for the H.264 video stream
|
||||
1. The client issues a `watch` request to the µStreamer Janus plugin, which initiates the H.264 stream in the plugin itself. When attaching the plugin for the first time, this doesn’t immediately succeed, but it takes a few seconds for the H.264 media to become available. The client has to retry the `watch` request in this case.
|
||||
1. Once the H.264 media is available for streaming in the µStreamer Janus plugin, client and server have to negotiate the underlying parameters of the WebRTC session, such as port or codec information. This procedure is called JSEP (JavaScript Session Establishment Protocol), where the server makes a `jsepOffer` to the client, and the client responds with a `jsepAnswer`.
|
||||
1. After the client and the server complete the JSEP exchange, the WebRTC connection is eventually established. The µStreamer Janus plugin automatically begins to deliver the H.264 video stream to the client via WebRTC.
|
||||
1. Once the video data starts to arrive on the client, the Janus client library invokes the `onremotetrack` callback. The client attaches the received stream to the `<video>` element. This will render the video stream on the screen.
|
||||
1. The client issues a `watch` request to the µStreamer Janus plugin, which initiates the H.264 stream in the plugin itself.
|
||||
- It takes a few seconds for uStreamer's video stream to become available to Janus. The first `watch` request may fail, so the client must retry the `watch` request.
|
||||
1. The client and server negotiate the underlying parameters of the WebRTC session. This procedure is called JavaScript Session Establishment Protocol (JSEP). The server makes a `jsepOffer` to the client, and the client responds with a `jsepAnswer`.
|
||||
1. The client issues a `start` request to the µStreamer Janus plugin to indicate that the client wants to begin consuming the video stream.
|
||||
1. The µStreamer Janus plugin delivers the H.264 video stream to the client via WebRTC.
|
||||
1. The Janus client library invokes the `onremotetrack` callback. The client attaches the video stream to the `<video>` element, rendering the video stream in the browser window.
|
||||
|
||||
### Sample Code
|
||||
|
||||
@ -112,9 +128,9 @@ The control flow inside the client-side JavaScript application is this:
|
||||
<meta charset="utf-8">
|
||||
<title>µStreamer H.264 demo</title>
|
||||
<script src="https://webrtc.github.io/adapter/adapter-8.1.0.js"></script>
|
||||
<!-- `janus.js` is the JavaScript client library of Janus, as specified above
|
||||
in the prerequisites section of the client setup. You might need to
|
||||
change the `src` path, depending on where you serve this file from. -->
|
||||
<!-- janus.js is the JavaScript client library of Janus, as specified above in
|
||||
the prerequisites section of the client setup. You might need to change
|
||||
the `src` path, depending on where you serve this file from. -->
|
||||
<script src="janus.js"></script>
|
||||
<style>
|
||||
video {
|
||||
@ -136,15 +152,15 @@ The control flow inside the client-side JavaScript application is this:
|
||||
dependencies: Janus.useDefaultDependencies(),
|
||||
});
|
||||
|
||||
// Establish a network connection to the server.
|
||||
// Establish a WebSockets connection to the server.
|
||||
const janus = new Janus({
|
||||
// The URL of the server’s websocket endpoint.
|
||||
// Specify the URL of the Janus server’s WebSockets endpoint.
|
||||
server: `ws://${window.location.hostname}:8188/`,
|
||||
|
||||
// Callback function, for when the client connected successfully.
|
||||
// Callback function if the client connects successfully.
|
||||
success: attachUStreamerPlugin,
|
||||
|
||||
// Callback function, for when the client failed to connect.
|
||||
// Callback function if the client fails to connect.
|
||||
error: console.error,
|
||||
});
|
||||
|
||||
@ -160,24 +176,24 @@ The control flow inside the client-side JavaScript application is this:
|
||||
// successfully.
|
||||
success: function (pluginHandle) {
|
||||
uStreamerPluginHandle = pluginHandle;
|
||||
// Instruct the µStreamer Janus plugin to initiate H.264 stream.
|
||||
// Instruct the µStreamer Janus plugin to initiate the video stream.
|
||||
uStreamerPluginHandle.send({ message: { request: "watch" } });
|
||||
},
|
||||
|
||||
// Callback function, for when the server failed to attach the plugin.
|
||||
// Callback function if the server fails to attach the plugin.
|
||||
error: console.error,
|
||||
|
||||
// Callback function, for when a message arrived from the server.
|
||||
// Callback function for processing messages from the Janus server.
|
||||
onmessage: function (msg, jsepOffer) {
|
||||
// `503` indicates that the plugin is not ready to stream yet. Retry
|
||||
// the watch request, until the H.264 stream is available.
|
||||
// 503 indicates that the plugin is not ready to stream yet. Retry the
|
||||
// watch request until the video stream is available.
|
||||
if (msg.error_code === 503) {
|
||||
uStreamerPluginHandle.send({ message: { request: "watch" } });
|
||||
return;
|
||||
}
|
||||
|
||||
// If there is a JSEP offer, respond to it. This will eventually
|
||||
// start the WebRTC connection.
|
||||
// If there is a JSEP offer, respond to it. This starts the WebRTC
|
||||
// connection.
|
||||
if (jsepOffer) {
|
||||
uStreamerPluginHandle.createAnswer({
|
||||
jsep: jsepOffer,
|
||||
@ -198,9 +214,9 @@ The control flow inside the client-side JavaScript application is this:
|
||||
// Callback function, for when the video stream arrives.
|
||||
onremotetrack: function (mediaStreamTrack, mediaId, isAdded) {
|
||||
if (isAdded) {
|
||||
// Attach the received media track to the video element. By cloning
|
||||
// the `mediaStreamTrack`, we ensure that the stream gets a
|
||||
// distinct, globally unique stream id.
|
||||
// Attach the received media track to the video element. Cloning the
|
||||
// mediaStreamTrack creates a new object with a distinct, globally
|
||||
// unique stream identifier.
|
||||
const videoElement = document.getElementById("webrtc-output");
|
||||
const stream = new MediaStream();
|
||||
stream.addTrack(mediaStreamTrack.clone());
|
||||
|
||||
Loading…
x
Reference in New Issue
Block a user