mirror of
https://github.com/pikvm/ustreamer.git
synced 2026-03-12 10:33:42 +00:00
Update H264 streaming guide to include audio streaming (#181)
* Support multiple (audio+video) streams in demo janus client (#4) * Support multiple (audio+video) streams in demo janus client * Adjust wording in H264 guide * Use consistent braces style Co-authored-by: Louis Goessling <louis@goessling.com>
This commit is contained in:
29
docs/h264.md
29
docs/h264.md
@@ -78,6 +78,17 @@ memsink: {
|
|||||||
EOF
|
EOF
|
||||||
```
|
```
|
||||||
|
|
||||||
|
If you're using a TC358743-based video capture device that supports audio capture, run the following command to enable audio streaming:
|
||||||
|
|
||||||
|
```sh
|
||||||
|
cat << EOF >> /opt/janus/lib/janus/configs/janus.plugin.ustreamer.jcfg
|
||||||
|
audio: {
|
||||||
|
device = "hw:1"
|
||||||
|
tc358743 = "/dev/video0"
|
||||||
|
}
|
||||||
|
EOF
|
||||||
|
```
|
||||||
|
|
||||||
### Start µStreamer and the Janus WebRTC Server
|
### Start µStreamer and the Janus WebRTC Server
|
||||||
|
|
||||||
For µStreamer to share the video stream with the µStreamer Janus plugin, µStreamer must run with the following command-line flags:
|
For µStreamer to share the video stream with the µStreamer Janus plugin, µStreamer must run with the following command-line flags:
|
||||||
@@ -111,13 +122,14 @@ The client-side JavaScript application uses the following control flow:
|
|||||||
1. The client instructs the Janus server to attach the µStreamer Janus plugin.
|
1. The client instructs the Janus server to attach the µStreamer Janus plugin.
|
||||||
1. On success, the client obtains a plugin handle through which it can send requests directly to the µStreamer Janus plugin. The client processes responses via the `attach` callbacks:
|
1. On success, the client obtains a plugin handle through which it can send requests directly to the µStreamer Janus plugin. The client processes responses via the `attach` callbacks:
|
||||||
- `onmessage` for general messages
|
- `onmessage` for general messages
|
||||||
- `onremotetrack` for the H.264 video stream
|
- `onremotetrack` for the H.264 video stream and (optionally) an Opus audio stream
|
||||||
1. The client issues a `watch` request to the µStreamer Janus plugin, which initiates the H.264 stream in the plugin itself.
|
1. The client issues a `watch` request to the µStreamer Janus plugin, which initiates the H.264 stream in the plugin itself.
|
||||||
- It takes a few seconds for uStreamer's video stream to become available to Janus. The first `watch` request may fail, so the client must retry the `watch` request.
|
- It takes a few seconds for uStreamer's video stream to become available to Janus. The first `watch` request may fail, so the client must retry the `watch` request.
|
||||||
1. The client and server negotiate the underlying parameters of the WebRTC session. This procedure is called JavaScript Session Establishment Protocol (JSEP). The server makes a `jsepOffer` to the client, and the client responds with a `jsepAnswer`.
|
1. The client and server negotiate the underlying parameters of the WebRTC session. This procedure is called JavaScript Session Establishment Protocol (JSEP). The server makes a `jsepOffer` to the client, and the client responds with a `jsepAnswer`.
|
||||||
1. The client issues a `start` request to the µStreamer Janus plugin to indicate that the client wants to begin consuming the video stream.
|
1. The client issues a `start` request to the µStreamer Janus plugin to indicate that the client wants to begin consuming the video stream.
|
||||||
1. The µStreamer Janus plugin delivers the H.264 video stream to the client via WebRTC.
|
1. The µStreamer Janus plugin delivers the H.264 video stream and (optionally) an Opus audio stream to the client via WebRTC.
|
||||||
1. The Janus client library invokes the `onremotetrack` callback. The client attaches the video stream to the `<video>` element, rendering the video stream in the browser window.
|
1. The Janus client library invokes the `onremotetrack` callback with the video stream. The client attaches the video stream to the `<video>` element, rendering the video stream in the browser window.
|
||||||
|
1. (if an audio track is available) The Janus client library invokes the `onremotetrack` callback with the Opus audio stream. The client adds the audio stream to the `<video>` element, rendering the audio in the browser window.
|
||||||
|
|
||||||
### Sample Code
|
### Sample Code
|
||||||
|
|
||||||
@@ -176,7 +188,7 @@ The client-side JavaScript application uses the following control flow:
|
|||||||
// successfully.
|
// successfully.
|
||||||
success: function (pluginHandle) {
|
success: function (pluginHandle) {
|
||||||
uStreamerPluginHandle = pluginHandle;
|
uStreamerPluginHandle = pluginHandle;
|
||||||
// Instruct the µStreamer Janus plugin to initiate the video stream.
|
// Instruct the µStreamer Janus plugin to initiate streaming.
|
||||||
uStreamerPluginHandle.send({ message: { request: "watch" } });
|
uStreamerPluginHandle.send({ message: { request: "watch" } });
|
||||||
},
|
},
|
||||||
|
|
||||||
@@ -211,16 +223,17 @@ The client-side JavaScript application uses the following control flow:
|
|||||||
}
|
}
|
||||||
},
|
},
|
||||||
|
|
||||||
// Callback function, for when the video stream arrives.
|
// Callback function, for when a media stream arrives.
|
||||||
onremotetrack: function (mediaStreamTrack, mediaId, isAdded) {
|
onremotetrack: function (mediaStreamTrack, mediaId, isAdded) {
|
||||||
if (isAdded) {
|
if (isAdded) {
|
||||||
// Attach the received media track to the video element. Cloning the
|
// Attach the received media track to the video element. Cloning the
|
||||||
// mediaStreamTrack creates a new object with a distinct, globally
|
// mediaStreamTrack creates a new object with a distinct, globally
|
||||||
// unique stream identifier.
|
// unique stream identifier.
|
||||||
const videoElement = document.getElementById("webrtc-output");
|
const videoElement = document.getElementById("webrtc-output");
|
||||||
const stream = new MediaStream();
|
if (videoElement.srcObject === null) {
|
||||||
stream.addTrack(mediaStreamTrack.clone());
|
videoElement.srcObject = new MediaStream();
|
||||||
videoElement.srcObject = stream;
|
}
|
||||||
|
videoElement.srcObject.addTrack(mediaStreamTrack.clone());
|
||||||
}
|
}
|
||||||
},
|
},
|
||||||
});
|
});
|
||||||
|
|||||||
Reference in New Issue
Block a user