Update H264 streaming guide to include audio streaming (#181)

* Support multiple (audio+video) streams in demo janus client (#4)

* Support multiple (audio+video) streams in demo janus client

* Adjust wording in H264 guide

* Use consistent braces style

Co-authored-by: Louis Goessling <louis@goessling.com>
This commit is contained in:
Michael Lynch 2022-10-21 21:43:38 -04:00 committed by GitHub
parent 95df13b7cb
commit c87ad5703c
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23

View File

@ -78,6 +78,17 @@ memsink: {
EOF
```
If you're using a TC358743-based video capture device that supports audio capture, run the following command to enable audio streaming:
```sh
cat << EOF >> /opt/janus/lib/janus/configs/janus.plugin.ustreamer.jcfg
audio: {
device = "hw:1"
tc358743 = "/dev/video0"
}
EOF
```
### Start µStreamer and the Janus WebRTC Server
For µStreamer to share the video stream with the µStreamer Janus plugin, µStreamer must run with the following command-line flags:
@ -111,13 +122,14 @@ The client-side JavaScript application uses the following control flow:
1. The client instructs the Janus server to attach the µStreamer Janus plugin.
1. On success, the client obtains a plugin handle through which it can send requests directly to the µStreamer Janus plugin. The client processes responses via the `attach` callbacks:
- `onmessage` for general messages
- `onremotetrack` for the H.264 video stream
- `onremotetrack` for the H.264 video stream and (optionally) an Opus audio stream
1. The client issues a `watch` request to the µStreamer Janus plugin, which initiates the H.264 stream in the plugin itself.
- It takes a few seconds for uStreamer's video stream to become available to Janus. The first `watch` request may fail, so the client must retry the `watch` request.
1. The client and server negotiate the underlying parameters of the WebRTC session. This procedure is called JavaScript Session Establishment Protocol (JSEP). The server makes a `jsepOffer` to the client, and the client responds with a `jsepAnswer`.
1. The client issues a `start` request to the µStreamer Janus plugin to indicate that the client wants to begin consuming the video stream.
1. The µStreamer Janus plugin delivers the H.264 video stream to the client via WebRTC.
1. The Janus client library invokes the `onremotetrack` callback. The client attaches the video stream to the `<video>` element, rendering the video stream in the browser window.
1. The µStreamer Janus plugin delivers the H.264 video stream and (optionally) an Opus audio stream to the client via WebRTC.
1. The Janus client library invokes the `onremotetrack` callback with the video stream. The client attaches the video stream to the `<video>` element, rendering the video stream in the browser window.
1. (if an audio track is available) The Janus client library invokes the `onremotetrack` callback with the Opus audio stream. The client adds the audio stream to the `<video>` element, rendering the audio in the browser window.
### Sample Code
@ -176,7 +188,7 @@ The client-side JavaScript application uses the following control flow:
// successfully.
success: function (pluginHandle) {
uStreamerPluginHandle = pluginHandle;
// Instruct the µStreamer Janus plugin to initiate the video stream.
// Instruct the µStreamer Janus plugin to initiate streaming.
uStreamerPluginHandle.send({ message: { request: "watch" } });
},
@ -211,16 +223,17 @@ The client-side JavaScript application uses the following control flow:
}
},
// Callback function, for when the video stream arrives.
// Callback function, for when a media stream arrives.
onremotetrack: function (mediaStreamTrack, mediaId, isAdded) {
if (isAdded) {
// Attach the received media track to the video element. Cloning the
// mediaStreamTrack creates a new object with a distinct, globally
// unique stream identifier.
const videoElement = document.getElementById("webrtc-output");
const stream = new MediaStream();
stream.addTrack(mediaStreamTrack.clone());
videoElement.srcObject = stream;
if (videoElement.srcObject === null) {
videoElement.srcObject = new MediaStream();
}
videoElement.srcObject.addTrack(mediaStreamTrack.clone());
}
},
});