From c87ad5703c5cde8f2ca55dbedc8200148513a53e Mon Sep 17 00:00:00 2001
From: Michael Lynch <mtlynch@users.noreply.github.com>
Date: Fri, 21 Oct 2022 21:43:38 -0400
Subject: [PATCH] Update H264 streaming guide to include audio streaming (#181)

* Support multiple (audio+video) streams in demo janus client (#4)

* Support multiple (audio+video) streams in demo janus client

* Adjust wording in H264 guide

* Use consistent braces style

Co-authored-by: Louis Goessling <louis@goessling.com>
---
 docs/h264.md | 29 +++++++++++++++++++++--------
 1 file changed, 21 insertions(+), 8 deletions(-)

diff --git a/docs/h264.md b/docs/h264.md
index ed95f64..4a8be50 100644
--- a/docs/h264.md
+++ b/docs/h264.md
@@ -78,6 +78,17 @@ memsink: {
 EOF
 ```
 
+If you're using a TC358743-based video capture device that supports audio capture, run the following command to enable audio streaming:
+
+```sh
+cat << EOF >> /opt/janus/lib/janus/configs/janus.plugin.ustreamer.jcfg
+audio: {
+    device = "hw:1"
+    tc358743 = "/dev/video0"
+}
+EOF
+```
+
 ### Start µStreamer and the Janus WebRTC Server
 
 For µStreamer to share the video stream with the µStreamer Janus plugin, µStreamer must run with the following command-line flags:
@@ -111,13 +122,14 @@ The client-side JavaScript application uses the following control flow:
 1. The client instructs the Janus server to attach the µStreamer Janus plugin.
 1. On success, the client obtains a plugin handle through which it can send requests directly to the µStreamer Janus plugin. The client processes responses via the `attach` callbacks:
    - `onmessage` for general messages
-   - `onremotetrack` for the H.264 video stream
+   - `onremotetrack` for the H.264 video stream and (optionally) an Opus audio stream
 1. The client issues a `watch` request to the µStreamer Janus plugin, which initiates the H.264 stream in the plugin itself.
    - It takes a few seconds for uStreamer's video stream to become available to Janus. The first `watch` request may fail, so the client must retry the `watch` request.
 1. The client and server negotiate the underlying parameters of the WebRTC session. This procedure is called JavaScript Session Establishment Protocol (JSEP). The server makes a `jsepOffer` to the client, and the client responds with a `jsepAnswer`.
 1. The client issues a `start` request to the µStreamer Janus plugin to indicate that the client wants to begin consuming the video stream.
-1. The µStreamer Janus plugin delivers the H.264 video stream to the client via WebRTC.
-1. The Janus client library invokes the `onremotetrack` callback. The client attaches the video stream to the `<video>` element, rendering the video stream in the browser window.
+1. The µStreamer Janus plugin delivers the H.264 video stream and (optionally) an Opus audio stream to the client via WebRTC.
+1. The Janus client library invokes the `onremotetrack` callback with the video stream. The client attaches the video stream to the `<video>` element, rendering the video stream in the browser window.
+1. (if an audio track is available) The Janus client library invokes the `onremotetrack` callback with the Opus audio stream. The client adds the audio stream to the `<video>` element, rendering the audio in the browser window.
 
 ### Sample Code
 
@@ -176,7 +188,7 @@ The client-side JavaScript application uses the following control flow:
         // successfully.
         success: function (pluginHandle) {
           uStreamerPluginHandle = pluginHandle;
-          // Instruct the µStreamer Janus plugin to initiate the video stream.
+          // Instruct the µStreamer Janus plugin to initiate streaming.
           uStreamerPluginHandle.send({ message: { request: "watch" } });
         },
 
@@ -211,16 +223,17 @@ The client-side JavaScript application uses the following control flow:
           }
         },
 
-        // Callback function, for when the video stream arrives.
+        // Callback function, for when a media stream arrives.
         onremotetrack: function (mediaStreamTrack, mediaId, isAdded) {
           if (isAdded) {
             // Attach the received media track to the video element. Cloning the
             // mediaStreamTrack creates a new object with a distinct, globally
             // unique stream identifier.
             const videoElement = document.getElementById("webrtc-output");
-            const stream = new MediaStream();
-            stream.addTrack(mediaStreamTrack.clone());
-            videoElement.srcObject = stream;
+            if (videoElement.srcObject === null) {
+              videoElement.srcObject = new MediaStream();
+            }
+            videoElement.srcObject.addTrack(mediaStreamTrack.clone());
           }
         },
       });