Desktop and media video streaming server with OBS, nginx, RTMP, HLS and DASH

Published in in Tech. Last updated: Sun 12 April 2020
#dash #hls #nginx #obs #rtmp #streaming #video #videojs

Update: The article was extended with DASH and RTMP endpoint authentication after the initial publication.

Due to current circumstances, video streaming and desktop sharing has found an enormous increase in usage. This post contains a quick guide to set up an independent video streaming server with nginx and RTMP, which OBS Studio clients can stream to. Viewers can connect via a website and receive the stream in their browser HTML5 players. OBS is so unbelievably easy to use, it's fantastic for remote tutorials and application/game streaming where you wish to use multiple media inputs (microphone, desktop, webcam) and have good control of the output composition.

The following guide is based on Debian 10 (Buster), nginx 1.14.2 and OBS Studio 25.

Step 1: Install and set up nginx with RTMP module

Install the nginx package and the RTMP module with apt install nginx-full libnginx-mod-rtmp. Configure the nginx.conf http section similar to the one shown below, e.g. enable gzip for certain file types and add/delete entries from the available TLS versions. On the top level, add the rtmp section. See the nginx-rtmp-module directives reference for more information on the individual keywords.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
http {
    sendfile on;  # use sendfile for static files (.ts, .m3u8)
    tcp_nopush on;  # used in combination with sendfile
    tcp_nodelay on;  # decrease latency for small packets

    include /etc/nginx/mime.types;
    default_type application/octet-stream;

    ssl_protocols TLSv1.2 TLSv1.3;  # remove old protocols, add TLS 1.3
    ssl_prefer_server_ciphers on;

    access_log /var/log/nginx/access.log;
    error_log /var/log/nginx/error.log;

    gzip on;  # use gzip for file types defined below
    gzip_comp_level 9;
    gzip_http_version 1.1;
    gzip_types text/plain text/css application/json application/javascript text/xml application/xml application/xml+rss text/javascript;

    include /etc/nginx/conf.d/*.conf;
    include /etc/nginx/sites-enabled/*;
}

rtmp {
    server {
        listen 1935;  # Standard RTMP port
        listen [::]:1935 ipv6only=on;
        ping 30s;

        application live {
            live on;
            deny play all;  # disable RTMP viewer clients (not streaming clients)

            # Use HLS transcoding
            hls on;
            hls_path /var/www/hls;  # where fragments are stored
            hls_fragment 3s;  # length of each segment in seconds
            hls_playlist_length 30s;   # length of total recording in seconds
            hls_cleanup on;  # delete fragments on restart/shutdown

            # Same with DASH
            dash on;
            dash_path /var/www/dash;
            dash_fragment 2s;
            dash_playlist_length 1m;
            dash_cleanup on;
        }
    }
}

Then create the directories /var/www/hls and /var/www/dash, and let www-data own it. With this configuration, nginx now receives incoming RTMP streams on port 1935 (automatically derived from rtmp://SERVER_NAME/live) and stores each stream as HLS and DASH fragments. Note that the module does not support RTMPS (RTMP over TLS), so everything is sent unencrypted (source: Debian module repository). For authentication, see section authentication.

Next, set up a virtual host in /etc/nginx/sites-enabled, for example use the default vhost that exists already. In it, the following server content is needed. Be aware this server is configured to use HTTPS with a certificate. You can obtain automatically issued free certificates with certbot (package certbot) for your hostname, but that's out of scope for now.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
server {
    listen 443 ssl http2;
    listen [::]:443 ssl http2;

    server_name SERVER_NAME;
    ssl_certificate ...;
    ssl_certificate_key ...;

    root /var/www/html;
    index index.html;

    # Source: https://docs.peer5.com/guides/setting-up-hls-live-streaming-server-using-nginx/
    add_header 'Access-Control-Allow-Origin' '*' always;
    add_header 'Access-Control-Expose-Headers' 'Content-Length';
    if ($request_method = 'OPTIONS') {  # allow CORS preflight requests
        add_header 'Access-Control-Allow-Origin' '*';
        add_header 'Access-Control-Max-Age' 1728000;
        add_header 'Content-Type' 'text/plain charset=UTF-8';
        add_header 'Content-Length' 0;
        return 204;
    }

    types {
        application/vnd.apple.mpegurl m3u8;
        video/mp2t ts;
    }

    location /hls/ {
        add_header Cache-Control no-cache;  # disable client-side caching
        root /var/www;
    }

    location /dash/ {
        root /var/www;
        add_header Cache-Control no-cache;
    }
}

The HLS/DASH fragments are now available at https://SERVER_NAME/hls and /dash. CORS configuration is needed if you embed this resource from other domains.

Step 2: Add HTML streaming client

If you finished step one, nginx is now ready to receive RTMP streams and to distribute these packets via HLS and DASH. Next, we will add an HTML page which offers a video client for viewers.

The page is pretty simple, just save it as index.html in /var/www/html/. You also need the video.js and dash.js libraries. video.min.js and video-js.min.css you can get from the video.js Github releases page, videojs-http-streaming.min.js is available at unpkg.com/@videojs and dash.all.min.js at the DASH Github repository.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
<html>
    <head>
        <title>streaming client</title>
        <meta charset="utf-8"/>
        <link href="video-js.min.css" rel="stylesheet" />
    </head>
    <body>
        <center>
        <h1>live streaming</h1>
        <video-js id="liveplayer" controls autoplay width=600 class="vjs-default-skin">
            <source src="https://SERVER_NAME/hls/STREAM_KEY.m3u8" type="application/x-mpegURL">
        </video-js>

        <br><br>

        <video width=600 data-dashjs-player autoplay controls
            src=https://SERVER_NAME/dash/STREAM_KEY.mpd></video>

        </center>

        <script src="video.min.js"></script>
        <script src="videojs-http-streaming.min.js"></script>
        <script src="dash.all.min.js"></script>
        <script>
            var player = videojs("liveplayer");
            player.play();
        </script>
    </body>
</html>

Replace the SERVER_NAME placeholders with the hostname your nginx is reachable at (server_name configuration entry). The STREAM_KEY URI part must also be replaced, the value being the name of the stream you wish to view in this video element. We will come back to this value in part three. In case you are sure to never need to serve Apple devices (iOS), you can also remove the HLS player entirely.

Step 3: Configure OBS

Now it is time to connect the streaming client. OBS supports a wide set of video streaming services like Twitch or YouTube Gaming out of the box, but we will use our own!

Start by going into the settings menu. In the stream settings dialog, use custom from the drop down service list. Two empty input fields will appear. In the server field enter rtmp://SERVER_NAME/live, the second field stream key being the name of the stream. For example, if you use mylivestream as the key, the video source URLs created by nginx (used in the HTML code above) would be https://SERVER_NAME/hls/mylivestream.m3u8 and https://SERVER_NAME/dash/mylivestream.mpd.

Don't forget to also configure your audio and video settings. I reached good performance with a video bitrate of 90% of my total upload speed, setting the resolution to native full HD 1920x1080 (no downscaling) and 30 FPS. If available, enable hardware-based video encoding (e.g. NVIDIA NVENC).

As soon as you hit the "Start Streaming" button, OBS connects to the RTMP endpoint, sends its packets and nginx transforms them into HLS/DASH fragments. A viewer can then visit the URL where the index.html from step two is delivered at and receives a web page with the video players!

During recording/streaming the stats window allows you to view statistics about upload size and dropped frames. You might also notice that there might be a delay of up to 30 seconds between sending and receiving, which might be acceptable in most scenarios, but may interfere with your workflow if viewers have a possibility to comment your content live during the streaming session. There are ways to reduce this delay, with the cost of increasing traffic (more smaller packets mean more HTTP requests by clients). In a test I could reduce the time difference between sending and receiving to six seconds (key frame every two frames in OBS, HSL fragment size of 2 and list length 10). If you need realtime bidirectional conferencing, jitsi might be a better alternative.

Authentication for RTMP streamers

To not let everybody who knows the RTMP endpoint use the stream, we will also add authentication. This is done by utilising the notification feature of the nginx RTMP module. Extend your RTMP config in nginx as follows:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
rtmp {
    server {
        listen 1935;
        listen [::]:1935 ipv6only=on;
        ping 30s;

        notify_method get;  # forward notify request as GET (default POST)
        on_connect http://localhost/rtmp_auth;

        application live {
            live on;
            [...]

Now every time someone connects with RTMP to begin streaming on port 1935, the http://localhost/rtmp_auth URL will be called by nginx, with all parameters delivered by the OBS client. Next, the corresponding authentication route needs to be added. In this example we are using a very simple parameter check built into nginx. Here, if the client provides the token parameter with the MyVerySecurePassword123 value, nginx returns with code 200, which means authentication success. All other cases are returned with 403 error code (unauthorized). This works with every route returning 200/403 though, for example with another webapp which talks to a database, or an embedded Lua script (content_by_lua_block or content_by_lua_file).

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
server {
    listen 127.0.0.1:80 default_server;
    listen [::1]:80 default_server;

    location = /rtmp_auth {
        if ($arg_token = "MyVerySecurePassword123") {
            return 200;
        }
        return 403;
    }
}

If you already have a configuration for the localhost virtual server, add the snippet to this server block.

As the last thing, you need to adapt OBS to match the new config! Simply change your streaming server to rtmp://SERVER_NAME/live?token=MyVerySecurePassword123. The server will now accept your requests, but deny any others which have either no token parameter in their request, or which provide a wrong token (denied means a return code of 403).

Background

For reference, here's a list of terms mentioned in this article:

  • RTMP: Real-Time Messaging Protocol, the sender's streaming protocol. Originally developed by Macromedia (Flash, now Adobe) and pretty old standard. Yet still used widely by many video streaming providers.
  • HLS: HTTP Live Streaming, by Apple. Takes an incoming stream (e.g. RTMP) and chunks it into fragments, creating downloadable files for clients. HLS clients understand the .m3u8 playlist with its .ts entries and continously request the next fragments, generating an uninterrupted output stream.
  • MPEG-DASH: Dynamic Adaptive Streaming over HTTP, similar to HLS, but standardised by the MPEG and backed by the DASH Industry Forum. It should therefore be your preferred protocol, because it's an open standard an browsers are more likely to implement DASH into the Media Source Extensions (MSE) API. Uses fragmenting as well, creating .m4v and .m4a files which are referenced in an .mpd playlist.
  • SRT: Secure Reliable Transport, developed by Haivision and backed by the SRT Alliance community. New transport standard published under an open source license, with encryption, packet loss detection and dynamically adapting transport as core features. Support in OBS is experimental, referencing the state of implementation in ffmpeg in their forums.

Thanks for reading and happy streaming!

By the way, the OBS client used in this setup is running inside my virtualised Windows machine with PCI passthrough. I have blogged about this in the past: "Using a PCI graphics card in KVM/QEMU on Debian Stretch". All peripheral stuff like a webcam or USB microphone for OBS can be passed as USB host devices as well.