Setting up HAProxy + SSL Offload in front of Nginx-VOD

paco_medina · June 15, 2020, 3:00pm

Hello,

I try to use my HAProxy with SSL Offloading in front of the Nginx VOD package.

I allready fixed the CORS header with Lua but I’m still having a 404 error :

Nginx says :

[error] 7471#7471: *74 ngx_child_request_wev_handler: upstream returned a bad status 400 while sending to client

Do you have any idea how to fix this ?

Regards

jess · June 15, 2020, 3:33pm

Hello,

See Install kaltura Nginx VOD module

paco_medina · June 15, 2020, 4:40pm

I did, but it’s a bit different on my scenario.

I’d like to use HAProxy as SSL Offload, I followed your suggestion about customising Nginx with “proxy_pass” to https for exemple I got this before :

==> /opt/kaltura/log/kaltura_nginx_access.log <==

172.16.0.253 - - [15/Jun/2020:18:36:08 +0200] "GET /hls/p/102/sp/10200/serveFlavor/entryId/0_9le3e0go/v/2/ev/7/flavorId/0_sektb0kw/name/a.mp4/index.m3u8 HTTP/1.1" 404 375 0.002 "https://media.ouivid.com/apps/studio/v2.2.1/index.html" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/83.0.4103.97 Safari/537.36" "-" - "-" "vod.ouivid.com" 22112 - - 606 "-" "80.15.149.130" "-" "-" "-" 2634 

==> /opt/kaltura/log/kaltura_nginx_errors.log <==

2020/06/15 18:36:08 [error] 22112#22112: *2634 upstream prematurely closed connection while reading response header from upstream, client: 172.16.0.253, server: vod.ouivid.com, request: "GET /hls/p/102/sp/10200/serveFlavor/entryId/0_9le3e0go/v/2/ev/7/flavorId/0_sektb0kw/name/a.mp4/index.m3u8 HTTP/1.1", subrequest: "/kalapi_proxy/hls/p/102/sp/10200/serveFlavor/entryId/0_9le3e0go/v/2/ev/7/flavorId/0_sektb0kw/name/a.mp4", upstream: "http://80.15.149.130:443/hls/p/102/sp/10200/serveFlavor/entryId/0_9le3e0go/v/2/ev/7/flavorId/0_sektb0kw/name/a.mp4?pathOnly=1", host: "vod.ouivid.com", referrer: "https://media.ouivid.com/apps/studio/v2.2.1/index.html"

2020/06/15 18:36:08 [error] 22112#22112: *2634 open() "/etc/nginx/html/50x.html" failed (2: No such file or directory), client: 172.16.0.253, server: vod.ouivid.com, request: "GET /hls/p/102/sp/10200/serveFlavor/entryId/0_9le3e0go/v/2/ev/7/flavorId/0_sektb0kw/name/a.mp4/index.m3u8 HTTP/1.1", host: "vod.ouivid.com", referrer: "https://media.ouivid.com/apps/studio/v2.2.1/index.html"

But now I still have :

==> /opt/kaltura/log/kaltura_nginx_errors.log <==

2020/06/15 18:44:10 [error] 24210#24210: *11 ngx_http_vod_hls_parse_uri_file_name: unidentified request, client: 172.16.0.253, server: vod.ouivid.com, request: "GET /hls/p/102/sp/10200/serveFlavor/entryId/0_9le3e0go/v/2/ev/7/flavorId/0_cef8wgxi/name/a.mp4?pathOnly=1 HTTP/1.0", host: "vod.ouivid.com", referrer: "https://media.ouivid.com/apps/studio/v2.2.1/index.html"

==> /opt/kaltura/log/kaltura_nginx_access.log <==

172.16.0.253 - - [15/Jun/2020:18:44:10 +0200] "GET /hls/p/102/sp/10200/serveFlavor/entryId/0_9le3e0go/v/2/ev/7/flavorId/0_cef8wgxi/name/a.mp4?pathOnly=1 HTTP/1.0" 400 713 0.000 "https://media.ouivid.com/apps/studio/v2.2.1/index.html" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/83.0.4103.97 Safari/537.36" "-" - "-" "vod.ouivid.com" 24210 - - 556 "-" "80.15.149.130, 80.15.149.130" "-" "-" "-" 11 

==> /opt/kaltura/log/kaltura_nginx_errors.log <==

2020/06/15 18:44:10 [error] 24210#24210: *1 ngx_child_request_wev_handler: upstream returned a bad status 400 while sending to client, client: 172.16.0.253, server: vod.ouivid.com, request: "GET /hls/p/102/sp/10200/serveFlavor/entryId/0_9le3e0go/v/2/ev/7/flavorId/0_cef8wgxi/name/a.mp4/index.m3u8 HTTP/1.1", host: "vod.ouivid.com", referrer: "https://media.ouivid.com/apps/studio/v2.2.1/index.html"

2020/06/15 18:44:10 [error] 24210#24210: *1 open() "/etc/nginx/html/50x.html" failed (2: No such file or directory), client: 172.16.0.253, server: vod.ouivid.com, request: "GET /hls/p/102/sp/10200/serveFlavor/entryId/0_9le3e0go/v/2/ev/7/flavorId/0_cef8wgxi/name/a.mp4/index.m3u8 HTTP/1.1", host: "vod.ouivid.com", referrer: "https://media.ouivid.com/apps/studio/v2.2.1/index.html"

From the BDD, I think it’s all right :

MariaDB [kaltura]> select id,name,url,host_name from delivery_profile;
+------+------------------------------------------------+------------------------------+----------------+
| id   | name                                           | url                          | host_name      |
+------+------------------------------------------------+------------------------------+----------------+
|    1 | Default HTTP Delivery Profile                  | media.ouivid.com             | NULL           |
|    2 | Default HTTP Delivery Profile                  | media.ouivid.com             | NULL           |
|    3 | Default HLS Live Delivery Profile              | NULL                         | NULL           |
|    4 | Default HLS Network Live Delivery Profile      | NULL                         | NULL           |
|    5 | Default HLS To Multicast Live Delivery Profile | NULL                         | NULL           |
|  301 | Default MPEG-DASH Live Delivery Profile        | NULL                         | NULL           |
|  302 | Default HD Network Live Delivery Profile       | NULL                         | NULL           |
|  303 | Default HDS Live Delivery Profile              | NULL                         | NULL           |
|  304 | Kaltura Live Packager Dash segmentation        | vod.ouivid.com:443/live/dash | vod.ouivid.com |
|  305 | Kaltura Live Packager HDS segmentation         | vod.ouivid.com:443/live/hds  | vod.ouivid.com |
|  306 | Kaltura Live Packager HLS segmentation         | vod.ouivid.com:443/live/hls  | vod.ouivid.com |
|  307 | Kaltura Live Packager MSS segmentation         | vod.ouivid.com:443/live/mss  | vod.ouivid.com |
|  308 | Default RTMP Live Delivery Profile             | NULL                         | NULL           |
| 1001 | Kaltura HLS segmentation                       | vod.ouivid.com:443/hls       | vod.ouivid.com |
| 1002 | Kaltura HDS segmentation                       | vod.ouivid.com:443/hds       | vod.ouivid.com |
| 1003 | Kaltura DASH segmentation                      | vod.ouivid.com:443/dash      | vod.ouivid.com |
+------+------------------------------------------------+------------------------------+----------------+
16 rows in set (0.01 sec)

paco_medina · June 15, 2020, 5:01pm

Also :

Header Request :

GET /hls/p/102/sp/10200/serveFlavor/entryId/0_9le3e0go/v/2/ev/7/flavorId/0_hjrc8rfw/name/a.mp4/index.m3u8 HTTP/1.1
Host: vod.ouivid.com
Connection: keep-alive
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/83.0.4103.97 Safari/537.36
Accept: */*
Origin: https://media.ouivid.com
Sec-Fetch-Site: same-site
Sec-Fetch-Mode: cors
Sec-Fetch-Dest: empty
Referer: https://media.ouivid.com/html5/html5lib/v2.82.2/mwEmbedFrame.php/p/102/uiconf_id/23448169/entry_id/0_9le3e0go?wid=_102&iframeembed=true&flashvars[kAnalony.plugin]=false&flashvars[closedCaptions.plugin]=true&flashvars[closedCaptions.hideWhenEmpty]=true&flashvars[ks]=MTIxN2M5NzA3YmRiNzkyYjljM2I1ZmNkZGJiOWNlYzFhNmM5YWRmNnwxMDI7MTAyOzE1OTIzMTA0MDI7MjsxNTkyMjI0MDAyLjEzOTtwbWVkaW5hQG91aW1haWwuZnI7ZGlzYWJsZWVudGl0bGVtZW50LGFwcGlkOmttYzs7&flashvars[disableAlerts]=true&entry_id=0_9le3e0go&hash=1
Accept-Encoding: gzip, deflate, br
Accept-Language: fr-FR,fr;q=0.9,en-US;q=0.8,en;q=0.7

Header Response :

HTTP/1.1 404 Not Found
Server: nginx/1.17.10
Date: Mon, 15 Jun 2020 17:01:35 GMT
Content-Type: text/html
Transfer-Encoding: chunked
Content-Encoding: gzip
Vary: Accept-Encoding,Origin
Access-Control-Allow-Origin: https://media.ouivid.com

Response :

<html>
<head><title>404 Not Found</title></head>
<body>
<center><h1>404 Not Found</h1></center>
<hr><center>nginx/1.17.10</center>
</body>
</html>

But maybe the problem in not on HAProxy, because when I send the request directly to the Nginx VOD (http port 88) I got the same :

GET /hls/p/102/sp/10200/serveFlavor/entryId/0_9le3e0go/v/2/ev/7/flavorId/0_cef8wgxi/name/a.mp4/index.m3u8 HTTP/1.1
Host: vod.ouivid.com:88
Connection: keep-alive
Upgrade-Insecure-Requests: 1
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/83.0.4103.97 Safari/537.36
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9
Accept-Encoding: gzip, deflate
Accept-Language: fr-FR,fr;q=0.9,en-US;q=0.8,en;q=0.7

HTTP/1.1 404 Not Found
Server: nginx/1.17.10
Date: Mon, 15 Jun 2020 17:06:49 GMT
Content-Type: text/html
Transfer-Encoding: chunked
Connection: keep-alive
Content-Encoding: gzip

<html>
<head><title>404 Not Found</title></head>
<body>
<center><h1>404 Not Found</h1></center>
<hr><center>nginx/1.17.10</center>
</body>
</html>

david.eusse · June 16, 2020, 2:14pm

Hello @paco_medina,

We have a custom CDN based on nginx in front of kaltura-nginx and it works well.

I suggest you set your backend like this:

upstream nginxvod {
server 192.168.0.1:88 ;
server 192.168.0.2:88 ;
etc.
}

Keep in mind two things:

You need to send the proper headers to kaltura-nginx so your manifests and delivery use https.
Do it like this:
location ~* ^/(hls|dash|mss)/.* {
add_header ‘Access-Control-Allow-Credentials’ ‘true’ always;
add_header X-CdnHost $hostname always;
add_header X-Cached $upstream_cache_status always;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header X-Forwarded-Host $host;
expires 7d;
proxy_cache_valid 7d ;
proxy_cache_valid 404 1m;
proxy_pass http://nginxvod;
}
Let kaltira-nginx respond to the delivery domains you ar using that might be different from kaltura’s own domain. Add them to nginx.conf like this:

server {
listen 88;
server_name kaltura.mydomain.com delivery.mydomain.com;
include /etc/nginx/conf.d/live.conf;
include /etc/nginx/conf.d/kaltura.conf;

Kaltura-nginx doesn’t need to have an ssl configuration. We don’t need LUA, either.

Hope this will help,

David