Docker + Cloud-front + S3


#1

I setup a server with Docker + S3 everything is working except when I upload a file it uploads to s3 but files dont play. Basically the manifest link is the main domain but it errors our and gives 404.

Anybody have this working?


#2

Hi @melaleuca5,

Kaltura CE includes the kaltura-nginx package which provides Nginx compiled with the VOD module which supports several operational modes, see:

By default, kaltura-nginx is configured to work in mapped mode against your Kaltura Server. Meaning the files will be served from /opt/kaltura/web, which may be a local dir on one of the server’s disks or a remote volume mounted on each of the front nodes [in the event of a cluster rather than an all in one instance]. You can modify the Nginx configuration so that it fetches the media files from a CF endpoint.

The paths for the Nginx conf files vary between the deb and RPM packages. For RPM, the main file is /etc/nginx/nginx.conf, for deb /opt/kaltura/nginx/conf/nginx.conf, the Nginx and module versions are the same and so, regardless of the packaging format, apart from the paths, the contents should be the same.

Below is a very basic example of how to fetch the files from a CF endpoint but the same can be used with other vendors, of course. This assumes the S3 bucket is public and doesn’t require a token but naturally, you can modify it if authorisation is required.

ngnix.conf:

include /etc/nginx/conf.d/main.conf;

http {
        upstream media {
                server somecfspace.cloudfront.net;
                keepalive 32;
        }

        include /etc/nginx/conf.d/http.conf;

        # vod remote settings
        vod_mode remote;
        vod_upstream_location /media_proxy;

        server {
                listen 88;
                server_name your.nginx.server.name;
                include /etc/nginx/conf.d/server.conf;
        }
}

main.conf:

user  kaltura;
worker_processes  auto;

error_log  /opt/kaltura/log/nginx/kaltura_nginx_errors.log;

pid             /var/run/nginx.pid;

events {
        worker_connections  1024;
        worker_aio_requests 512;
        multi_accept on;
        use epoll;
}

http.conf:

        include    mime.types;
        default_type  application/octet-stream;

        log_format  main  '$remote_addr - $remote_user [$time_local] "$request" '
                '$status $bytes_sent $request_time "$http_referer" "$http_user_agent" "-" - '
                '"$sent_http_x_kaltura" "$http_host" $pid $sent_http_x_kaltura_session - '
                '$request_length "$sent_http_content_range" "$http_x_forwarded_for" '
                '"$http_x_forwarded_server" "$http_x_forwarded_host" "$sent_http_cache_control" '
                '$connection ';

        access_log /opt/kaltura/log/nginx/kaltura_nginx_access.log main;

        # general nginx tuning
        sendfile on;
        tcp_nopush on;
        tcp_nodelay on;

        keepalive_timeout 60;
        keepalive_requests 1000;
        client_header_timeout 20;
        client_body_timeout 20;
        reset_timedout_connection on;
        send_timeout 20;

        # manifest compression
        gzip on;
        gzip_types application/vnd.apple.mpegurl video/f4m application/dash+xml text/xml text/vtt;
        gzip_proxied any;

        # shared memory zones
        vod_metadata_cache metadata_cache 512m;
        vod_response_cache response_cache 64m;
        vod_performance_counters perf_counters;

        # common vod settings
        vod_last_modified 'Sun, 19 Nov 2000 08:52:00 GMT';
        vod_last_modified_types *;
        vod_expires 100d;
        vod_expires_live 30;
        vod_expires_live_time_dependent 3;
        vod_align_segments_to_key_frames on;
        vod_output_buffer_pool 64k 32;

server.conf:


                # internal location for vod subrequests
                location ~ /media_proxy/[^/]+/(.*) {
                        internal;
                        proxy_pass http://media/$1;
                        proxy_http_version 1.1;
                        proxy_set_header Host somecfspace.cloudfront.net;
                        proxy_set_header Connection "";
                }

                # base locations
                include /etc/nginx/conf.d/base.conf;

                # serve flavor progressive
                location /pd/ {
                        vod none;

                        directio 512;
                        output_buffers 1 512k;

                        include /etc/nginx/conf.d/cors.conf;
                }

                # serve flavor HLS
                location /hls/ {
                        vod hls;
                        vod_bootstrap_segment_durations 2000;
                        vod_bootstrap_segment_durations 2000;
                        vod_bootstrap_segment_durations 2000;
                        vod_bootstrap_segment_durations 4000;

                        include /etc/nginx/conf.d/cors.conf;
                }

                # serve flavor DASH
                location /dash/ {
                        vod dash;
                        vod_segment_duration 4000;
                        vod_dash_manifest_format segmenttemplate;
                        vod_manifest_duration_policy min;

                        include /etc/nginx/conf.d/cors.conf;
                }

                # serve flavor HDS
                location /hds/ {
                        vod hds;
                        vod_segment_duration 6000;
                        vod_segment_count_policy last_rounded;

                        include /etc/nginx/conf.d/cors.conf;
                }

                # serve flavor MSS
                location /mss/ {
                        vod mss;
                        vod_segment_duration 4000;
                        vod_manifest_segment_durations_mode accurate;

                        include /etc/nginx/conf.d/cors.conf;
                }

                # static files (crossdomain.xml, robots.txt etc.) + fallback to api
                location / {
                        root   @STATIC_FILES_PATH@;
                }

cors.conf

add_header Access-Control-Allow-Headers "Origin,Range,Accept-Encoding,Referer,Cache-Control";
add_header Access-Control-Expose-Headers "Server,Content-Length,Content-Range,Date";
add_header Access-Control-Allow-Methods "GET, HEAD, OPTIONS";
add_header Access-Control-Allow-Origin "*";

base.conf:

# nginx status page
location = /nginx_status {
stub_status on;
access_log off;
}

# vod status page
location = /vod_status {
vod_status;
access_log off;
}

# redirect server error pages to the static page /50x.html
error_page 500 502 503 504 /50x.html;

location = /50x.html {
root   html;
}

For detailed documentation of the VOD module configuration, see https://github.com/kaltura/nginx-vod-module.
kaltura-nginx is also shipped with the secure-token and nginx-akamai-token-validate modules which may interest you.


#3

@jess Thanks for the info, I am just not sure why this info is not in the documentation

I will try this and let you know if it works


#4

Hi @melaleuca5,

It’s not there because that document was contributed a long long time ago, before the VOD module was available:)

After you’re done configuring and testing, you’re more than welcome to update the documentation and make a pull request to the https://github.com/kaltura/platform-install-packages repo.
I started working on such a document myself a while back, I’ll commit it soon but I’m always happy to get outside contributors involved.

And of course, if you have follow up questions about the configuration, feel free to post them here.