"Remote Storage Only" and "Delete Exported Files" checked- but files still being stored locally?

1

Hey, I just experienced some issues because my disk filled up with media that I assumed was all being exported and deleted to my S3 bucket (I have confirmed that this is also happening). According to my settings, I have remote storage only enabled so why could this be happening? Is this because it’s a source file (no transcoding)? Is there any way to stop it or will I just have to clear out that folder of media daily?

Actually, now i’m noticing additional issues. When I look at my Kaltura Base Feed, I see media:content URLs on files that are pointing to paths like this, through my local host: /p/101/sp/10100/playManifest/entryId/0_5t9vzdax/flavorId/0_78mck2sf/protocol/http/format/url/a.mp4?clientTag=feed:0_ch0tl9bx

yet my thumbnails go through right, and appropriately point to the right cloudfront url which I have configured through remote storage:
cloudfront.net/content/entry/data/0/0/0_cu8c0be3_0_dpseh13a_2.jpg

Interestingly, these thumbs are making it to local storage as well.

Now, I could probably easily fix this with some XSLT, but it seems a clunky solution. I’ve no idea why my media:thumbnails make it through correctly, but my media:content URLs don’t.

Here’s an error log when I try to upload a video:
2016-08-24 12:51:55 [0.000114] [45.59.236.158] [1504529011] [56] [PS2] [kFileSyncUtils::getReadyFileSyncForKey] NOTICE: FileSync was not found
2016-08-24 12:51:55 [0.000888] [45.59.236.158] [1504529011] [57] [PS2] [kCoreException->__construct] ERR: exception ‘kFileSyncException’ with message ‘no ready filesync on current DC’ in /opt/kaltura/app/alpha/apps/kaltura/lib/myEntryUtils.class.php:755
Stack trace:
#0 /opt/kaltura/app/alpha/lib/model/entry.php(3346): myEntryUtils::resizeEntryImage(Object(entry), 0, 120, 90, 2, ‘F7F7F7’, NULL, 0, 0, 0, 0, 0, -1, ‘-1’, ‘-1’)

2016-08-24 12:51:55 [0.000115] [45.59.236.158] [1504529011] [65] [PS2] [kFileSyncUtils::getReadyFileSyncForKey] NOTICE: FileSync was not found
2016-08-24 12:51:55 [0.000155] [45.59.236.158] [1504529011] [66] [PS2] [entry->getLocalThumbFilePath] ERR: exception ‘Exception’ with message ‘No ready fileSync found on any DC.’ in /opt/kaltura/app/infra/log/KalturaLog.php:83
Stack trace:
#0 /opt/kaltura/app/alpha/lib/model/entry.php(3363): KalturaLog::err(‘No ready fileSy…’)

#16 {main}
2016-08-24 12:51:55 [0.000169] [45.59.236.158] [1504529011] [67] [PS2] [KExternalErrors::dieError] ERR: exception ‘Exception’ with message ‘exiting on error 10 - missing thumbnail fileSync for entry’ in /opt/kaltura/app/infra/log/KalturaLog.php:83
Stack trace:
#0 /opt/kaltura/app/alpha/apps/kaltura/lib/KExternalErrors.class.php(128): KalturaLog::err(‘exiting on erro…’)

        [tmp_name] => /opt/kaltura/web/tmp/php2XNSW7
        [error] => 0
        [size] => 9608834

–
[tmp_name] => /opt/kaltura/web/tmp/php2XNSW7
[error] => 0
[size] => 9608834

2016-08-24 12:53:09 [0.000119] [45.59.236.158] [1460464244] [533] [API] [kMetadataMrssManager->contributeMetadata] ALERT: exception ‘Exception’ with message ‘ready file sync was not found for key[object_type:[5], object_id:[259], version:[2], object_sub_type[1], partner_id[101]]’ in /opt/kaltura/app/infra/log/KalturaLog.php:65
Stack trace:
#0 /opt/kaltura/app/plugins/metadata/lib/kMetadataMrssManager.php(50): KalturaLog::alert(‘ready file sync…’)

2

I’m basically just trying to figure out how to get my mediaUrls in feeds to stop looking like this:
/p/101/sp/10100/playManifest/entryId/0_5t9vzdax/flavorId/0_78mck2sf/protocol/http/format/url/a.mp4?clientTag=feed:0_ch0tl9bx

and start looking like this:
/content/entry/data/0/0/0_srg0jdpa_0_h8hk5709_12.mp4

3

Hello,

Source files and additional supporting files, such as logs for the transcoding, assets that failed to convert, etc, will still be kept on your ‘local’ storage [i.e /opt/kaltura/web/content]. /opt/kaltura/web also contains other important files, such as UI confs and multiple widgets including KMC, the code for the HTML5 players, etc.

About changing the storage path, take a look at:
/opt/kaltura/app/alpha/apps/kaltura/lib/storage/kPathManager.php
/opt/kaltura/app/alpha/apps/kaltura/lib/storage/kExternalPathManager.php

You can extend these classes and then implement the needed changes, you will then also need to make similar changes for the video delivery.

There is this guide written by Panda OS:

It is a bit outdated and changes to the infra were made since it was written but it can be a good starting point.

I highly recommend experimenting with this on a test ENV and not on Production, since making such changes can break current operations.

Hope this helps,