Cannot get mediaProxy.entry.categories on player API

felipe.borrero · November 24, 2017, 1:48am

Hi

I am having a very strange problem. Since we setup a DropFolder and ingested some video entries, we are no able to use the mediaProxy.entry.categories attribute.

It only happens for a specific partner ID

I have checked the database and everything looks ok.

I have one video entry associated to category ID 61 as shown in the query below

    select e.id,e.name, d.category_id,d.status from entry e, category_entry d where e.id = d.entry_id and e.partner_id = XXX;
+------------+-------------------------+-------------+--------+
| id         | name                    | category_id | status |
+------------+-------------------------+-------------+--------+
| 0_xmrjls3w | En vivo NoticiasRCN 01  |          35 |      2 |
| 0_wbfqlpt7 | En vivo CanalRCN 01     |          35 |      2 |
| 0_6g348xkw | En vivo 02              |          35 |      2 |
| 0_9bngl8hb | En Vivo DeportesRCN HD2 |          35 |      2 |
| 0_iokziu83 | En Vivo Infantil        |          35 |      2 |
| 0_4jeh0s9v | Aidita90 - Cellular     |          61 |      2 |
+------------+-------------------------+-------------+--------+

But when using the Kaltura Test Api and trying to get EntryID 0_4jeh0s9v this is what I get:

 <result>
        <objectType>KalturaMediaEntry</objectType>
        <relatedObjects>
        </relatedObjects>
        <id>0_4jeh0s9v</id>
        <name>Aidita90 - Cellular</name>
        <description>
        </description>
        <partnerId>XXX</partnerId>
        <userId>XXXXX</userId>
        <creatorId>XXXXX</creatorId>
        <tags>
        </tags>
        <adminTags>
        </adminTags>
        <categories>
        </categories>
        <categoriesIds>
        </categoriesIds>
        <status>2</status>
        <moderationStatus>6</moderationStatus>
        <moderationCount>0</moderationCount>

As you might see, the categories and categoriesID elements are not populated with any thing

If I try to get mediaProxy.entry.name, it works but for categories it is not working.

I have already run kaltura-sphinx-reindex.sh but the problem still exist.

Can any one give me a hint on where to look for?

Jess, this one is for you.

Any help will be appreciated

Regards,

Felipe Borrero
Operations Director
Calipso Comunicaciones S.A.

jess · November 24, 2017, 11:39am

Hi Felipe,

My first guess is that you have content entitlement enabled for this partner.
What does:

mysql> select * from category_entry where entry_id='0_4jeh0s9v';

return?

If indeed, content entitlement is enabled, you can set:

$privileges = "disableentitlement"

when generating a KS and then make the same request.
This is what KMC does, thus bypassing the entitlement mechanism.

If that does not help, first check in KMC whether you see this entry attached to the category in question, if you do, obviously it is not a Sphinx issue, if you don’t see it attached to the category in question in KMC, make the request again and check the queries that go to both MySQL and Sphinx in /opt/kaltura/log/kaltura_api_v3.log and let’s see where it goes wrong…

felipe.borrero · November 24, 2017, 2:43pm

Hi Jess,

This is what I get from the query:

+-----+------------+------------+-------------+---------------------+-------------+---------------------+-------------------+--------+-----------------+------------------+
| id  | partner_id | entry_id   | category_id | created_at          | custom_data | updated_at          | category_full_ids | status | privacy_context | creator_kuser_id |
+-----+------------+------------+-------------+---------------------+-------------+---------------------+-------------------+--------+-----------------+------------------+
| 177 |        101 | 0_4jeh0s9v |          61 | 2017-11-23 18:20:23 | NULL        | 2017-11-23 18:20:23 | 61                |      2 | NULL            |                9 |
+-----+------------+------------+-------------+---------------------+-------------+---------------------+-------------------+--------+-----------------+------------------+

We set custom-data to null before as we were comparing entries for another partner.

We assigned a new category to a different entry and this is what the query returns:

 select * from category_entry where entry_id='0_gdo4msd4';
+-----+------------+------------+-------------+---------------------+----------------------------------------------------------------------+---------------------+-------------------+--------+-----------------+------------------+
| id  | partner_id | entry_id   | category_id | created_at          | custom_data                                                          | updated_at          | category_full_ids | status | privacy_context | creator_kuser_id |
+-----+------------+------------+-------------+---------------------+----------------------------------------------------------------------+---------------------+-------------------+--------+-----------------+------------------+
| 179 |        101 | 0_gdo4msd4 |          63 | 2017-11-23 20:24:46 | a:1:{s:14:"creatorPuserId";s:32:"cloudvideo-canalrcnhd@cdn.net.co";} | 2017-11-23 20:24:46 | 63                |      2 | NULL            |                9 |
+-----+------------+------------+-------------+---------------------+----------------------------------------------------------------------+---------------------+-------------------+--------+-----------------+------------------+

How can I confirm that content entitlement is enabled and how can we disable it globally for the partner?

Regars

Felipe Borrero
Operations Director
Calipso Comunicaciones S.A.

felipe.borrero · November 24, 2017, 3:23pm

Hi Jess

I have just found this snippet within the customdata in table partner

s:11:"entitlement";a:1:{s:29:"defaultEntitlementEnforcement";b:0;}

Does this mean that the partner has entitlements enabled?

How hav we disable them?

Regards,

Felipe Borrero
Operations Director
Calipso Comunicaciones S.A.

jess · November 24, 2017, 6:51pm

Hi @felipe.borrero,

If passing “disableentitlement” as privileges when generating a Kaltura session [KS] and then using that KS when making the request does not return the categories then it’s not an entitlement issue.

Also, the entitlement enforcement mechanism requires that you set the privacy_context which, according to your output above is NULL so, this is most probably not the issue at all.

Either way, like I said, KMC bypasses the content entitlement enforcement, even when it is enabled [also by passing “disableentitlement” as privileges].
Do you see this entry as attached to the category ID in question [61] in KMC or not?
If not, it’s time to check the queries going down to MySQL and Sphinx when making the request. Like I said, these can be found in /opt/kaltura/log/kaltura_api_v3.log.

As a side note, if you’re interested in the content entitlement mechanism, you can read more about it here:
https://vpaas.kaltura.com/documentation/Secure_Control_and_Govern/Content-Categories-Management.html#configure-and-use-content-entitlements

Even though it seems unrelated to your problem, to answer your question, you can check whether “Default Entitlement Enforcement” is enabled by going to Admin Console->Publishers->Your Partner->Configure

Just for general knowledge, the partner.custom_data value is the result of calling PHP’s serialize() so, you can call unserialize() to get the values, for example, here is my partner.custom_data:

'a:71:{s:9:"firstName";s:12:"Jess Portnoy";s:12:"isFirstLogin";b:0;s:24:"i18n_template_partner_id";i:99;s:23:"admin_login_users_quota";d:10;s:19:"role_cache_dirty_at";i:1511370003;s:26:"defaultConversionProfileId";i:9;s:16:"featuresStatuses";a:0:{}s:22:"account_owner_kuser_id";i:9;s:24:"storageDeleteFromKaltura";i:1;s:20:"storageServePriority";i:3;s:22:"verticalClasiffication";s:0:"";s:28:"partnerPackageClassOfService";s:0:"";s:8:"language";s:2:"en";s:20:"delivery_profile_ids";a:0:{}s:25:"live_delivery_profile_ids";a:0:{}s:39:"shouldApplyAccessControlOnEntryMetadata";b:0;s:4:"host";s:1:"0";s:7:"cdnHost";s:1:"0";s:13:"thumbnailHost";s:1:"0";s:14:"defThumbOffset";i:3;s:28:"importRemoteSourceForConvert";b:0;s:22:"allowMultiNotification";b:0;s:20:"login_blocked_period";i:0;s:26:"num_prev_passwords_to_keep";i:0;s:21:"password_replace_freq";i:432000000;s:15:"enforceDelivery";b:0;s:28:"extendedFreeTrailEndsWarning";b:0;s:27:"extendedFreeTrailExpiryDate";i:0;s:17:"extendedFreeTrail";i:0;s:21:"restrictThumbnailByKs";i:0;s:25:"supportAnimatedThumbnails";b:0;s:35:"enableBulkUploadNotificationsEmails";b:0;s:28:"bulkUploadNotificationsEmail";s:16:"jess@kaltura.com";s:11:"internalUse";b:0;s:37:"default_live_stream_entry_source_type";s:2:"29";s:11:"entitlement";a:1:{s:29:"defaultEntitlementEnforcement";b:1;}s:20:"cache_flavor_version";i:0;s:19:"defaultDeliveryType";s:4:"auto";s:20:"defaultEmbedCodeType";s:4:"auto";s:19:"customDeliveryTypes";a:0:{}s:21:"disabledDeliveryTypes";a:0:{}s:21:"timeAlignedRenditions";b:0;s:21:"htmlPurifierBehaviour";i:0;s:25:"htmlPurifierBaseListUsage";b:0;s:18:"max_login_attempts";d:5000;s:15:"bandwidth_quota";d:0;s:23:"bandwidth_overage_price";d:0;s:22:"bandwidth_overage_unit";d:0;s:15:"monthly_storage";d:0;s:29:"monthly_storage_overage_price";d:0;s:28:"monthly_storage_overage_unit";d:0;s:29:"monthly_storage_and_bandwidth";d:0;s:43:"monthly_storage_and_bandwidth_overage_price";d:0;s:42:"monthly_storage_and_bandwidth_overage_unit";d:0;s:31:"admin_login_users_overage_price";N;s:30:"admin_login_users_overage_unit";N;s:16:"publishers_quota";d:0;s:24:"publishers_overage_price";N;s:23:"publishers_overage_unit";N;s:20:"stream_entries_quota";d:0;s:28:"stream_entries_overage_price";N;s:27:"stream_entries_overage_unit";N;s:9:"end_users";N;s:23:"end_users_overage_price";N;s:22:"end_users_overage_unit";N;s:13:"entries_quota";d:0;s:21:"entries_overage_price";N;s:20:"entries_overage_unit";N;s:15:"access_controls";d:24;s:18:"live_stream_inputs";N;s:18:"e_search_languages";a:0:{}}'

so calling this little snippet:

var_dump(unserialize('a:71:{s:9:"firstName";s:12:"Jess Portnoy";s:12:"isFirstLogin";b:0;s:24:"i18n_template_partner_id";i:99;s:23:"admin_login_users_quota";d:10;s:19:"role_cache_dirty_at";i:1511370003;s:26:"defaultConversionProfileId";i:9;s:16:"featuresStatuses";a:0:{}s:22:"account_owner_kuser_id";i:9;s:24:"storageDeleteFromKaltura";i:1;s:20:"storageServePriority";i:3;s:22:"verticalClasiffication";s:0:"";s:28:"partnerPackageClassOfService";s:0:"";s:8:"language";s:2:"en";s:20:"delivery_profile_ids";a:0:{}s:25:"live_delivery_profile_ids";a:0:{}s:39:"shouldApplyAccessControlOnEntryMetadata";b:0;s:4:"host";s:1:"0";s:7:"cdnHost";s:1:"0";s:13:"thumbnailHost";s:1:"0";s:14:"defThumbOffset";i:3;s:28:"importRemoteSourceForConvert";b:0;s:22:"allowMultiNotification";b:0;s:20:"login_blocked_period";i:0;s:26:"num_prev_passwords_to_keep";i:0;s:21:"password_replace_freq";i:432000000;s:15:"enforceDelivery";b:0;s:28:"extendedFreeTrailEndsWarning";b:0;s:27:"extendedFreeTrailExpiryDate";i:0;s:17:"extendedFreeTrail";i:0;s:21:"restrictThumbnailByKs";i:0;s:25:"supportAnimatedThumbnails";b:0;s:35:"enableBulkUploadNotificationsEmails";b:0;s:28:"bulkUploadNotificationsEmail";s:16:"jess@kaltura.com";s:11:"internalUse";b:0;s:37:"default_live_stream_entry_source_type";s:2:"29";s:11:"entitlement";a:1:{s:29:"defaultEntitlementEnforcement";b:1;}s:20:"cache_flavor_version";i:0;s:19:"defaultDeliveryType";s:4:"auto";s:20:"defaultEmbedCodeType";s:4:"auto";s:19:"customDeliveryTypes";a:0:{}s:21:"disabledDeliveryTypes";a:0:{}s:21:"timeAlignedRenditions";b:0;s:21:"htmlPurifierBehaviour";i:0;s:25:"htmlPurifierBaseListUsage";b:0;s:18:"max_login_attempts";d:5000;s:15:"bandwidth_quota";d:0;s:23:"bandwidth_overage_price";d:0;s:22:"bandwidth_overage_unit";d:0;s:15:"monthly_storage";d:0;s:29:"monthly_storage_overage_price";d:0;s:28:"monthly_storage_overage_unit";d:0;s:29:"monthly_storage_and_bandwidth";d:0;s:43:"monthly_storage_and_bandwidth_overage_price";d:0;s:42:"monthly_storage_and_bandwidth_overage_unit";d:0;s:31:"admin_login_users_overage_price";N;s:30:"admin_login_users_overage_unit";N;s:16:"publishers_quota";d:0;s:24:"publishers_overage_price";N;s:23:"publishers_overage_unit";N;s:20:"stream_entries_quota";d:0;s:28:"stream_entries_overage_price";N;s:27:"stream_entries_overage_unit";N;s:9:"end_users";N;s:23:"end_users_overage_price";N;s:22:"end_users_overage_unit";N;s:13:"entries_quota";d:0;s:21:"entries_overage_price";N;s:20:"entries_overage_unit";N;s:15:"access_controls";d:24;s:18:"live_stream_inputs";N;s:18:"e_search_languages";a:0:{}}'));

would return:

array(71) {
  ["firstName"]=>
  string(12) "Jess Portnoy"
  ["isFirstLogin"]=>
  bool(false)
  ["i18n_template_partner_id"]=>
  int(99)
  ["admin_login_users_quota"]=>
  float(10)
  ["role_cache_dirty_at"]=>
  int(1511370003)
  ["defaultConversionProfileId"]=>
  int(9)
  ["featuresStatuses"]=>
  array(0) {
  }
  ["account_owner_kuser_id"]=>
  int(9)
  ["storageDeleteFromKaltura"]=>
  int(1)
  ["storageServePriority"]=>
  int(3)
  ["verticalClasiffication"]=>
  string(0) ""
  ["partnerPackageClassOfService"]=>
  string(0) ""
  ["language"]=>
  string(2) "en"
  ["delivery_profile_ids"]=>
  array(0) {
  }
  ["live_delivery_profile_ids"]=>
  array(0) {
  }
  ["shouldApplyAccessControlOnEntryMetadata"]=>
  bool(false)
  ["host"]=>
  string(1) "0"
  ["cdnHost"]=>
  string(1) "0"
  ["thumbnailHost"]=>
  string(1) "0"
  ["defThumbOffset"]=>
  int(3)
  ["importRemoteSourceForConvert"]=>
  bool(false)
  ["allowMultiNotification"]=>
  bool(false)
  ["login_blocked_period"]=>
  int(0)
  ["num_prev_passwords_to_keep"]=>
  int(0)
  ["password_replace_freq"]=>
  int(432000000)
  ["enforceDelivery"]=>
  bool(false)
  ["extendedFreeTrailEndsWarning"]=>
  bool(false)
  ["extendedFreeTrailExpiryDate"]=>
  int(0)
  ["extendedFreeTrail"]=>
  int(0)
  ["restrictThumbnailByKs"]=>
  int(0)
  ["supportAnimatedThumbnails"]=>
  bool(false)
  ["enableBulkUploadNotificationsEmails"]=>
  bool(false)
  ["bulkUploadNotificationsEmail"]=>
  string(16) "jess@kaltura.com"
  ["internalUse"]=>
  bool(false)
  ["default_live_stream_entry_source_type"]=>
  string(2) "29"
  ["entitlement"]=>
  array(1) {
    ["defaultEntitlementEnforcement"]=>
    bool(true)
  }
  ["cache_flavor_version"]=>
  int(0)
  ["defaultDeliveryType"]=>
  string(4) "auto"
  ["defaultEmbedCodeType"]=>
  string(4) "auto"
  ["customDeliveryTypes"]=>
  array(0) {
  }
  ["disabledDeliveryTypes"]=>
  array(0) {
  }
  ["timeAlignedRenditions"]=>
  bool(false)
  ["htmlPurifierBehaviour"]=>
  int(0)
  ["htmlPurifierBaseListUsage"]=>
  bool(false)
  ["max_login_attempts"]=>
  float(5000)
  ["bandwidth_quota"]=>
  float(0)
  ["bandwidth_overage_price"]=>
  float(0)
  ["bandwidth_overage_unit"]=>
  float(0)
  ["monthly_storage"]=>
  float(0)
  ["monthly_storage_overage_price"]=>
  float(0)
  ["monthly_storage_overage_unit"]=>
  float(0)
  ["monthly_storage_and_bandwidth"]=>
  float(0)
  ["monthly_storage_and_bandwidth_overage_price"]=>
  float(0)
  ["monthly_storage_and_bandwidth_overage_unit"]=>
  float(0)
  ["admin_login_users_overage_price"]=>
  NULL
  ["admin_login_users_overage_unit"]=>
  NULL
  ["publishers_quota"]=>
  float(0)
  ["publishers_overage_price"]=>
  NULL
  ["publishers_overage_unit"]=>
  NULL
  ["stream_entries_quota"]=>
  float(0)
  ["stream_entries_overage_price"]=>
  NULL
  ["stream_entries_overage_unit"]=>
  NULL
  ["end_users"]=>
  NULL
  ["end_users_overage_price"]=>
  NULL
  ["end_users_overage_unit"]=>
  NULL
  ["entries_quota"]=>
  float(0)
  ["entries_overage_price"]=>
  NULL
  ["entries_overage_unit"]=>
  NULL
  ["access_controls"]=>
  float(24)
  ["live_stream_inputs"]=>
  NULL
  ["e_search_languages"]=>
  array(0) {
  }
}

The part relevant to entitlement is:

  ["entitlement"]=>
  array(1) {
    ["defaultEntitlementEnforcement"]=>
    bool(true)
  }

felipe.borrero · November 28, 2017, 12:19am

Hi Jess

Many thanks for your help.

It seems like the problem was related to an inconsistent data as we were also working with DropFolder ingestion and some categories were created from the batch processing.

Looks like those categories conflicted with the Kaltura created ones as they were not totally defined.

Once the xml files for the DropFolder were fixed, we had no trouble again.

Nevertheless, we had to delete all categories, even the once we had created before and the recreate them again from scratch to solve the problem.

It seems we had two different issues colliding in this case.

Regards

Felipe Borrero
Operations Director
Calipso Comunicaciones S.A.

jess · November 28, 2017, 11:22am

Hi @felipe.borrero,

Welcome, of course. How many Sphinx instances do you have? If more than one, perhaps they were out of sync?
Certainly worth ensuring they are all synchronised and that the kaltura-populate daemon is running.