Entry Count discrepancy

Hey all,

I’m working on an old Kaltura 5 installation (single app server installation) and I’ve noticed the following. Can you help me explain it?

Consider the following php code snippet.

....
$type = KalturaSessionType::ADMIN;
$config = new KalturaConfiguration($partnerId);
$config->serviceUrl = $argv[3];
$client = new KalturaClient($config);
$ks = $client->session->start($secret, $userId, $type, $partnerId, $expiry, $privileges);
$client->setKs($ks);
$filter = new KalturaBaseEntryFilter();
$total_media_entries = $client->media->count($filter);
echo "\nTotal Media Entries: ".$total_media_entries;

This displays a count of 1078 entries.

When I look in KMC, I see this number next to All categories

I am assuming these numbers should be the same. Perhaps this is a false assumption.

If they’re supposed to be the same, what might cause them not to be? If they’re not supposed to be the same, where is the difference?

I’ve re-indexed sphinx by following the shell script logic that can be found in newer versions. Essentially stop the searchd daemon and watcher, move the sphinx data and binlog files. Start the searchd daemon again and run the populate PHP scripts. When I did this, these numbers changed but they did not sync.

Thanks for the help!

Hi @david.hahn1,

KMC calls baseentry->list(), not media->list() but in most cases, the result would be the same as most entries are of type media anyhow so I doubt calling baseentry->list() would result in a different count being returned.

When you say you have re-indexed Sphinx, can you please specify all the scripts you have executed? Also, do you only have one Sphinx instance? My guess is you have more than one and some of your requests hit one and some the other.
If you’ve only re-indexed one of them, that would explain the discrepancy and you should do the same for all instances.

Additionally, is the kaltura_populate script running on all your Sphinx instances?

Hey @jess

Thanks for the info on the differences in the API interaction. I agree that generally they’ll be the same.

As far as re-indexing sphinx, I didn’t use the scripts in the RPM packages because they weren’t installed. I didn’t install them because I thought they likely wouldn’t function on such an old installation. So, I took a look at the kaltura-sphinx-reindex.sh script and mimic’d it’s behavior with some shell commands. Here is what I did.

./sphinx_watch.sh stop  

(this didn’t seem to actually stop searchd, so I killed the processes manually. It did stop PopulateFromLog.php)

mv /opt/kaltura/sphinx/kaltura_* /opt/sphinx-bak
mv /opt/kaltura/log/sphinx/data/binlog.* /opt/sphinx-bak
/etc/init.d/sphinx_watch.sh start
php /opt/kaltura/app/deployment/base/scripts/populateSphinxCuePoints.php
php /opt/kaltura/app/deployment/base/scripts/populateSphinxEntries.php
php /opt/kaltura/app/deployment/base/scripts/populateSphinxEntryDistributions.php

There is only one sphinx node because this is an all in one deployment.

The PopulateFromLog php script is running on the server. Maybe this will help:

[root@omaqi-kltw1 admin_ddhahn]# ps -ef | grep sphinx
root      2658     1  0 Jun20 ?        00:15:00 /bin/bash /opt/kaltura/app/plugins/sphinx_search/scripts/watch.daemon.onprem.sh
root     11537     1  0 13:37 ?        00:00:18 php populateFromLog.php     /opt/kaltura/app/plugins/sphinx_search/scripts/configs/server-sphinx.php
root     13064     1  0 13:49 ?        00:00:00 /opt/kaltura/bin/sphinx/searchd --config     /opt/kaltura/app/configurations/sphinx/kaltura.conf
root     13066 13064  0 13:49 ?        00:00:17 /opt/kaltura/bin/sphinx/searchd --config opt/kaltura/app/configurations/sphinx/kaltura.conf
root     15990     1  0 14:12 ?        00:00:02 /bin/bash opt/kaltura/app/plugins/sphinx_search/scripts/watch.daemon.onprem.sh
root     16008     1  0 14:12 ?        00:00:02 /bin/bash opt/kaltura/app/plugins/sphinx_search/scripts/watch.daemon.onprem.sh
root     17303 10699  0 20:09 pts/0    00:00:00 grep sphinx

I’m working on a clone of our current production machine. I’m using it to test my migration project out.

edit:

If I try to retrieve baseEntry objects, I get 1082 as the count. Closer to 1085 but not quite.