Kaltura analytics stopped working/GEO Location feature unavailable

Community Forums Backend Server Support
1

Hello, I am running Kaltura on a single server, CentOS 7, latest versions of both.
Initially the analytics were running OK, however after a few days they stopped updating.
I have checked other forum entries relating to Analytics but none seem to help with my situation.
The DB locks are at 0.
When manually running kaltura-run-dwh.sh, kaltlog gives out multiple of the following:
==> /opt/kaltura/log/kaltura_scripts.log <==
2020-07-16 10:03:14 [1399814226] [updateEntryPlaysViews.php] [global] ERR: Exception: Couldn’t find entry [0_5kbs4uw7] in /opt/kaltura/app/infra/log/KalturaLog.php:88
Stack trace:
#0 /opt/kaltura/app/alpha/scripts/dwh/updateEntryPlaysViews.php(19): KalturaLog::err(‘Couldn’t find e…’)

2020-07-16 10:03:14 [KalturaStatement->execute] DEBUG: Sql took - 0.0006248950958252 seconds
2020-07-16 10:03:14 [global] ERR: Exception: Couldn’t find entry [0_72qiik00] in /opt/kaltura/app/infra/log/KalturaLog.php:88
Stack trace:
#0 /opt/kaltura/app/alpha/scripts/dwh/updateEntryPlaysViews.php(19): KalturaLog::err(‘Couldn’t find e…’)

Same with dwh_plays_views_sync.sh
2020-07-16 10:08:09 [153274352] [updateEntryPlaysViews.php] [global] ERR: Exception: Couldn’t find entry [0_qjz96if7] in /opt/kaltura/app/infra/log/Kaltug.php:88
Stack trace:
#0 /opt/kaltura/app/alpha/scripts/dwh/updateEntryPlaysViews.php(19): KalturaLog::err(‘Couldn’t find e…’)
#1 {main}

Is there a way to force Kaltura to update its index DB or do I have to drop the DWH DB and start from scratch? I have not done so yet as I tried this on another server and afterwards kaltura-run-dwh.sh could not run properly:
ERROR 1305 (42000) at line 1: PROCEDURE kalturadw.get_data_for_operational does not exist
ERROR 1305 (42000) at line 1: PROCEDURE kalturadw.mark_operational_sync_as_done does not exist
kaltura-dwh-config.sh was run after kaltura-drop-dwh.sh

I also have another Analytics problem that I could not find on the forums, the GEO Location feature does not work properly, when I click on a specific country, which has data in it, I receive: ERROR Internal server error occurred, kaltlog shows:

2020-07-16 10:24:09 [0.000136] [89.136.235.146] [260354244] [25] [API] [myReportsMgr::executeQueryByType] NOTICE: Cannot find mapping for []
2020-07-16 10:24:09 [0.001010] [89.136.235.146] [260354244] [26] [API] [KalturaFrontController->getExceptionObject] CRIT: Exception: Cannot find mapping for [] in /opt/kaltura/app/alpha/apps/kaltura/lib/reports/myReportsSqlFileMgr.class.php:664
Stack trace:
#0 /opt/kaltura/app/alpha/apps/kaltura/lib/reports/myReportsSqlFileMgr.class.php(32): myReportsSqlFileMgr::getFileNameMappingConfig(NULL, ‘count’, ‘_no_text’, ‘’, false, ‘’)

2020-07-16 10:24:09 [0.000161] [89.136.235.146] [260354244] [34] [API] [myReportsMgr::executeQueryByType] NOTICE: Cannot find mapping for []
2020-07-16 10:24:09 [0.000236] [89.136.235.146] [260354244] [35] [API] [KalturaFrontController->getExceptionObject] CRIT: Exception: Cannot find mapping for [] in /opt/kaltura/app/alpha/apps/kaltura/lib/reports/myReportsSqlFileMgr.class.php:664
Stack trace:
#0 /opt/kaltura/app/alpha/apps/kaltura/lib/reports/myReportsSqlFileMgr.class.php(32): myReportsSqlFileMgr::getFileNameMappingConfig(NULL, ‘total’, ‘_no_text’, ‘’, false, ‘’)

2

Hello @Andi_OC,

With regards to geo location detection, that’s done using IP2location so in order for it to work, you must download the IP2location CSV and place it under /opt/kaltura/tmp/IP-COUNTRY-REGION-CITY-ISP-DOMAIN.CSV (or edit /opt/kaltura/dwh/.kettle/kettle.properties and set IP2LocationFile to a different path).

As to your other issue, what do you see in /opt/kaltura/dwh/logs?

3

Hello, regarding GEO IP Location, I’ve downloaded the CSV file from here:
https://lite.ip2location.com/database/ip-country and uploaded/renamed as instructed, as well as chmod/chown to follow the format of the rest of the files, however I receive the same internal error message, kaltlog only displays the following:
2020-07-22 09:46:17 [0.000157] [89.136.235.146] [86214746] [25] [API] [myReportsMgr::executeQueryByType] NOTICE: Cannot find mapping for []
2020-07-22 09:46:17 [0.000617] [89.136.235.146] [86214746] [26] [API] [KalturaFrontController->getExceptionObject] CRIT: Exception: Cannot find mapping for [] in /opt/kaltura/app/alpha/apps/kaltura/lib/reports/myReportsSqlFileMgr.class.php:664
Stack trace:
#0 /opt/kaltura/app/alpha/apps/kaltura/lib/reports/myReportsSqlFileMgr.class.php(32): myReportsSqlFileMgr::getFileNameMappingConfig(NULL, ‘count’, ‘_no_text’, ‘’, false, ‘’)

2020-07-22 09:46:17 [0.000105] [89.136.235.146] [86214746] [34] [API] [myReportsMgr::executeQueryByType] NOTICE: Cannot find mapping for []
2020-07-22 09:46:17 [0.000146] [89.136.235.146] [86214746] [35] [API] [KalturaFrontController->getExceptionObject] CRIT: Exception: Cannot find mapping for [] in /opt/kaltura/app/alpha/apps/kaltura/lib/reports/myReportsSqlFileMgr.class.php:664
Stack trace:
#0 /opt/kaltura/app/alpha/apps/kaltura/lib/reports/myReportsSqlFileMgr.class.php(32): myReportsSqlFileMgr::getFileNameMappingConfig(NULL, ‘total’, ‘_no_text’, ‘’, false, ‘’)

As for the logs, it seems that the Kaltura cron is not running, the last update was when I ran /opt/kaltura/bin/kaltura-run-dwh.sh manually. The following error was found:
2020/07/16 10:01:41 - Write to log.0 - Javascript error:
2020/07/16 10:01:41 - Write to log.0 - TypeError: [JavaPackage eu.bitwalker.useragentutils.UserAgent.parseUserAgentString] is not a function, it is object. (script#7)
2020/07/16 10:01:41 - Write to log.0 -
2020/07/16 10:01:41 - Write to log.0 - null
2020/07/16 10:01:41 - Write to log.0 - fields = null
2020/07/16 10:01:41 - Write to log.0 - codes = SCR-001

The kaltura-run-dwh.sh script seems to give out the following error:
bash: /opt/kaltura/log/cron.log: Permission denied
untill I manually chmod it.

After the script is run, the logs no longer show any errors.

I forgot to mention initially:
use kalturadw;
select * from dwh_entry_plays_views;
Has no entry newer than 2020-06-14, I believe this is the first update done to Kaltura since the install
/opt/kaltura/bin/kaltura-config-all.sh was run after every update to Kaltura
/opt/kaltura/bin/kaltura-sanity.sh reports all OK, except for DWH

I also tried running the scripts from /opt/kaltura/dwh/etlsource/execute/
I am not sure if relevant however in etl_logs.sh
ROOT_DIR=/home/etl/
ETLHOME=${ROOT_DIR}
The path is not in use on the server

4

I decided to try dumping the DWH DB after performing an sqldump, however I am unable to run
/opt/kaltura/dwh/setup/dwh_drop_databases.sh
The error received is:
ERROR 1045 (28000): Access denied for user ‘etl’@‘localhost’ (using password: NO)
1
Error - bailing out!

From the code I am not sure from where it gets the password for user etl.
I was able to manually connect to the db with user etl and its password from /etc/kaltura.d/system.ini
Also tried to set the password as an argument but it doesnt seem to be configured in this way.
Neither adding an PW field in the code with the correct password.
/opt/kaltura/dwh/logs/dwh_drop_databases.log is empty

I eventually just dropped the databases, however then:
/opt/kaltura/dwh/setup/dwh_setup.sh
1
Error - bailing out!

The DBs were restored from the dump.

5

One last thing, I see a lot of these errors:
2020-07-26 10:09:33 [0.000178] [162.244.81.56] [2039716361] [340] [API] [elasticClient->sendRequest] ERR: Exception: Elastic client curl error code[6] message[Could not resolve host: ELASTIC_HOST@; Unknown error] in /opt/kaltura/app/infra/log/KalturaLog.php:88
Stack trace:

But I am not sure how elastic search is supposed to work with Kaltura or if these messages are relevant. ElasticSearch is not running on the server.

6

thanks for the awesome information.