hi
Installation of Kaltura CE 10.8 is ok, but we have no stat. I think the trouble is the empty file:
/opt/kaltura/app/alpha/config/databases.yml
Can you show me an example of this file ?
thx,
Frat
Hello,
This is not the problem, the file should be empty.
To start troubleshooting:
0. Are you using debs or RPMs?
- is the file /opt/kaltura/dwh/.kettle/kettle.properties properly populated with the creds and DB host?
- are there any errors in /opt/kaltura/dwh/logs or in the cron log? which for RHEL based systems will be written to /var/log/cron and on deb based ones will reside under /var/log/syslog by default?
After we have the answer for these questions, we’ll be smarter:)
We use rpm as the server runs centos.
the file kettle.properties seems ok, i’ve checked the db entries and it’s ok.
they are some locks trouble again, I’ve run “UPDATE kalturadw_ds.locks set lock_state = 0;” again. May I do this every day ?
no error in /var/log/cron
thx for the reply, as I can check db credentials. We wait for the locks.
Frat
The lock would typically stay there if there process did not exist correctly… any errors in /opt/kaltura/dwh/logs/?
Hi, there is no error in log:
etl_update_dims-20150707-17.log:ERROR 07-07 17:00:05,521 - Lock is already seized - Aborting after having seen 1 rows.
etl_update_dims-20150707-17.log:ERROR 07-07 17:00:05,521 - seize_lock_by_name - Errors detected!
etl_update_dims-20150707-17.log:ERROR 07-07 17:00:05,522 - seize_lock_by_name - Errors detected!
etl_update_dims-20150707-18.log:table_name = batch_job_error_type
etl_update_dims-20150707-19.log:table_name = batch_job_error_type
etl_update_dims-20150707-20.log:table_name = batch_job_error_type
etl_update_dims-20150707-21.log:table_name = batch_job_error_type
etl_update_dims-20150707-22.log:ERROR 07-07 22:00:05,248 - Lock is already seized - Row nr 1 causing abort : [update_dims_lock], [1], [0], [null]
etl_update_dims-20150707-22.log:ERROR 07-07 22:00:05,249 - Lock is already seized - Aborting after having seen 1 rows.
etl_update_dims-20150707-22.log:ERROR 07-07 22:00:05,253 - seize_lock_by_name - Errors detected!
etl_update_dims-20150707-22.log:ERROR 07-07 22:00:05,254 - seize_lock_by_name - Errors detected!
As you can see, 3 logs are ok (finished by “INFO 07-07 18:00:08,530 - Kitchen - Finished!”) but not etl_update_dims-20150707-21.log. Here the end of this log file:
.====================
INFO 07-07 21:00:06,349 - Write to log - Finished processing (I=0, O=0, R=32, W=32, U=0, E=0)
INFO 07-07 21:00:06,396 - Update Partners - Dispatching started for transformation [Update Partners]
INFO 07-07 21:00:06,532 - parse_partner_custom_data - Dispatching started for transformation [parse_partner_custom_data]
INFO 07-07 21:00:06,564 - Read Table Partners - Finished reading query, closing connection.
INFO 07-07 21:00:06,575 - input - Finished processing (I=0, O=0, R=1, W=1, U=0, E=0)
INFO 07-07 21:00:06,695 - update_entries - Dispatching started for transformation [update_entries]
INFO 07-07 21:00:06,800 - Read Table entry - Finished reading query, closing connection.
INFO 07-07 21:00:06,812 - input - Finished processing (I=0, O=0, R=1, W=1, U=0, E=0)
INFO 07-07 21:00:06,887 - update_flavor_asset - Dispatching started for transformation [update_flavor_asset]
INFO 07-07 21:00:06,944 - Read Flavor Asset - Finished reading query, closing connection.
INFO 07-07 21:00:06,958 - input - Finished processing (I=0, O=0, R=1, W=1, U=0, E=0)
INFO 07-07 21:00:07,015 - update_file_sync - Dispatching started for transformation [update_file_sync]
INFO 07-07 21:00:07,118 - Attempt to insert to flavor asset - Connected to database [KalturaDW] (commit=1000)
INFO 07-07 21:00:07,163 - Table input - Finished reading query, closing connection.
INFO 07-07 21:00:07,167 - input - Finished processing (I=0, O=0, R=1, W=1, U=0, E=0)
INFO 07-07 21:00:07,208 - update_media_info - Dispatching started for transformation [update_media_info]
INFO 07-07 21:00:07,287 - Read Media Info - Finished reading query, closing connection.
INFO 07-07 21:00:07,293 - input - Finished processing (I=0, O=0, R=1, W=1, U=0, E=0)
INFO 07-07 21:00:07,344 - update_flavor_params - Dispatching started for transformation [update_flavor_params]
INFO 07-07 21:00:07,401 - Read flavor params - Finished reading query, closing connection.
INFO 07-07 21:00:07,419 - input - Finished processing (I=0, O=0, R=1, W=1, U=0, E=0)
INFO 07-07 21:00:07,457 - update_flavor_params_output - Dispatching started for transformation [update_flavor_params_output]
INFO 07-07 21:00:07,471 - Attempt to insert to flavor params - Connected to database [KalturaDW] (commit=1000)
INFO 07-07 21:00:07,483 - Read Flavor Params Output - Finished reading query, closing connection.
INFO 07-07 21:00:07,485 - input - Finished processing (I=0, O=0, R=1, W=1, U=0, E=0)
INFO 07-07 21:00:07,504 - Update new flavor params - Finished reading query, closing connection.
INFO 07-07 21:00:07,514 - update_category - Dispatching started for transformation [update_category]
INFO 07-07 21:00:07,534 - Read Table category - Finished reading query, closing connection.
INFO 07-07 21:00:07,535 - input - Finished processing (I=0, O=0, R=1, W=1, U=0, E=0)
INFO 07-07 21:00:07,554 - update_category_entry - Dispatching started for transformation [update_category_entry]
INFO 07-07 21:00:07,573 - Read Table category_entry - Finished reading query, closing connection.
INFO 07-07 21:00:07,576 - input - Finished processing (I=0, O=0, R=1, W=1, U=0, E=0)
INFO 07-07 21:00:07,600 - Update Locations - Dispatching started for transformation [Update Locations]
INFO 07-07 21:00:07,649 - Table input - Finished reading query, closing connection.
INFO 07-07 21:00:07,652 - input - Finished processing (I=0, O=0, R=1, W=1, U=0, E=0)
INFO 07-07 21:00:07,683 - Update Kusers - Dispatching started for transformation [Update Kusers]
INFO 07-07 21:00:07,711 - Read Table Kusers - Finished reading query, closing connection.
INFO 07-07 21:00:07,713 - input - Finished processing (I=0, O=0, R=1, W=1, U=0, E=0)
INFO 07-07 21:00:07,737 - Update UI Conf - Dispatching started for transformation [Update UI Conf]
INFO 07-07 21:00:07,754 - Read Table UI Conf - Finished reading query, closing connection.
INFO 07-07 21:00:07,760 - input - Finished processing (I=0, O=0, R=1, W=1, U=0, E=0)
INFO 07-07 21:00:07,781 - Update widget - Dispatching started for transformation [Update widget]
INFO 07-07 21:00:07,789 - Read Table widget - Finished reading query, closing connection.
INFO 07-07 21:00:07,791 - input - Finished processing (I=0, O=0, R=1, W=1, U=0, E=0)
INFO 07-07 21:00:07,810 - Update Convertsion Profile - Dispatching started for transformation [Update Convertsion Profile]
INFO 07-07 21:00:07,819 - Read Conversion Profile - Finished reading query, closing connection.
INFO 07-07 21:00:07,821 - input - Finished processing (I=0, O=0, R=1, W=1, U=0, E=0)
INFO 07-07 21:00:07,846 - update_flavor_params_conversion_profile - Dispatching started for transformation [update_flavor_params_conversion_profile]
INFO 07-07 21:00:07,854 - Flavor Params Conversions Profile - Finished reading query, closing connection.
INFO 07-07 21:00:07,857 - input - Finished processing (I=0, O=0, R=1, W=1, U=0, E=0)
INFO 07-07 21:00:07,880 - update_permissions - Dispatching started for transformation [update_permissions]
INFO 07-07 21:00:07,896 - Read Table Permissions - Finished reading query, closing connection.
INFO 07-07 21:00:07,909 - input - Finished processing (I=0, O=0, R=1, W=1, U=0, E=0)
INFO 07-07 21:00:07,931 - update_batch_job_sep - Dispatching started for transformation [update_batch_job_sep]
INFO 07-07 21:00:07,954 - Table input - Finished reading query, closing connection.
INFO 07-07 21:00:07,957 - input - Finished processing (I=0, O=0, R=1, W=1, U=0, E=0)
INFO 07-07 21:00:07,993 - Mapping (sub-transformation) - Finished processing (I=0, O=0, R=12, W=0, U=0, E=0)
Nothing to help me.
Does this happen frequently or is it a one time issue you’ve seen?
we test again, cu you later.
No error since yesterday, but still no stats 
We go for an update to 10.14. This thread is closed, thank you.
Hello,
OK, when you upgrade, if you still experience issues, run and paste results for:
select * from kalturadw_ds.files where insert_time >=20150708\G
select * from kalturadw.dwh_fact_events where event_date_id >=20150708\G
Note that the latter one returns user_ip as well, so, you might want to mask that when pasting here.
Hello,
we have some stat, but “Plays” are missing. here the result of the select you ask:
mysql> select * from kalturadw_ds.files where insert_time >=20150708
-> ;
±--------±----------------------------------------------------------------±------------±------------±--------------------±---------±--------------±------±----------±-------------±-----------±---------±-------------------±-------+
| file_id | file_name | file_status | prev_status | insert_time | run_time | transfer_time | lines | err_lines | file_size_kb | process_id | cycle_id | compression_suffix | subdir |
±--------±----------------------------------------------------------------±------------±------------±--------------------±---------±--------------±------±----------±-------------±-----------±---------±-------------------±-------+
| 121 | ns330062.ip-46-105-114.eu-kaltura_apache_access.log-20150708-03 | IN_CYCLE | NULL | 2015-07-08 04:00:06 | NULL | NULL | NULL | NULL | 3632 | 1 | 63 | gz | |
| 122 | ns330062.ip-46-105-114.eu-kaltura_apache_access.log-20150709-03 | IN_CYCLE | NULL | 2015-07-09 04:00:07 | NULL | NULL | NULL | NULL | 2996 | 1 | 64 | gz | |
| 123 | ns330062.ip-46-105-114.eu-kaltura_apache_access.log-20150709-15 | IN_CYCLE | NULL | 2015-07-09 16:00:06 | NULL | NULL | NULL | NULL | 1097 | 1 | 65 | gz | |
| 124 | ns330062.ip-46-105-114.eu-kaltura_apache_access.log-20150709-16 | IN_CYCLE | NULL | 2015-07-09 17:00:08 | NULL | NULL | NULL | NULL | 247 | 1 | 66 | gz | |
| 125 | ns330062.ip-46-105-114.eu-kaltura_apache_access.log-20150710-03 | IN_CYCLE | NULL | 2015-07-10 04:00:06 | NULL | NULL | NULL | NULL | 1778 | 1 | 67 | gz | |
±--------±----------------------------------------------------------------±------------±------------±--------------------±---------±--------------±------±----------±-------------±-----------±---------±-------------------±-------+
5 rows in set (0.00 sec)
mysql> select * from kalturadw.dwh_fact_events where event_date_id >=20150708
-> ;
Empty set (0.01 sec)
can you look at the relevant gz files under /opt/kaltura/web/logs and make sure you see calls like:
//api_v3/index.php?service=stats&action=collect
?
If not, then no stats requests were made by the player. Remember that stats requests are only made when playing from an embed code and NOT when playing from KMC. If you only played entries in KMC, you will have no stats.
can you try to manually run:
rm /opt/kaltura/dwh/logs/*
logrotate -vvv -f /etc/logrotate.d/kaltura_apache
su kaltura -c "/opt/kaltura/dwh/etlsource/execute/etl_hourly.sh -p /opt/kaltura/dwh -k /opt/kaltura/pentaho/pdi/kitchen.sh"
su kaltura -c "/opt/kaltura/dwh/etlsource/execute/etl_update_dims.sh -p /opt/kaltura/dwh -k /opt/kaltura/pentaho/pdi/kitchen.sh"
su kaltura -c "/opt/kaltura/dwh/etlsource/execute/etl_daily.sh -p /opt/kaltura/dwh -k /opt/kaltura/pentaho/pdi/kitchen.sh"
su kaltura -c "/opt/kaltura/dwh/etlsource/execute/etl_perform_retention_policy.sh -p /opt/kaltura/dwh -k /opt/kaltura/pentaho/pdi/kitchen.sh"
su kaltura -c “/opt/kaltura/app/alpha/scripts/dwh/dwh_plays_views_sync.sh >> /opt/kaltura/log/cron.log”
And see if you get any errors?
for example: ns330062.ip-46-105-114.eu-kaltura_apache_access.log-20150710-03.gz
164.2.255.244 - - [10/Jul/2015:00:00:08 +0200] "GET //api_v3/index.php?service=stats&action=collect&kals…
The videos are embed.
With the last command, we have an error, an access denied on file:/opt/kaltura/app/cache//scripts/classMap.cache. After chmod, I rerun the su kaltura -c “/opt/kaltura/app/alpha/scripts/dwh/dwh_plays_views_sync.sh >> /opt/kaltura/log/cron.log” with no error.
do we wait for the plays to appear?
please look at my reply here:
Hi,
We look for stats in Analytics tab and “plays” in content tabs.
And thank you jess, now we have both.
Regards,
Great. Glad to hear all is well.