Kaltura CE 10.13.0 released - DWH Trouble

Hi Jess,

I have to more read the documentation about analytics include in player.
I keep you inform

Mathieu

Hi Jess,

I read the documentation without success.
I have no error anymore in logs but in Analytics tab:

  • Content Report >> Top Content: display no graph

  • Content Report >> Content Drop-off : display no graph

  • Content Report >> Content Interactions: display no graph

  • Content Report >> Content Contributions: display graph without problem

  • Users & Community reports >> Top Contributors: display graph without problem

  • Users & Community reports >> Geographics Distribution: display no graph
    etc …

I don’t know what I forget to do.
My Embeded player seems to be right configured.

All is working fine except these analytics.

Have you got an idea ?

Thanks

Mathieu

Hi,

I’m going to try this issue:

I’ll keep you informed

Mathieu

Hello Mathieu,

Can you please ensure /opt/kaltura/web/logs/$HOSTNAME-kaltura_apache_access.log-*gz files exist and contain entries similar to:
’//api_v3/index.php?service=stats&action=collect’?

If so, next step would be to ensure these were processed and exist in:
mysql> select * from kalturadw_ds.files where insert_time >=%Y%m%d;

Please look at:
https://github.com/kaltura/platform-install-packages/blob/Jupiter-10.17.0/doc/kaltura-packages-faq.md#analytics-issues

And see if it helps, if you see one of the steps has failed, please let me know which and we can take it from there.

Thanks,

Hi Jess,

The logs exist with right contains.
I just notice a trouble with owner.

root@kaltest:/opt/kaltura/web# ll | grep logs
drwxr-xr-x  2 root     root    4096 juil. 30 14:01 logs/

and

root@kaltest:/opt/kaltura/web/logs# ll
total 1956
drwxr-xr-x 2 root    root       4096 juil. 30 14:01 ./
drwxrwxr-x 8 root    root       4096 juil. 20 12:17 ../
-rw-rw-r-- 1 kaltura www-data 240183 juil. 20 18:11 kaltest-kaltura_apache_access.log-20150720-18.gz
-rw-rw-r-- 1 kaltura www-data 166612 juil. 21 06:56 kaltest-kaltura_apache_access.log-20150721-06.gz
-rw-rw-r-- 1 kaltura www-data 336765 juil. 22 06:48 kaltest-kaltura_apache_access.log-20150722-06.gz
-rw-r--r-- 1 root    root     395022 juil. 23 06:37 kaltest-kaltura_apache_access.log-20150723-06.gz
-rw-r--r-- 1 root    root     229677 juil. 24 06:28 kaltest-kaltura_apache_access.log-20150724-06.gz
-rw-r--r-- 1 root    root     107840 juil. 25 06:30 kaltest-kaltura_apache_access.log-20150725-06.gz
-rw-r--r-- 1 root    root        523 juil. 26 06:44 kaltest-kaltura_apache_access.log-20150726-06.gz
-rw-r--r-- 1 root    root        545 juil. 27 06:00 kaltest-kaltura_apache_access.log-20150727-06.gz
-rw-r--r-- 1 root    root       2702 juil. 27 12:24 kaltest-kaltura_apache_access.log-20150727-14.gz
-rw-r--r-- 1 root    root      20763 juil. 28 13:48 kaltest-kaltura_apache_access.log-20150728-13.gz
-rw-r--r-- 1 root    root      11909 juil. 28 14:28 kaltest-kaltura_apache_access.log-20150728-14.gz
-rw-r--r-- 1 root    root      48557 juil. 28 15:53 kaltest-kaltura_apache_access.log-20150728-15.gz
-rw-r--r-- 1 root    root     158665 juil. 29 06:50 kaltest-kaltura_apache_access.log-20150729-06.gz
-rw-r--r-- 1 root    root     156947 juil. 29 23:08 kaltest-kaltura_apache_access.log-20150729-23.gz
-rw-r--r-- 1 root    root      82508 juil. 30 06:32 kaltest-kaltura_apache_access.log-20150730-06.gz

For SQL:

mysql> select * from kalturadw_ds.locks;
+---------+------------------+---------------------+------------+
| lock_id | lock_name        | lock_time           | lock_state |
+---------+------------------+---------------------+------------+
|       1 | daily_lock       | 2015-07-29 23:09:42 |          0 |
|       2 | retention_lock   | 2015-07-29 23:09:52 |          0 |
|       3 | hourly_kaltest   | 2015-07-29 23:09:26 |          0 |
|       4 | update_dims_lock | 2015-07-29 23:09:35 |          0 |
+---------+------------------+---------------------+------------+


mysql> select * from kalturadw_ds.files where insert_time >='%Y%m%d';
+---------+-----------------------------------------------+-------------+-------------+---------------------+----------+---------------+-------+-----------+--------------+------------+----------+--------------------+--------+
| file_id | file_name                                     | file_status | prev_status | insert_time         | run_time | transfer_time | lines | err_lines | file_size_kb | process_id | cycle_id | compression_suffix | subdir |
+---------+-----------------------------------------------+-------------+-------------+---------------------+----------+---------------+-------+-----------+--------------+------------+----------+--------------------+--------+
|       1 | kaltest-kaltura_apache_access.log-20150720-18 | IN_CYCLE    | NULL        | 2015-07-27 14:50:24 | NULL     | NULL          |  NULL |      NULL |          234 |          1 |        1 | gz                 |        |
|       2 | kaltest-kaltura_apache_access.log-20150721-06 | IN_CYCLE    | NULL        | 2015-07-27 14:50:24 | NULL     | NULL          |  NULL |      NULL |          162 |          1 |        1 | gz                 |        |
|       3 | kaltest-kaltura_apache_access.log-20150722-06 | IN_CYCLE    | NULL        | 2015-07-27 14:50:24 | NULL     | NULL          |  NULL |      NULL |          328 |          1 |        1 | gz                 |        |
|       4 | kaltest-kaltura_apache_access.log-20150723-06 | IN_CYCLE    | NULL        | 2015-07-27 14:50:24 | NULL     | NULL          |  NULL |      NULL |          385 |          1 |        1 | gz                 |        |
|       5 | kaltest-kaltura_apache_access.log-20150724-06 | IN_CYCLE    | NULL        | 2015-07-27 14:50:24 | NULL     | NULL          |  NULL |      NULL |          224 |          1 |        1 | gz                 |        |
|       6 | kaltest-kaltura_apache_access.log-20150725-06 | IN_CYCLE    | NULL        | 2015-07-27 14:50:24 | NULL     | NULL          |  NULL |      NULL |          105 |          1 |        1 | gz                 |        |
|       7 | kaltest-kaltura_apache_access.log-20150726-06 | IN_CYCLE    | NULL        | 2015-07-27 14:50:24 | NULL     | NULL          |  NULL |      NULL |            0 |          1 |        1 | gz                 |        |
|       8 | kaltest-kaltura_apache_access.log-20150727-06 | IN_CYCLE    | NULL        | 2015-07-27 14:50:24 | NULL     | NULL          |  NULL |      NULL |            0 |          1 |        1 | gz                 |        |
|       9 | kaltest-kaltura_apache_access.log-20150727-14 | IN_CYCLE    | NULL        | 2015-07-27 14:50:24 | NULL     | NULL          |  NULL |      NULL |            2 |          1 |        1 | gz                 |        |
|      10 | kaltest-kaltura_apache_access.log-20150728-13 | IN_CYCLE    | NULL        | 2015-07-28 13:48:17 | NULL     | NULL          |  NULL |      NULL |           20 |          1 |        2 | gz                 |        |
|      11 | kaltest-kaltura_apache_access.log-20150728-14 | IN_CYCLE    | NULL        | 2015-07-28 14:28:48 | NULL     | NULL          |  NULL |      NULL |           11 |          1 |        3 | gz                 |        |
|      12 | kaltest-kaltura_apache_access.log-20150728-15 | IN_CYCLE    | NULL        | 2015-07-28 15:53:37 | NULL     | NULL          |  NULL |      NULL |           47 |          1 |        4 | gz                 |        |
|      13 | kaltest-kaltura_apache_access.log-20150729-06 | IN_CYCLE    | NULL        | 2015-07-29 23:09:14 | NULL     | NULL          |  NULL |      NULL |          154 |          1 |        5 | gz                 |        |
|      14 | kaltest-kaltura_apache_access.log-20150729-23 | IN_CYCLE    | NULL        | 2015-07-29 23:09:14 | NULL     | NULL          |  NULL |      NULL |          153 |          1 |        5 | gz                 |        |
+---------+-----------------------------------------------+-------------+-------------+---------------------+----------+---------------+-------+-----------+--------------+------------+----------+--------------------+--------+
14 rows in set, 1 warning (0.01 sec)

mysql> select * from kalturadw.dwh_fact_events where event_date_id >="%Y%m%d";
Empty set, 1 warning (0.00 sec)

I try to change the owner or I follow the analytics issues ? : Drop the current DWH DBs and Reinstall and config DWH.

Thanks

Mathieu

Change the owner of files and directory provide the same thing …no graph

Hi Mathieu,

Ownership is fine, the process only needs read permissions [and exec to enter the dir] and it has these.
Do you see actual collect requests in the access logs?

Hi Jess,

Sorry but I don’t understand your last question ?
What do you mean exactly ?

Thanks

Hi,

I mean, open the files under /opt/kaltura/web/logs/kaltest-kaltura_apache_access.log-* and make sure you actually see calls for ‘//api_v3/index.php?service=stats&action=collect’ if you do not, then no analytics calls were made by the player/…

Hi,

Ok.
Like this ?:

kaltest-kaltura_apache_access.log-20150725-06.gz:147.94.210.250 - - [24/Jul/2015:09:02:22 +0200] "GET //api_v3/index.php?service=stats&action=collect&kalsig=cd0c21970aa1558c015ab8cf1a96bee3&event%3AcurrentPoint=0&event%3AentryId=0%5Ffm3olxzr&event%3AeventTimestamp=1437721349135&event%3AeventType=2&event%3Areferrer=http%253A%2F%2Fkaltest%2Euniv%2Damu%2Efr%2Findex%2Ephp%2Fextwidget%2Fpreview%2Fpartner%5Fid%2F100%2Fuiconf%5Fid%2F23448135%2Fentry%5Fid%2F0%5Fuciw10ky%2Fembed%2Fdynamic%253F%2526flashvars%255BstreamerType%255D%253Dauto&clientTag=kdp%3Av3%2E9%2E9%2Ccache%5Fst%3A1437721777&event%3Aduration=0&event%3AobjectType=KalturaStatsEvent&event%3AsessionId=5E33A7FF%2DC7D4%2DB819%2DA67C%2DBED7A2B2068A&ignoreNull=1&event%3AfeatureType=1&ks=MDRiZWI1NGQ4OTdjZjVhMGI5YmQyOTc5NjEzOGIzM2FkOTZmNzQ5MXwxMDA7MTAwOzE0Mzc4MDcyNzk7MDsxNDM3NzIwODc5LjQzNDc7MDt2aWV3Oiosd2lkZ2V0OjE7Ow%3D%3D&apiVersion=3%2E1%2E5&event%3AclientVer=3%2E0%3Av3%2E9%2E9&event%3AuiconfId=23448135&partnerId=100&event%3ApartnerId=100&event%3Aseek=true&event%3AisFirstInSession=false HTTP/1.1" 200 7 0/384 "http://kaltest.univ-amu.fr/index.php/kwidget/wid/_100/uiconf_id/23448135/entry_id/0_uciw10ky/cache_st/1437721777" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.10; rv:39.0) Gecko/20100101 Firefox/39.0" "-" 147.94.210.250 "-" "kaltest.univ-amu.fr"
kaltest-kaltura_apache_access.log-20150725-06.gz:147.94.210.250 - - [24/Jul/2015:09:02:23 +0200] "GET //api_v3/index.php?service=stats&action=collect&kalsig=1264eaeace60125121ec99fbc1a8f381&event%3AcurrentPoint=0&event%3AentryId=0%5Ffm3olxzr&event%3AeventTimestamp=1437721349720&event%3AeventType=3&event%3Areferrer=http%253A%2F%2Fkaltest%2Euniv%2Damu%2Efr%2Findex%2Ephp%2Fextwidget%2Fpreview%2Fpartner%5Fid%2F100%2Fuiconf%5Fid%2F23448135%2Fentry%5Fid%2F0%5Fuciw10ky%2Fembed%2Fdynamic%253F%2526flashvars%255BstreamerType%255D%253Dauto&clientTag=kdp%3Av3%2E9%2E9%2Ccache%5Fst%3A1437721777&event%3Aduration=0&event%3AobjectType=KalturaStatsEvent&event%3AsessionId=5E33A7FF%2DC7D4%2DB819%2DA67C%2DBED7A2B2068A&ignoreNull=1&event%3AfeatureType=1&ks=MDRiZWI1NGQ4OTdjZjVhMGI5YmQyOTc5NjEzOGIzM2FkOTZmNzQ5MXwxMDA7MTAwOzE0Mzc4MDcyNzk7MDsxNDM3NzIwODc5LjQzNDc7MDt2aWV3Oiosd2lkZ2V0OjE7Ow%3D%3D&apiVersion=3%2E1%2E5&event%3AclientVer=3%2E0%3Av3%2E9%2E9&event%3AuiconfId=23448135&partnerId=100&event%3ApartnerId=100&event%3Aseek=true&event%3AisFirstInSession=false HTTP/1.1" 200 7 0/265 "http://kaltest.univ-amu.fr/index.php/kwidget/wid/_100/uiconf_id/23448135/entry_id/0_uciw10ky/cache_st/1437721777" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.10; rv:39.0) Gecko/20100101 Firefox/39.0" "-" 147.94.210.250 "-" "kaltest.univ-amu.fr"
kaltest-kaltura_apache_access.log-20150728-13.gz:147.94.140.30 - - [27/Jul/2015:15:37:14 +0200] "GET //api_v3/index.php?service=stats&action=collect&kalsig=90e0b3e4a2986da3fe0d53ac7dc9885c&ks=YTk1OGViOTMxOTczYjA0ZDk2ZTVmNzNmMjJhZWUwM2Y3YWFkY2MwZXwxMDA7MTAwOzE0MzgwODc5MDQ7MjsxNDM4MDAxNTA0LjIxNzI7bWF0aGlldS5tb2xpbmVyaXNAdW5pdi1hbXUuZnI7ZGlzYWJsZWVudGl0bGVtZW50Ozs%3D&event%3AisFirstInSession=false&event%3AobjectType=KalturaStatsEvent&apiVersion=3%2E1%2E5&event%3ApartnerId=100&event%3Areferrer=http%253A%2F%2Fkaltest%2Euniv%2Damu%2Efr%2Findex%2Ephp%2Fkmc%2Fkmc4%2523content%257Cmanage&event%3Aduration=0&partnerId=100&event%3AclientVer=3%2E0%3Av3%2E9%2E9&event%3AuiconfId=23448134&event%3AentryId=0%5Fuciw10ky&event%3AeventType=1&ignoreNull=1&event%3Aseek=false&event%3AcurrentPoint=0&event%3AeventTimestamp=1438004241494&clientTag=kdp%3Av3%2E9%2E9%2Ccache%5Fst%3A1438005133&event%3AsessionId=F5A3F50B%2DDC2D%2DE8CE%2D1B3D%2DCFBB46711802 HTTP/1.1" 200 7 0/1009 "http://kaltest.univ-amu.fr/index.php/kmc/kmc4" "Mozilla/5.0 (Windows NT 6.3; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/44.0.2403.107 Safari/537.36" "-" 147.94.140.30 "-" "kaltest.univ-amu.fr"
kaltest-kaltura_apache_access.log-20150728-13.gz:147.94.140.30 - - [27/Jul/2015:15:37:14 +0200] "GET //api_v3/index.php?service=stats&action=collect&kalsig=2411d5151d2f98f6de06cb152244af59&ks=YTk1OGViOTMxOTczYjA0ZDk2ZTVmNzNmMjJhZWUwM2Y3YWFkY2MwZXwxMDA7MTAwOzE0MzgwODc5MDQ7MjsxNDM4MDAxNTA0LjIxNzI7bWF0aGlldS5tb2xpbmVyaXNAdW5pdi1hbXUuZnI7ZGlzYWJsZWVudGl0bGVtZW50Ozs%3D&event%3AisFirstInSession=false&event%3AobjectType=KalturaStatsEvent&apiVersion=3%2E1%2E5&event%3ApartnerId=100&event%3Areferrer=http%253A%2F%2Fkaltest%2Euniv%2Damu%2Efr%2Findex%2Ephp%2Fkmc%2Fkmc4%2523content%257Cmanage&event%3Aduration=0&partnerId=100&event%3AclientVer=3%2E0%3Av3%2E9%2E9&event%3AuiconfId=23448134&event%3AentryId=0%5Fuciw10ky&event%3AeventType=2&ignoreNull=1&event%3Aseek=false&event%3AcurrentPoint=0&event%3AeventTimestamp=1438004241503&clientTag=kdp%3Av3%2E9%2E9%2Ccache%5Fst%3A1438005133&event%3AsessionId=F5A3F50B%2DDC2D%2DE8CE%2D1B3D%2DCFBB46711802 HTTP/1.1" 200 7 0/606 "http://kaltest.univ-amu.fr/index.php/kmc/kmc4" "Mozilla/5.0 (Windows NT 6.3; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/44.0.2403.107 Safari/537.36" "-" 147.94.140.30 "-" "kaltest.univ-amu.fr" 

Mathieu

Yes, this looks fine.
Can you try to play a file via an embed and then run /opt/kaltura/bin/kaltura-run-dwh.sh?
Then look for the string ‘ERROR’ in /opt/kaltura/dwh/logs and let me know what you see?

Here it is:

log_events_events.log:2015/07/30 16:38:55 - Parse date time to DWH timezone.0 - ERROR (version 4.2.1-stable, build 15952 from 2011- 10-25 15.27.10 by buildguy) : at org.pentaho.di.trans.steps.scriptvalues_mod.ScriptValuesMod.addValues(ScriptValuesMod.java:376)
log_events_events.log:2015/07/30 16:38:55 - Parse date time to DWH timezone.0 - ERROR (version 4.2.1-stable, build 15952 from 2011- 10-25 15.27.10 by buildguy) : … 3 more
log_events_events.log:2015/07/30 16:38:55 - parse bandwidth lines.1 - ERROR (version 4.2.1-stable, build 15952 from 2011-10-25 15.2 7.10 by buildguy) : Unexpected error
log_events_events.log:2015/07/30 16:38:55 - parse bandwidth lines.1 - ERROR (version 4.2.1-stable, build 15952 from 2011-10-25 15.2 7.10 by buildguy) : org.pentaho.di.core.exception.KettleValueException:
log_events_events.log:2015/07/30 16:38:55 - parse bandwidth lines.1 - ERROR (version 4.2.1-stable, build 15952 from 2011-10-25 15.2 7.10 by buildguy) : Javascript error:
log_events_events.log:2015/07/30 16:38:55 - parse bandwidth lines.1 - ERROR (version 4.2.1-stable, build 15952 from 2011-10-25 15.2 7.10 by buildguy) : Could not apply the given format dd/MMM/yyyy:HH:mm:ss on the string for 30/Jul/2015:15:54:39 : Format.parseObje ct(String) failed (script#14)
log_events_events.log:2015/07/30 16:38:55 - parse bandwidth lines.1 - ERROR (version 4.2.1-stable, build 15952 from 2011-10-25 15.2 7.10 by buildguy) :
log_events_events.log:2015/07/30 16:38:55 - parse bandwidth lines.1 - ERROR (version 4.2.1-stable, build 15952 from 2011-10-25 15.2 7.10 by buildguy) : at org.pentaho.di.trans.steps.scriptvalues_mod.ScriptValuesMod.addValues(ScriptValuesMod.java:457)

Thanks

OK, this is progress:) Can you please paste the full error block from the log?

Yes it is;-)

etl_hourly-20150730-16.log:ERROR 30-07 16:38:55,804 -
Parse date time to DWH timezone - Unexpected error
etl_hourly-20150730-16.log:ERROR 30-07 16:38:55,806 - Parse date time to DWH
timezone - org.pentaho.di.core.exception.KettleValueException: 

etl_hourly-20150730-16.log:ERROR 30-07 16:38:55,861 -
parse bandwidth lines - Unexpected error etl_hourly-20150730-16.log:ERROR 30-07
16:38:55,871 - parse bandwidth lines -
org.pentaho.di.core.exception.KettleValueException: 

etl_hourly-20150730-16.log:ERROR 30-07 16:38:55,881 -
process file - Errors detected!

etl_hourly-20150730-16.log:ERROR 30-07 16:38:55,888 -
process file - Errors detected!

log_events_events.log:2015/07/30 16:38:55 - Parse date
time to DWH timezone.0 - ERROR (version 4.2.1-stable, build 15952 from 2011-10-25
15.27.10 by buildguy) : Unexpected error

log_events_events.log:2015/07/30 16:38:55 - Parse date
time to DWH timezone.0 - ERROR (version 4.2.1-stable, build 15952 from
2011-10-25 15.27.10 by buildguy) :
org.pentaho.di.core.exception.KettleValueException: 

log_events_events.log:2015/07/30 16:38:55 - Parse date
time to DWH timezone.0 - ERROR (version 4.2.1-stable, build 15952 from
2011-10-25 15.27.10 by buildguy) : Javascript error: 

log_events_events.log:2015/07/30 16:38:55 - Parse date
time to DWH timezone.0 - ERROR (version 4.2.1-stable, build 15952 from
2011-10-25 15.27.10 by buildguy) : Could not apply the given format
dd/MMM/yyyy:HH:mm:ss on the string for 30/Jul/2015:13:54:40 :
Format.parseObject(String) failed (script#7)

log_events_events.log:2015/07/30 16:38:55 - Parse date
time to DWH timezone.0 - ERROR (version 4.2.1-stable, build 15952 from
2011-10-25 15.27.10 by buildguy) : 

log_events_events.log:2015/07/30 16:38:55 - Parse date
time to DWH timezone.0 - ERROR (version 4.2.1-stable, build 15952 from
2011-10-25 15.27.10 by buildguy) :       at
org.pentaho.di.trans.steps.scriptvalues_mod.ScriptValuesMod.addValues(ScriptValuesMod.java:457)

log_events_events.log:2015/07/30 16:38:55 - Parse date
time to DWH timezone.0 - ERROR (version 4.2.1-stable, build 15952 from
2011-10-25 15.27.10 by buildguy) :       at
org.pentaho.di.trans.steps.scriptvalues_mod.ScriptValuesMod.processRow(ScriptValuesMod.java:688)

log_events_events.log:2015/07/30 16:38:55 - Parse date
time to DWH timezone.0 - ERROR (version 4.2.1-stable, build 15952 from
2011-10-25 15.27.10 by buildguy) :       at
org.pentaho.di.trans.step.RunThread.run(RunThread.java:40)

log_events_events.log:2015/07/30 16:38:55 - Parse date
time to DWH timezone.0 - ERROR (version 4.2.1-stable, build 15952 from
2011-10-25 15.27.10 by buildguy) :       at
java.lang.Thread.run(Thread.java:745)

log_events_events.log:2015/07/30 16:38:55 - Parse date
time to DWH timezone.0 - ERROR (version 4.2.1-stable, build 15952 from
2011-10-25 15.27.10 by buildguy) : Caused by: org.mozilla.javascript.EvaluatorException:
Could not apply the given format dd/MMM/yyyy:HH:mm:ss on the string for
30/Jul/2015:13:54:40 : Format.parseObject(String) failed (script#7)

log_events_events.log:2015/07/30 16:38:55 - Parse date
time to DWH timezone.0 - ERROR (version 4.2.1-stable, build 15952 from
2011-10-25 15.27.10 by buildguy) :       at
org.mozilla.javascript.DefaultErrorReporter.runtimeError(DefaultErrorReporter.java:109)

log_events_events.log:2015/07/30 16:38:55 - Parse date
time to DWH timezone.0 - ERROR (version 4.2.1-stable, build 15952 from
2011-10-25 15.27.10 by buildguy) :       at
org.mozilla.javascript.Context.reportRuntimeError(Context.java:938)

log_events_events.log:2015/07/30 16:38:55 - Parse date
time to DWH timezone.0 - ERROR (version 4.2.1-stable, build 15952 from
2011-10-25 15.27.10 by buildguy) :       at
org.mozilla.javascript.Context.reportRuntimeError(Context.java:994)

log_events_events.log:2015/07/30 16:38:55 - Parse date
time to DWH timezone.0 - ERROR (version 4.2.1-stable, build 15952 from
2011-10-25 15.27.10 by buildguy) :       at
org.pentaho.di.trans.steps.scriptvalues_mod.ScriptValuesAddedFunctions.str2date(ScriptValuesAddedFunctions.java:909)

log_events_events.log:2015/07/30 16:38:55 - Parse date
time to DWH timezone.0 - ERROR (version 4.2.1-stable, build 15952 from
2011-10-25 15.27.10 by buildguy) :       at
sun.reflect.GeneratedMethodAccessor4.invoke(Unknown Source)

log_events_events.log:2015/07/30 16:38:55 - Parse date
time to DWH timezone.0 - ERROR (version 4.2.1-stable, build 15952 from
2011-10-25 15.27.10 by buildguy) :       at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

log_events_events.log:2015/07/30 16:38:55 - Parse date
time to DWH timezone.0 - ERROR (version 4.2.1-stable, build 15952 from
2011-10-25 15.27.10 by buildguy) :       at
java.lang.reflect.Method.invoke(Method.java:606)

log_events_events.log:2015/07/30 16:38:55 - Parse date
time to DWH timezone.0 - ERROR (version 4.2.1-stable, build 15952 from
2011-10-25 15.27.10 by buildguy) :       at
org.mozilla.javascript.MemberBox.invoke(MemberBox.java:161)

log_events_events.log:2015/07/30 16:38:55 - Parse date
time to DWH timezone.0 - ERROR (version 4.2.1-stable, build 15952 from
2011-10-25 15.27.10 by buildguy) :       at
org.mozilla.javascript.FunctionObject.call(FunctionObject.java:413)

log_events_events.log:2015/07/30 16:38:55 - Parse date
time to DWH timezone.0 - ERROR (version 4.2.1-stable, build 15952 from 2011-10-25
15.27.10 by buildguy) :       at
org.mozilla.javascript.optimizer.OptRuntime.callName(OptRuntime.java:97)

log_events_events.log:2015/07/30 16:38:55 - Parse date
time to DWH timezone.0 - ERROR (version 4.2.1-stable, build 15952 from
2011-10-25 15.27.10 by buildguy) :       at
org.mozilla.javascript.gen.c21._c0(script:7)

log_events_events.log:2015/07/30 16:38:55 - Parse date
time to DWH timezone.0 - ERROR (version 4.2.1-stable, build 15952 from
2011-10-25 15.27.10 by buildguy) :       at
org.mozilla.javascript.gen.c21.call(script)

log_events_events.log:2015/07/30 16:38:55 - Parse date
time to DWH timezone.0 - ERROR (version 4.2.1-stable, build 15952 from
2011-10-25 15.27.10 by buildguy) :       at
org.mozilla.javascript.ContextFactory.doTopCall(ContextFactory.java:398)

log_events_events.log:2015/07/30 16:38:55 - Parse date
time to DWH timezone.0 - ERROR (version 4.2.1-stable, build 15952 from
2011-10-25 15.27.10 by buildguy) :       at
org.mozilla.javascript.ScriptRuntime.doTopCall(ScriptRuntime.java:3065)

log_events_events.log:2015/07/30 16:38:55 - Parse date
time to DWH timezone.0 - ERROR (version 4.2.1-stable, build 15952 from
2011-10-25 15.27.10 by buildguy) :       at
org.mozilla.javascript.gen.c21.call(script)

log_events_events.log:2015/07/30 16:38:55 - Parse date
time to DWH timezone.0 - ERROR (version 4.2.1-stable, build 15952 from
2011-10-25 15.27.10 by buildguy) :       at
org.mozilla.javascript.gen.c21.exec(script)

log_events_events.log:2015/07/30 16:38:55 - Parse date
time to DWH timezone.0 - ERROR (version 4.2.1-stable, build 15952 from
2011-10-25 15.27.10 by buildguy) :       at
org.pentaho.di.trans.steps.scriptvalues_mod.ScriptValuesMod.addValues(ScriptValuesMod.java:376)

log_events_events.log:2015/07/30 16:38:55 - Parse date
time to DWH timezone.0 - ERROR (version 4.2.1-stable, build 15952 from
2011-10-25 15.27.10 by buildguy) :       ...
3 more

log_events_events.log:2015/07/30 16:38:55 - parse
bandwidth lines.1 - ERROR (version 4.2.1-stable, build 15952 from 2011-10-25
15.27.10 by buildguy) : Unexpected error

log_events_events.log:2015/07/30 16:38:55 - parse
bandwidth lines.1 - ERROR (version 4.2.1-stable, build 15952 from 2011-10-25
15.27.10 by buildguy) : org.pentaho.di.core.exception.KettleValueException: 

log_events_events.log:2015/07/30 16:38:55 - parse
bandwidth lines.1 - ERROR (version 4.2.1-stable, build 15952 from 2011-10-25
15.27.10 by buildguy) : Javascript error: 

log_events_events.log:2015/07/30 16:38:55 - parse
bandwidth lines.1 - ERROR (version 4.2.1-stable, build 15952 from 2011-10-25
15.27.10 by buildguy) : Could not apply the given format dd/MMM/yyyy:HH:mm:ss
on the string for 30/Jul/2015:15:54:39 : Format.parseObject(String) failed
(script#14)

log_events_events.log:2015/07/30 16:38:55 - parse
bandwidth lines.1 - ERROR (version 4.2.1-stable, build 15952 from 2011-10-25 15.27.10
by buildguy) : 

log_events_events.log:2015/07/30 16:38:55 - parse
bandwidth lines.1 - ERROR (version 4.2.1-stable, build 15952 from 2011-10-25
15.27.10 by buildguy) :          at
org.pentaho.di.trans.steps.scriptvalues_mod.ScriptValuesMod.addValues(ScriptValuesMod.java:457)

log_events_events.log:2015/07/30 16:38:55 - parse
bandwidth lines.1 - ERROR (version 4.2.1-stable, build 15952 from 2011-10-25
15.27.10 by buildguy) :          at
org.pentaho.di.trans.steps.scriptvalues_mod.ScriptValuesMod.processRow(ScriptValuesMod.java:688)

log_events_events.log:2015/07/30 16:38:55 - parse
bandwidth lines.1 - ERROR (version 4.2.1-stable, build 15952 from 2011-10-25
15.27.10 by buildguy) :          at
org.pentaho.di.trans.step.RunThread.run(RunThread.java:40)

log_events_events.log:2015/07/30 16:38:55 - parse
bandwidth lines.1 - ERROR (version 4.2.1-stable, build 15952 from 2011-10-25
15.27.10 by buildguy) :          at
java.lang.Thread.run(Thread.java:745)

log_events_events.log:2015/07/30 16:38:55 - parse
bandwidth lines.1 - ERROR (version 4.2.1-stable, build 15952 from 2011-10-25
15.27.10 by buildguy) : Caused by: org.mozilla.javascript.EvaluatorException:
Could not apply the given format dd/MMM/yyyy:HH:mm:ss on the string for
30/Jul/2015:15:54:39 : Format.parseObject(String) failed (script#14)

log_events_events.log:2015/07/30 16:38:55 - parse
bandwidth lines.1 - ERROR (version 4.2.1-stable, build 15952 from 2011-10-25
15.27.10 by buildguy) :          at
org.mozilla.javascript.DefaultErrorReporter.runtimeError(DefaultErrorReporter.java:109)

log_events_events.log:2015/07/30 16:38:55 - parse
bandwidth lines.1 - ERROR (version 4.2.1-stable, build 15952 from 2011-10-25
15.27.10 by buildguy) :          at
org.mozilla.javascript.Context.reportRuntimeError(Context.java:938)

log_events_events.log:2015/07/30 16:38:55 - parse
bandwidth lines.1 - ERROR (version 4.2.1-stable, build 15952 from 2011-10-25
15.27.10 by buildguy) :          at
org.mozilla.javascript.Context.reportRuntimeError(Context.java:994)

log_events_events.log:2015/07/30 16:38:55 - parse
bandwidth lines.1 - ERROR (version 4.2.1-stable, build 15952 from 2011-10-25
15.27.10 by buildguy) :          at
org.pentaho.di.trans.steps.scriptvalues_mod.ScriptValuesAddedFunctions.str2date(ScriptValuesAddedFunctions.java:909)

log_events_events.log:2015/07/30 16:38:55 - parse
bandwidth lines.1 - ERROR (version 4.2.1-stable, build 15952 from 2011-10-25
15.27.10 by buildguy) :          at
sun.reflect.GeneratedMethodAccessor4.invoke(Unknown Source)

log_events_events.log:2015/07/30 16:38:55 - parse
bandwidth lines.1 - ERROR (version 4.2.1-stable, build 15952 from 2011-10-25
15.27.10 by buildguy) :          at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

log_events_events.log:2015/07/30 16:38:55 - parse
bandwidth lines.1 - ERROR (version 4.2.1-stable, build 15952 from 2011-10-25
15.27.10 by buildguy) :          at
java.lang.reflect.Method.invoke(Method.java:606)

log_events_events.log:2015/07/30 16:38:55 - parse
bandwidth lines.1 - ERROR (version 4.2.1-stable, build 15952 from 2011-10-25
15.27.10 by buildguy) :          at
org.mozilla.javascript.MemberBox.invoke(MemberBox.java:161)

log_events_events.log:2015/07/30 16:38:55 - parse
bandwidth lines.1 - ERROR (version 4.2.1-stable, build 15952 from 2011-10-25
15.27.10 by buildguy) :          at
org.mozilla.javascript.FunctionObject.call(FunctionObject.java:413)

log_events_events.log:2015/07/30 16:38:55 - parse
bandwidth lines.1 - ERROR (version 4.2.1-stable, build 15952 from 2011-10-25
15.27.10 by buildguy) :          at
org.mozilla.javascript.optimizer.OptRuntime.callName(OptRuntime.java:97)

log_events_events.log:2015/07/30 16:38:55 - parse
bandwidth lines.1 - ERROR (version 4.2.1-stable, build 15952 from 2011-10-25
15.27.10 by buildguy) :          at
org.mozilla.javascript.gen.c17._c0(script:14)

log_events_events.log:2015/07/30 16:38:55 - parse
bandwidth lines.1 - ERROR (version 4.2.1-stable, build 15952 from 2011-10-25
15.27.10 by buildguy) :          at
org.mozilla.javascript.gen.c17.call(script)

log_events_events.log:2015/07/30 16:38:55 - parse bandwidth
lines.1 - ERROR (version 4.2.1-stable, build 15952 from 2011-10-25 15.27.10 by
buildguy) :          at
org.mozilla.javascript.ContextFactory.doTopCall(ContextFactory.java:398)

log_events_events.log:2015/07/30 16:38:55 - parse
bandwidth lines.1 - ERROR (version 4.2.1-stable, build 15952 from 2011-10-25
15.27.10 by buildguy) :          at
org.mozilla.javascript.ScriptRuntime.doTopCall(ScriptRuntime.java:3065)

log_events_events.log:2015/07/30 16:38:55 - parse
bandwidth lines.1 - ERROR (version 4.2.1-stable, build 15952 from 2011-10-25
15.27.10 by buildguy) :          at
org.mozilla.javascript.gen.c17.call(script)

log_events_events.log:2015/07/30 16:38:55 - parse
bandwidth lines.1 - ERROR (version 4.2.1-stable, build 15952 from 2011-10-25
15.27.10 by buildguy) :          at
org.mozilla.javascript.gen.c17.exec(script)

log_events_events.log:2015/07/30 16:38:55 - parse
bandwidth lines.1 - ERROR (version 4.2.1-stable, build 15952 from 2011-10-25
15.27.10 by buildguy) :          at
org.pentaho.di.trans.steps.scriptvalues_mod.ScriptValuesMod.addValues(ScriptValuesMod.java:376)

log_events_events.log:2015/07/30 16:38:55 - parse
bandwidth lines.1 - ERROR (version 4.2.1-stable, build 15952 from 2011-10-25
15.27.10 by buildguy) :          ... 3
more

log_events_events.log:2015/07/30 16:38:55 - process file
- ERROR (version 4.2.1-stable, build 15952 from 2011-10-25 15.27.10 by
buildguy) : Errors detected!

log_events_events.log:2015/07/30 16:38:55 - process file
- ERROR (version 4.2.1-stable, build 15952 from 2011-10-25 15.27.10 by
buildguy) : Errors detected!

Thanks, Mathieu,

Will consult our Analytics dev and get back to you on this one.

Especially thank you to you.

Mathieu

Hello Mathieu,

We THINK we have a solution that might work:)
edit /opt/kaltura/dwh/etlsource/common/parse_date_to_dwh_timezone.ktr and replace line 298:

eventTime = str2date(calculatedDateTime, &apos;dd&#47;MMM&#47;yyyy:HH:mm:ss&apos;);</jsScript_script>

with:

eventTime = str2date(calculatedDateTime,"dd/MMM/yyyy:HH:mm:ss", 'EN');</jsScript_script>

Then rerun the whole cycle and let me know if you still see issues.

Thanks,

Hello Jess,

I think it’s better, but I have an other error and no display on Analytics.
The solution is not far.

Thanks for your help

Mathieu

Here’s the log:

https://amubox.univ-amu.fr/index.php/s/S4KDuEJe9RnW4lF

Hey Mathieu,

Log still says:
2015/08/13 11:25:57 - Write to log.0 - Could not apply the given format dd/MMM/yyyy:HH:mm:ss on the string for 31/Jul/2015:22:45:28 : Format.parseObject(String) failed (script#30)

Can you make sure the file was indeed changed as I indicated above?

Hi Jess,

Yes, I change it

desiredTimeZone = java.util.TimeZone.getTimeZone(getVariable(&apos;DataTimeZone&apos;, &apos;America&#47;New_York&apos;));
dateFormat = java.text.SimpleDateFormat(&apos;[dd&#47;MMM&#47;yyyy:HH:mm:ssZ]&apos;,java.util.Locale.ENGLISH);
dateFormat.setTimeZone(desiredTimeZone);
calculatedDateTime = substr(dateFormat.format(dateFormat.parse(raw_datetime + timezone_offset)), 1, 20);
eventTime = str2date(calculatedDateTime,"dd/MMM/yyyy:HH:mm:ss", 'EN');</jsScript_script>

Perhaps,that I must also change this line before ?:

dateFormat = java.text.SimpleDateFormat(&apos;[dd&#47;MMM&#47;yyyy:HH:mm:ssZ]&apos;,java.util.Locale.ENGLISH);

in

dateFormat = java.text.SimpleDateFormat(&apos;[dd;MMM;yyyy:HH:mm:ssZ]&apos;,java.util.Locale.ENGLISH);

Mathieu