Hi @hiphopservers,
You can do it anyway you want so long as, in the end, you end up with the following DBs created but empty:
| kalturadw |
| kalturadw_bisources |
| kalturadw_ds |
| kalturalog
so that you can re-run /opt/kaltura/bin/kaltura-dwh-config.sh to create the needed tables and populate them.
Hello @jess
Thank you for the information. I was able to drop the tables using phpMyAdmin and ran the dwh_config script. However, I got this error in the process before it completed.
1
Error - bailing out!
DWH configured.
I have not idea what that means nor if it affects the proper running of the stats yet.
Are the DBs themselves still in place? if you removed them then it will not work.
Yes, the DB is intact it appears to have populated with no issue, but when I ran dwh I got errors.
/opt/kaltura/bin/kaltura-run-dwh.sh
The error I first saw said a permission was wrong for the cron.log. I change the permission to allow access to the logs.
bash: /opt/kaltura/log/cron.log: Permission denied
I then ran the following command to resolve the permission error.
# chown kaltura:apache /opt/kaltura/log/*.log
However, I got the following error after resolving the log permission issue. This is likely related to the failure/error I got during the re-configuration.
ERROR 1146 (42S02) at line 1: Table ‘kalturadw.dwh_entry_plays_views’ doesn’t exist
I do not know the structure of the missing table so I am not able to manually create it.
Also, is there a way to re-calculate the data from the videos and files already uploaded?
Hi @hiphopservers,
Seeing how you got “Error - bailing out!” when running dwh-config, the population obviously did not complete successfully.
I suggest you:
Drop all tables and stored procedures from the following DBs again:
| kalturadw |
| kalturadw_bisources |
| kalturadw_ds |
| kalturalog
Make sure they are truly empty. I.e, the DB exists but has no tables at all
Edit /opt/kaltura/dwh/setup/dwh_ddl_install.sh and change:
#!/bin/bash
to:
#!/bin/bash -x
adding -x will mean all the actions /opt/kaltura/dwh/setup/dwh_ddl_install.sh does will be outputted to STDERR, this will allow you to see where it fails
I add the caption code to the file /opt/kaltura/dwh/setup/dwh_ddl_install.sh as suggested and no errors were displayed on command line.
I can only see that when I run DWH that that specific table is missing from the database. Previously, I saw notes to disable monit and other services but even when I tried that I got an error. I did see errors in kaltlog but I do not think those are related to this issue.
[Tue Apr 19 18:30:13 2016] [error] [client 132.147.31.64] PHP Warning: mkdir(): Permission denied in /opt/kaltura/app/infra/cache/kFileSystemCacheWrapper.php on line 66
[Tue Apr 19 18:30:13 2016] [error] [client 132.147.31.64] PHP Warning: rename(/tmp/cache_v3-7907fd56878e67acf4e732a191fc57ea.cachebuAf2u,/web/cache/feed/cache_v3-86400/79/cache_v3-7907fd56878e67acf4e732a191fc57ea.cache): No such file or directory in /opt/kaltura/app/infra/cache/kFileSystemCacheWrapper.php on line 151
--
==> /opt/kaltura/log/batch/synchronizedistribution-0-2016-04-18.err.log <==
PHP Fatal error: Uncaught exception 'KalturaException' with message 'Internal server error occurred' in /opt/kaltura/app/batch/client/KalturaClientBase.php:894
Stack trace:
#0 /opt/kaltura/app/batch/client/KalturaPlugins/KalturaContentDistributionClientPlugin.php(2813): KalturaClientBase->throwExceptionIfError(Array)
--
thrown in /opt/kaltura/app/batch/client/KalturaClientBase.php on line 894
PHP Fatal error: Uncaught exception 'KalturaException' with message 'Internal server error occurred' in /opt/kaltura/app/batch/client/KalturaClientBase.php:894
Stack trace:
#0 /opt/kaltura/app/batch/client/KalturaClient.php(4130): KalturaClientBase->throwExceptionIfError(Array)
I never got this specific error in previous installs prior to upgrading the latest version.
Are you able to provide a way to manually add the missing tables or a sample of the correct DWH database structure?
Have you had a moment to review this issue? I am still having an issue even after completing an update to the lastest version on a secure and non-secure CentOS 6/6.7 install.
Hi,
The output above is a different issue, probably stemming from faulty permissions on the /opt/kaltura/app/cache but that has nothing to do with Analytics.
As far as the bailing out, never got the output after you put bash -x in /opt/kaltura/dwh/setup/dwh_ddl_install.sh.
What exactly is the issue with the new install? need to go over all the steps and check until something is found.
I will drop the DWH database tables again and run the reconfiguration with both the katlog open and I already added the bash -x to the file mention. No, errors are displayed other than the one mention when DHW configuration is running. That is the only thing failing during the upgrade or during a fresh install on CentOS v6 and v6.7.
I will address the errors during the most recent update in a new thread. As they are separate issues.
I was not able to provide output because other than the bailing out message there was not errors in the logs to post here in the forum. Oddly, this error occurs with both SSL install and non-SSL installs of Kaltura.
This bail out error is specifically related to using the setup dwh script.
/opt/kaltura/bin/kaltura-dwh-config.sh
I am not able to provide you with more information becasuse no error appear even after I add the ‘bash -x’ text to the specified file. This ‘bailed out’ error still exists in the current version of Kaltura CE.
I am not able to determine what is cause bail out error but I do know that the issue with stats working properly is a table is missing from the database. DWH setup appears to only setup 129 out of 130 tables and the errors in the log suggest a table is missing. If I had a copy of the actually DWH database structure I could insert the missing table and likely get the database work.
This would not resolve the issue as to why the database was not setup properly in the first place but I would guess whatever part is breaking is related to the missing table. [quote=“hiphopservers, post:13, topic:4653”]
ERROR 1146 (42S02) at line 1: Table ‘kalturadw.dwh_entry_plays_views’ doesn’t exist
[/quote]
Do you have any suggestions on how to produce more details related to this ‘bail out error’? As mention I was not able to get any errors to display or register with the ‘bash -x’ entry. I think the more direct way to to manually replace the missing table or install the DWH database and structure. This is assuming the only issue is the database not being properly setup during configuration.
I figured out what I was doing wrong that caused the error. I drop all table but not all the stored procedures and functions. Oddly, this has to be done one by one using phpMyAdmin.
After clearing the entire database tables as specified I was able to run the command without errors. Which appears to have fixed the issue on both servers having the DWH issue described.
okay. I got the DWH configuration script to run without a failure or bailout error. However, the stats are still not being recorded on either system I did this on. Any other suggestions or could another process be affecting the proper configuration?
Hi @hiphopservers,
If
# grep ERROR /opt/kaltura/dwh/logs/*.log
returns nothing and you are positive there are views from OUTSIDE the KMC, then let’s go through the troubleshooting steps from https://github.com/kaltura/platform-install-packages/blob/Kajam-11.16.0/doc/kaltura-packages-faq.md. specifically:
check if access files were processed:
mysql> select * from kalturadw_ds.files where insert_time >=%Y%m%d;
check if actual data about entries play and views was inserted:
mysql> select * from kalturadw.dwh_fact_events where event_date_id >=%Y%m%d ;
If no files were processed, check /opt/kaltura/web/logs/-kaltura_apache_accesslog
and ensure you actually see entries where the service is ‘stats’ and the action is ‘collect’. If there aren’t any, then try to play an entry from OUTSIDE KMC and check again. This is the first step.
We’ll know more once you’ve gone though these checks.
Hello,
I’ve read all the discussion and I have the same problem.
When I try this command :
mysql> select * from kalturadw_ds.files where insert_time >=%Y%m%d;
I have this error :
ERROR 1064 (42000): You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ‘%Y-%m-%d’ at line 1
When I check the DB (kalturadw_ds) with PhpMyAdmin, I have nothing in files table.
I think it’s a server configuration issue …
I do a clean install with the latest Kaltura CE version.
Thanks a lot,
GreG
Hello @ZgreG,
%Y%m%d should be replaced with <FULL_YEAR><TWO_DIGIT_MONTH><DAY>, like:
mysql> select * from kalturadw_ds.files where insert_time >=20170601;
I’m very sorry about my ignorance 
Ok, the command is going done and I have data in “kalturadw_ds.files” and “kalturadw.dwh_fact_events”.
So I don’t understand after a fresh install without any problems the Analytics isn’t working …
Everything works well except analytics and the view counter in Content.
Did I miss something, perhaps with Publishers rights or configurations ?
GreG
What’s the output for:
mysql> select * from kalturadw.dwh_fact_events where event_date_id >=%Y%m%d ;
And also:
# grep ERROR /opt/kaltura/dwh/logs/*
?
±--------±---------±--------------±---------------±--------------------±--------------±--------------±-------------------------------------±-----------±-----------±--------------±----------±-----------±-----±--------------±---------±---------------±---------------±-----------±------------±-----------------±-----------±-----±----------±----------±--------------------±-----------------±------------±------±-----------±-----------±--------±---------------+
| file_id | event_id | event_type_id | client_version | event_time | event_date_id | event_hour_id | session_id | partner_id | entry_id | unique_viewer | widget_id | ui_conf_id | uid | current_point | duration | user_ip | user_ip_number | country_id | location_id | process_duration | control_id | seek | new_point | domain_id | entry_media_type_id | entry_partner_id | referrer_id | os_id | browser_id | context_id | user_id | application_id |
±--------±---------±--------------±---------------±--------------------±--------------±--------------±-------------------------------------±-----------±-----------±--------------±----------±-----------±-----±--------------±---------±---------------±---------------±-----------±------------±-----------------±-----------±-----±----------±----------±--------------------±-----------------±------------±------±-----------±-----------±--------±---------------+
| 1 | 54 | 17 | 2.57 | 2017-06-06 15:26:45 | 20170606 | 15 | 85475a47-fe1d-40e5-1063-05cb594ea1d1 | 101 | 0_lvw3ht1g | | _101 | 23448170 | | 10 | 119 | 82.235.134.184 | 1391167160 | 68 | 68 | 0 | -1 | 0 | -1 | 1 | -1 | -1 | 1 | 1 | 1 | -1 | 0 | 0 |
| 1 | 56 | 16 | 2.57 | 2017-06-06 15:26:45 | 20170606 | 15 | 85475a47-fe1d-40e5-1063-05cb594ea1d1 | 101 | 0_lvw3ht1g | | _101 | 23448170 | | 10 | 119 | 82.235.134.184 | 1391167160 | 68 | 68 | 0 | -1 | 0 | -1 | 1 | -1 | -1 | 1 | 1 | 1 | -1 | 0 | 0 |
±--------±---------±--------------±---------------±--------------------±--------------±--------------±-------------------------------------±-----------±-----------±--------------±----------±-----------±-----±--------------±---------±---------------±---------------±-----------±------------±-----------------±-----------±-----±----------±----------±--------------------±-----------------±------------±------±-----------±-----------±--------±---------------+
2 rows in set (0.00 sec)
And nothing for the grep command.
OK, so there are two events here from today.
Do you not see them in KMC->Analytics->Top Content?