I know this question has been asked here several times, but I can’t really find a definitive answer. I have a new install on CentOS 6.7. I am trying to get it 100% good before I attempt to migrate an old (version 6 maybe) site to the new server. I have resolved all issues and get a clean sanity check except for Analytics. Everything passed except ‘[DWH cycle] [FAILED, RC: 255] - [36.212772065]’. I have gone down several rabbit holes following discussions here and elsewhere online. I am now currently back to a fresh database and configuration, to be sure I am starting clean again.
I can find no error in the logs in /opt/kaltura/dwh/logs
I have tried running kaltlog while running the sanity check and see nothing there
There are no locks in kalturadw_ds.locks
This is an all-in-one server, not a cluster
Is it possible that the fail in the sanity check is a false positive? Should I just leave it for a couple days and see if any data actually builds up?
The DWH test run via kaltura-sanity.sh sometimes fails due to too long processing.
What I suggest you do is:
A. make sure you are playing the entry from outside of KMC, because entries played via the KMC preview do not count. You can, for the test, go to preview and embed for the entry, open the tiny URL displayed there and play the entry from there
B. once you did that, in /opt/kaltura/log/kaltura_apache_access.log or /opt/kaltura/log/kaltura_apache_errors_ssl.log depending on whether it is SSL or not, you should see lines that have: /api_v3/index.php?service=stats&action=collect
These records is what the Analytics mechanism uses to populate the DB with.
Once you confirmed we have that, run:
# /opt/kaltura/bin/kaltura-run-dwh.sh
OK, @jess thank you very much. That does seem to be working. I’ll kick at it tomorrow too, to be sure. I did see one error at the end of the kaltura-run-dwh.sh, although the couple of test hits did seem to make it to analytics. The error in the last line of the scripts output was…
is apache:apache 644. I’m guessing it should maybe be kaltura:apache, but that said, a couple logs are owned by apache and a couple by root, and the bulk by kaltura
/opt/kaltura/log/cron.log should be owned by kaltura, group should be apache and have a 664 permission mask.
Less critical though since the only point where this matters in regards to analytics is an attempt to write the output of /opt/kaltura/app/alpha/scripts/dwh/dwh_plays_views_sync.sh to it.
I have this same issue with DWH stats not displaying and DWH is the only thing to fail duing my sanity check. I ran through the steps in this troubleshooting page but encountered several errors.
My stats are still not working and I have not found any specific errors related to stats when running ‘kaltlog’. What do you recommend on doing to track down this issue. I resolved it on my other system but this one is not taking the changes as expected. The option of dropping the data and reinstalling is not available in this situation as it is a live system.
I modified the ownership of the caption file ownership but it did not resolve the error.
I did find the following error message in ‘kaltlog’
==> /opt/kaltura/log/batch/dbcleanup-0-2016-04-03.err.log <==
PHP Fatal error: Uncaught exception 'KalturaClientException' with message 'couldn't connect to host' in /opt/kaltura/app/bient/KalturaClientBase.php:362
Stack trace:
#0 /opt/kaltura/app/batch/client/KalturaClient.php(1299): KalturaClientBase->doQueue()
--
The only thing I have not tried is making a backup of the database and reinstall Kaltura after dropping the database completely. Thing is my personal installation DWH was working before I upgraded to the latest version. That seems to have broke that installation too.
I tried the following commang and was able to login MySQL with out an issue.
mysql -h$DWH_HOST -uetl -p$DWH_PASS kalturadw_ds;
I am not sure why I am being return this error and I would likely associate some of my other errors with this issue. Specifically the ones that relate to stats and batch processing.
How do I properly use these commands to remove the Analytics DBs so I can repopulate them?
PASSW=$MYSQL_SUPER_USER_PASSWD
for i in `mysql -N -p$PASSWD kalturadw -e "show tables"`; do mysql -p$PASSWD kalturadw -e "drop table $i";done
for i in `mysql -N -p$PASSWD kalturadw_ds -e "show tables"`;do mysql -p$PASSWD kalturadw_ds -e "drop table $i";done
for i in `mysql -N -p$PASSWD kalturalog -e "show tables"`;do mysql -p$PASSWD kalturalog -e "drop table $i";done
for i in `mysql -p$PASSWD -e "Show procedure status" |grep kalturadw|awk -F " " '{print $2}'`;do mysql kalturadw -p$PASSWD -e "drop procedure $i;";done
for i in `mysql -p$PASSWD -e "Show procedure status" |grep kalturadw_ds|awk -F " " '{print $2}'`;do mysql kalturadw_ds -p$PASSWD -e "drop procedure $i;";done
Is there a way to do this using phpMyAdmin instead? I get errors or the command do not work properly. Well the system get stuck in a loop and keeps asking me to enter SQL password. Either the syntax is off for my OS or I am not doing it right.
You can do it anyway you want so long as, in the end, you end up with the following DBs created but empty:
| kalturadw |
| kalturadw_bisources |
| kalturadw_ds |
| kalturalog
so that you can re-run /opt/kaltura/bin/kaltura-dwh-config.sh to create the needed tables and populate them.
Thank you for the information. I was able to drop the tables using phpMyAdmin and ran the dwh_config script. However, I got this error in the process before it completed.
1
Error - bailing out!
DWH configured.
I have not idea what that means nor if it affects the proper running of the stats yet.
I then ran the following command to resolve the permission error.
# chown kaltura:apache /opt/kaltura/log/*.log
However, I got the following error after resolving the log permission issue. This is likely related to the failure/error I got during the re-configuration.
ERROR 1146 (42S02) at line 1: Table ‘kalturadw.dwh_entry_plays_views’ doesn’t exist
I do not know the structure of the missing table so I am not able to manually create it.
Also, is there a way to re-calculate the data from the videos and files already uploaded?
Seeing how you got “Error - bailing out!” when running dwh-config, the population obviously did not complete successfully.
I suggest you:
Drop all tables and stored procedures from the following DBs again:
| kalturadw |
| kalturadw_bisources |
| kalturadw_ds |
| kalturalog
Make sure they are truly empty. I.e, the DB exists but has no tables at all
Edit /opt/kaltura/dwh/setup/dwh_ddl_install.sh and change:
#!/bin/bash
to:
#!/bin/bash -x
Rerun /opt/kaltura/bin/kaltura-dwh-config.sh
adding -x will mean all the actions /opt/kaltura/dwh/setup/dwh_ddl_install.sh does will be outputted to STDERR, this will allow you to see where it fails
I add the caption code to the file /opt/kaltura/dwh/setup/dwh_ddl_install.sh as suggested and no errors were displayed on command line.
I can only see that when I run DWH that that specific table is missing from the database. Previously, I saw notes to disable monit and other services but even when I tried that I got an error. I did see errors in kaltlog but I do not think those are related to this issue.
[Tue Apr 19 18:30:13 2016] [error] [client 132.147.31.64] PHP Warning: mkdir(): Permission denied in /opt/kaltura/app/infra/cache/kFileSystemCacheWrapper.php on line 66
[Tue Apr 19 18:30:13 2016] [error] [client 132.147.31.64] PHP Warning: rename(/tmp/cache_v3-7907fd56878e67acf4e732a191fc57ea.cachebuAf2u,/web/cache/feed/cache_v3-86400/79/cache_v3-7907fd56878e67acf4e732a191fc57ea.cache): No such file or directory in /opt/kaltura/app/infra/cache/kFileSystemCacheWrapper.php on line 151
--
==> /opt/kaltura/log/batch/synchronizedistribution-0-2016-04-18.err.log <==
PHP Fatal error: Uncaught exception 'KalturaException' with message 'Internal server error occurred' in /opt/kaltura/app/batch/client/KalturaClientBase.php:894
Stack trace:
#0 /opt/kaltura/app/batch/client/KalturaPlugins/KalturaContentDistributionClientPlugin.php(2813): KalturaClientBase->throwExceptionIfError(Array)
--
thrown in /opt/kaltura/app/batch/client/KalturaClientBase.php on line 894
PHP Fatal error: Uncaught exception 'KalturaException' with message 'Internal server error occurred' in /opt/kaltura/app/batch/client/KalturaClientBase.php:894
Stack trace:
#0 /opt/kaltura/app/batch/client/KalturaClient.php(4130): KalturaClientBase->throwExceptionIfError(Array)
I never got this specific error in previous installs prior to upgrading the latest version.
Are you able to provide a way to manually add the missing tables or a sample of the correct DWH database structure?
Have you had a moment to review this issue? I am still having an issue even after completing an update to the lastest version on a secure and non-secure CentOS 6/6.7 install.
The output above is a different issue, probably stemming from faulty permissions on the /opt/kaltura/app/cache but that has nothing to do with Analytics.
As far as the bailing out, never got the output after you put bash -x in /opt/kaltura/dwh/setup/dwh_ddl_install.sh.
What exactly is the issue with the new install? need to go over all the steps and check until something is found.
I will drop the DWH database tables again and run the reconfiguration with both the katlog open and I already added the bash -x to the file mention. No, errors are displayed other than the one mention when DHW configuration is running. That is the only thing failing during the upgrade or during a fresh install on CentOS v6 and v6.7.
I will address the errors during the most recent update in a new thread. As they are separate issues.
I was not able to provide output because other than the bailing out message there was not errors in the logs to post here in the forum. Oddly, this error occurs with both SSL install and non-SSL installs of Kaltura.
This bail out error is specifically related to using the setup dwh script.
/opt/kaltura/bin/kaltura-dwh-config.sh
I am not able to provide you with more information becasuse no error appear even after I add the ‘bash -x’ text to the specified file. This ‘bailed out’ error still exists in the current version of Kaltura CE.
I am not able to determine what is cause bail out error but I do know that the issue with stats working properly is a table is missing from the database. DWH setup appears to only setup 129 out of 130 tables and the errors in the log suggest a table is missing. If I had a copy of the actually DWH database structure I could insert the missing table and likely get the database work.
This would not resolve the issue as to why the database was not setup properly in the first place but I would guess whatever part is breaking is related to the missing table. [quote=“hiphopservers, post:13, topic:4653”]
ERROR 1146 (42S02) at line 1: Table ‘kalturadw.dwh_entry_plays_views’ doesn’t exist
[/quote]
Do you have any suggestions on how to produce more details related to this ‘bail out error’? As mention I was not able to get any errors to display or register with the ‘bash -x’ entry. I think the more direct way to to manually replace the missing table or install the DWH database and structure. This is assuming the only issue is the database not being properly setup during configuration.
