Sanity Check Failed

hiphopservers · August 10, 2017, 7:09am

I have three errors related to my sanity checks…

[kaltura_logo_animated_blue.flv - 0_azo6xpwd status] [FAILED, RC: 1] - [-.531438368

[Generate thumb] [FAILED, RC: 255] - [.229995813]

I recall the previous notes on how to resolve the issue with DWH but the other two errors I am not familiar with how to resolve.

[DWH cycle] [FAILED, RC: 255] - [5.844153093]

jess · August 10, 2017, 10:28am

Hi @hiphopservers,

That means the attempt to convert the uploaded entry failed. Usually, that would indicate the batch daemon is not running at all. If it is running, then you should check the /opt/kaltura/log/kaltura_batch.log and the logs under /opt/kaltura/log/batch/ for error patterns.

You can start with:

# grep -A 1 -B 1 --color "ERR:\|PHP\|trace\|CRIT\|\[error\]"  /opt/kaltura/log/kaltura_batch.log  /opt/kaltura/log/batch/*log

and then open the relevant log files and trace up to understand the root cause for the failures.

hiphopservers · August 10, 2017, 4:13pm

Hello @jess

After running the following command I got the following errors in my log.

# grep -A 1 -B 1 --color "ERR:\|PHP\|trace\|CRIT\|\[error\]"  /opt/kaltura/log/kaltura_batch.log  /opt/kaltura/log/batch/*log

/opt/kaltura/log/kaltura_batch.log-2017-08-10 03:24:03 [0.000000] [919729595] [1] [BATCH] [KSchedulerConfig->load] NOTICE: loading configuration /opt/kaltura/app/configurations/batch at 1502336630
/opt/kaltura/log/kaltura_batch.log:PHP Warning:  file_put_contents(/opt/kaltura/app/cache//batch/config.ini): failed to open stream: Permission denied in /opt/kaltura/app/batch/scheduler/KSchedule                                                                                                                                    rConfig.class.php on line 185
/opt/kaltura/log/kaltura_batch.log-2017-08-10 03:24:03 [0.049532] [919729595] [2] [BATCH] [KGenericScheduler->loadConfig] INFO: 2.0.2
--
/opt/kaltura/log/batch/dbcleanup-0-2017-08-10.err.log:PHP Fatal error:  Uncaught exception 'KalturaClientException' with message 'Failed connect to cdn.hiphopservers.com:443; Connection refused' i                                                                                                                                    n /opt/kaltura/app/batch/client/KalturaClientBase.php:362
/opt/kaltura/log/batch/dbcleanup-0-2017-08-10.err.log:Stack trace:
/opt/kaltura/log/batch/dbcleanup-0-2017-08-10.err.log-#0 /opt/kaltura/app/batch/client/KalturaClient.php(1070): KalturaClientBase->doQueue()
--
/opt/kaltura/log/batch/dispatcheventnotification-0-2017-08-10.err.log-< Date: Thu, 10 Aug 2017 06:32:27 GMT
/opt/kaltura/log/batch/dispatcheventnotification-0-2017-08-10.err.log:< Server: Apache/2.4.6 (CentOS) OpenSSL/1.0.1e-fips PHP/5.4.16
/opt/kaltura/log/batch/dispatcheventnotification-0-2017-08-10.err.log-< Content-Length: 203
--
/opt/kaltura/log/batch/dispatcheventnotification-0-2017-08-10.err.log-< Date: Thu, 10 Aug 2017 06:33:33 GMT
/opt/kaltura/log/batch/dispatcheventnotification-0-2017-08-10.err.log:< Server: Apache/2.4.6 (CentOS) OpenSSL/1.0.1e-fips PHP/5.4.16
/opt/kaltura/log/batch/dispatcheventnotification-0-2017-08-10.err.log-< Content-Length: 203
--
/opt/kaltura/log/batch/dispatcheventnotification-0-2017-08-10.err.log-< Date: Thu, 10 Aug 2017 06:34:37 GMT
/opt/kaltura/log/batch/dispatcheventnotification-0-2017-08-10.err.log:< Server: Apache/2.4.6 (CentOS) OpenSSL/1.0.1e-fips PHP/5.4.16
/opt/kaltura/log/batch/dispatcheventnotification-0-2017-08-10.err.log-< Content-Length: 203
--
/opt/kaltura/log/batch/dispatcheventnotification-0-2017-08-10.err.log-< Date: Thu, 10 Aug 2017 06:35:57 GMT
/opt/kaltura/log/batch/dispatcheventnotification-0-2017-08-10.err.log:< Server: Apache/2.4.6 (CentOS) OpenSSL/1.0.1e-fips PHP/5.4.16
/opt/kaltura/log/batch/dispatcheventnotification-0-2017-08-10.err.log-< Content-Length: 203
--
/opt/kaltura/log/batch/dispatcheventnotification-0-2017-08-10.err.log-< Date: Thu, 10 Aug 2017 06:36:00 GMT
/opt/kaltura/log/batch/dispatcheventnotification-0-2017-08-10.err.log:< Server: Apache/2.4.6 (CentOS) OpenSSL/1.0.1e-fips PHP/5.4.16
/opt/kaltura/log/batch/dispatcheventnotification-0-2017-08-10.err.log-< Content-Length: 203
--
/opt/kaltura/log/batch/dispatcheventnotification-0-2017-08-10.err.log-< Date: Thu, 10 Aug 2017 06:36:01 GMT
/opt/kaltura/log/batch/dispatcheventnotification-0-2017-08-10.err.log:< Server: Apache/2.4.6 (CentOS) OpenSSL/1.0.1e-fips PHP/5.4.16
/opt/kaltura/log/batch/dispatcheventnotification-0-2017-08-10.err.log-< Content-Length: 203
--
/opt/kaltura/log/batch/dispatcheventnotification-0-2017-08-10.err.log-< Date: Thu, 10 Aug 2017 06:36:01 GMT
/opt/kaltura/log/batch/dispatcheventnotification-0-2017-08-10.err.log:< Server: Apache/2.4.6 (CentOS) OpenSSL/1.0.1e-fips PHP/5.4.16
/opt/kaltura/log/batch/dispatcheventnotification-0-2017-08-10.err.log-< Content-Length: 203
--
/opt/kaltura/log/batch/dispatcheventnotification-0-2017-08-10.err.log-< Date: Thu, 10 Aug 2017 06:36:01 GMT
/opt/kaltura/log/batch/dispatcheventnotification-0-2017-08-10.err.log:< Server: Apache/2.4.6 (CentOS) OpenSSL/1.0.1e-fips PHP/5.4.16
/opt/kaltura/log/batch/dispatcheventnotification-0-2017-08-10.err.log-< Content-Length: 203
--
/opt/kaltura/log/batch/dispatcheventnotification-0-2017-08-10.err.log-< Date: Thu, 10 Aug 2017 06:36:02 GMT
/opt/kaltura/log/batch/dispatcheventnotification-0-2017-08-10.err.log:< Server: Apache/2.4.6 (CentOS) OpenSSL/1.0.1e-fips PHP/5.4.16
/opt/kaltura/log/batch/dispatcheventnotification-0-2017-08-10.err.log-< Content-Length: 203
--
/opt/kaltura/log/batch/dispatcheventnotification-0-2017-08-10.err.log-< Date: Thu, 10 Aug 2017 06:36:02 GMT
/opt/kaltura/log/batch/dispatcheventnotification-0-2017-08-10.err.log:< Server: Apache/2.4.6 (CentOS) OpenSSL/1.0.1e-fips PHP/5.4.16
/opt/kaltura/log/batch/dispatcheventnotification-0-2017-08-10.err.log-< Content-Length: 203
--
/opt/kaltura/log/batch/dispatcheventnotification-0-2017-08-10.err.log-< Date: Thu, 10 Aug 2017 06:36:05 GMT
/opt/kaltura/log/batch/dispatcheventnotification-0-2017-08-10.err.log:< Server: Apache/2.4.6 (CentOS) OpenSSL/1.0.1e-fips PHP/5.4.16
/opt/kaltura/log/batch/dispatcheventnotification-0-2017-08-10.err.log-< Content-Length: 203

hiphopservers · August 10, 2017, 4:15pm

The system would not let me post the rest of the error information. In a single post for some reason here is the rest of the error login information

--
/opt/kaltura/log/batch/dispatcheventnotification-0-2017-08-10.log-Results [1]
/opt/kaltura/log/batch/dispatcheventnotification-0-2017-08-10.log:2017-08-10 02:32:27 [0.000346] [750694821] [30] [BATCH] [KJobHandlerWorker->closeJob] ERR: exception 'Exception' with message 'Err                                                                                                                                    or: Sending HTTP request failed [0]: ' in /opt/kaltura/app/infra/log/KalturaLog.php:83
/opt/kaltura/log/batch/dispatcheventnotification-0-2017-08-10.log:Stack trace:
/opt/kaltura/log/batch/dispatcheventnotification-0-2017-08-10.log-#0 /opt/kaltura/app/batch/batches/KJobHandlerWorker.class.php(364): KalturaLog::err('Error: Sending ...')
--
/opt/kaltura/log/batch/dispatcheventnotification-0-2017-08-10.log-Results [1]
/opt/kaltura/log/batch/dispatcheventnotification-0-2017-08-10.log:2017-08-10 02:33:33 [0.000243] [471200426] [30] [BATCH] [KJobHandlerWorker->closeJob] ERR: exception 'Exception' with message 'Err                                                                                                                                    or: Sending HTTP request failed [0]: ' in /opt/kaltura/app/infra/log/KalturaLog.php:83
/opt/kaltura/log/batch/dispatcheventnotification-0-2017-08-10.log:Stack trace:
/opt/kaltura/log/batch/dispatcheventnotification-0-2017-08-10.log-#0 /opt/kaltura/app/batch/batches/KJobHandlerWorker.class.php(364): KalturaLog::err('Error: Sending ...')
--
/opt/kaltura/log/batch/dispatcheventnotification-0-2017-08-10.log-Results [1]
/opt/kaltura/log/batch/dispatcheventnotification-0-2017-08-10.log:2017-08-10 02:34:37 [0.000269] [620789996] [30] [BATCH] [KJobHandlerWorker->closeJob] ERR: exception 'Exception' with message 'Err                                                                                                                                    or: Sending HTTP request failed [0]: ' in /opt/kaltura/app/infra/log/KalturaLog.php:83
/opt/kaltura/log/batch/dispatcheventnotification-0-2017-08-10.log:Stack trace:
/opt/kaltura/log/batch/dispatcheventnotification-0-2017-08-10.log-#0 /opt/kaltura/app/batch/batches/KJobHandlerWorker.class.php(364): KalturaLog::err('Error: Sending ...')
--
/opt/kaltura/log/batch/dispatcheventnotification-0-2017-08-10.log-Results [1]
/opt/kaltura/log/batch/dispatcheventnotification-0-2017-08-10.log:2017-08-10 02:35:57 [0.000293] [470478880] [30] [BATCH] [KJobHandlerWorker->closeJob] ERR: exception 'Exception' with message 'Err                                                                                                                                    or: Sending HTTP request failed [0]: ' in /opt/kaltura/app/infra/log/KalturaLog.php:83
/opt/kaltura/log/batch/dispatcheventnotification-0-2017-08-10.log:Stack trace:
/opt/kaltura/log/batch/dispatcheventnotification-0-2017-08-10.log-#0 /opt/kaltura/app/batch/batches/KJobHandlerWorker.class.php(364): KalturaLog::err('Error: Sending ...')
--
/opt/kaltura/log/batch/dispatcheventnotification-0-2017-08-10.log-Results [1]
/opt/kaltura/log/batch/dispatcheventnotification-0-2017-08-10.log:2017-08-10 02:36:00 [0.000304] [1177480084] [30] [BATCH] [KJobHandlerWorker->closeJob] ERR: exception 'Exception' with message 'Er                                                                                                                                    ror: Sending HTTP request failed [0]: ' in /opt/kaltura/app/infra/log/KalturaLog.php:83
/opt/kaltura/log/batch/dispatcheventnotification-0-2017-08-10.log:Stack trace:
/opt/kaltura/log/batch/dispatcheventnotification-0-2017-08-10.log-#0 /opt/kaltura/app/batch/batches/KJobHandlerWorker.class.php(364): KalturaLog::err('Error: Sending ...')
--
/opt/kaltura/log/batch/dispatcheventnotification-0-2017-08-10.log-Results [1]
/opt/kaltura/log/batch/dispatcheventnotification-0-2017-08-10.log:2017-08-10 02:36:01 [0.000132] [1177480084] [66] [BATCH] [KJobHandlerWorker->closeJob] ERR: exception 'Exception' with message 'Er                                                                                                                                    ror: Sending HTTP request failed [0]: ' in /opt/kaltura/app/infra/log/KalturaLog.php:83
/opt/kaltura/log/batch/dispatcheventnotification-0-2017-08-10.log:Stack trace:
/opt/kaltura/log/batch/dispatcheventnotification-0-2017-08-10.log-#0 /opt/kaltura/app/batch/batches/KJobHandlerWorker.class.php(364): KalturaLog::err('Error: Sending ...')
--
/opt/kaltura/log/batch/dispatcheventnotification-0-2017-08-10.log-Results [1]
/opt/kaltura/log/batch/dispatcheventnotification-0-2017-08-10.log:2017-08-10 02:36:01 [0.000108] [1177480084] [102] [BATCH] [KJobHandlerWorker->closeJob] ERR: exception 'Exception' with message 'E                                                                                                                                    rror: Sending HTTP request failed [0]: ' in /opt/kaltura/app/infra/log/KalturaLog.php:83
/opt/kaltura/log/batch/dispatcheventnotification-0-2017-08-10.log:Stack trace:
/opt/kaltura/log/batch/dispatcheventnotification-0-2017-08-10.log-#0 /opt/kaltura/app/batch/batches/KJobHandlerWorker.class.php(364): KalturaLog::err('Error: Sending ...')
--
/opt/kaltura/log/batch/dispatcheventnotification-0-2017-08-10.log-Results [1]
/opt/kaltura/log/batch/dispatcheventnotification-0-2017-08-10.log:2017-08-10 02:36:01 [0.000129] [1177480084] [138] [BATCH] [KJobHandlerWorker->closeJob] ERR: exception 'Exception' with message 'E                                                                                                                                    rror: Sending HTTP request failed [0]: ' in /opt/kaltura/app/infra/log/KalturaLog.php:83
/opt/kaltura/log/batch/dispatcheventnotification-0-2017-08-10.log:Stack trace:
/opt/kaltura/log/batch/dispatcheventnotification-0-2017-08-10.log-#0 /opt/kaltura/app/batch/batches/KJobHandlerWorker.class.php(364): KalturaLog::err('Error: Sending ...')
--
/opt/kaltura/log/batch/dispatcheventnotification-0-2017-08-10.log-Results [1]
/opt/kaltura/log/batch/dispatcheventnotification-0-2017-08-10.log:2017-08-10 02:36:02 [0.000112] [1177480084] [174] [BATCH] [KJobHandlerWorker->closeJob] ERR: exception 'Exception' with message 'E                                                                                                                                    rror: Sending HTTP request failed [0]: ' in /opt/kaltura/app/infra/log/KalturaLog.php:83
/opt/kaltura/log/batch/dispatcheventnotification-0-2017-08-10.log:Stack trace:
/opt/kaltura/log/batch/dispatcheventnotification-0-2017-08-10.log-#0 /opt/kaltura/app/batch/batches/KJobHandlerWorker.class.php(364): KalturaLog::err('Error: Sending ...')
--
/opt/kaltura/log/batch/dispatcheventnotification-0-2017-08-10.log-Results [1]
/opt/kaltura/log/batch/dispatcheventnotification-0-2017-08-10.log:2017-08-10 02:36:02 [0.000112] [1177480084] [210] [BATCH] [KJobHandlerWorker->closeJob] ERR: exception 'Exception' with message 'E                                                                                                                                    rror: Sending HTTP request failed [0]: ' in /opt/kaltura/app/infra/log/KalturaLog.php:83
/opt/kaltura/log/batch/dispatcheventnotification-0-2017-08-10.log:Stack trace:
/opt/kaltura/log/batch/dispatcheventnotification-0-2017-08-10.log-#0 /opt/kaltura/app/batch/batches/KJobHandlerWorker.class.php(364): KalturaLog::err('Error: Sending ...')
--
/opt/kaltura/log/batch/dispatcheventnotification-0-2017-08-10.log-Results [1]
/opt/kaltura/log/batch/dispatcheventnotification-0-2017-08-10.log:2017-08-10 02:36:05 [0.000283] [20672734] [30] [BATCH] [KJobHandlerWorker->closeJob] ERR: exception 'Exception' with message 'Erro                                                                                                                                    r: Sending HTTP request failed [0]: ' in /opt/kaltura/app/infra/log/KalturaLog.php:83
/opt/kaltura/log/batch/dispatcheventnotification-0-2017-08-10.log:Stack trace:
/opt/kaltura/log/batch/dispatcheventnotification-0-2017-08-10.log-#0 /opt/kaltura/app/batch/batches/KJobHandlerWorker.class.php(364): KalturaLog::err('Error: Sending ...')
--
/opt/kaltura/log/batch/dispatcheventnotification-0-2017-08-10.log-Results [1]
/opt/kaltura/log/batch/dispatcheventnotification-0-2017-08-10.log:2017-08-10 02:36:05 [0.000119] [20672734] [66] [BATCH] [KJobHandlerWorker->closeJob] ERR: exception 'Exception' with message 'Erro                                                                                                                                    r: Sending HTTP request failed [0]: ' in /opt/kaltura/app/infra/log/KalturaLog.php:83
/opt/kaltura/log/batch/dispatcheventnotification-0-2017-08-10.log:Stack trace:
/opt/kaltura/log/batch/dispatcheventnotification-0-2017-08-10.log-#0 /opt/kaltura/app/batch/batches/KJobHandlerWorker.class.php(364): KalturaLog::err('Error: Sending ...')
--
/opt/kaltura/log/batch/dispatcheventnotification-0-2017-08-10.log-Results [1]
/opt/kaltura/log/batch/dispatcheventnotification-0-2017-08-10.log:2017-08-10 02:36:06 [0.000123] [20672734] [102] [BATCH] [KJobHandlerWorker->closeJob] ERR: exception 'Exception' with message 'Err                                                                                                                                    or: Sending HTTP request failed [0]: ' in /opt/kaltura/app/infra/log/KalturaLog.php:83
/opt/kaltura/log/batch/dispatcheventnotification-0-2017-08-10.log:Stack trace:
/opt/kaltura/log/batch/dispatcheventnotification-0-2017-08-10.log-#0 /opt/kaltura/app/batch/batches/KJobHandlerWorker.class.php(364): KalturaLog::err('Error: Sending ...')
--
/opt/kaltura/log/batch/dispatcheventnotification-0-2017-08-10.log-Results [1]
/opt/kaltura/log/batch/dispatcheventnotification-0-2017-08-10.log:2017-08-10 02:36:06 [0.000130] [20672734] [138] [BATCH] [KJobHandlerWorker->closeJob] ERR: exception 'Exception' with message 'Err                                                                                                                                    or: Sending HTTP request failed [0]: ' in /opt/kaltura/app/infra/log/KalturaLog.php:83
/opt/kaltura/log/batch/dispatcheventnotification-0-2017-08-10.log:Stack trace:
/opt/kaltura/log/batch/dispatcheventnotification-0-2017-08-10.log-#0 /opt/kaltura/app/batch/batches/KJobHandlerWorker.class.php(364): KalturaLog::err('Error: Sending ...')
--
/opt/kaltura/log/batch/dispatcheventnotification-0-2017-08-10.log-Results [1]
/opt/kaltura/log/batch/dispatcheventnotification-0-2017-08-10.log:2017-08-10 02:36:06 [0.000244] [20672734] [174] [BATCH] [KJobHandlerWorker->closeJob] ERR: exception 'Exception' with message 'Err                                                                                                                                    or: Sending HTTP request failed [0]: ' in /opt/kaltura/app/infra/log/KalturaLog.php:83
/opt/kaltura/log/batch/dispatcheventnotification-0-2017-08-10.log:Stack trace:
/opt/kaltura/log/batch/dispatcheventnotification-0-2017-08-10.log-#0 /opt/kaltura/app/batch/batches/KJobHandlerWorker.class.php(364): KalturaLog::err('Error: Sending ...')
--
/opt/kaltura/log/batch/dispatcheventnotification-0-2017-08-10.log-Results [1]
/opt/kaltura/log/batch/dispatcheventnotification-0-2017-08-10.log:2017-08-10 02:36:07 [0.000109] [20672734] [210] [BATCH] [KJobHandlerWorker->closeJob] ERR: exception 'Exception' with message 'Err                                                                                                                                    or: Sending HTTP request failed [0]: ' in /opt/kaltura/app/infra/log/KalturaLog.php:83
/opt/kaltura/log/batch/dispatcheventnotification-0-2017-08-10.log:Stack trace:
/opt/kaltura/log/batch/dispatcheventnotification-0-2017-08-10.log-#0 /opt/kaltura/app/batch/batches/KJobHandlerWorker.class.php(364): KalturaLog::err('Error: Sending ...')
--
/opt/kaltura/log/batch/dispatcheventnotification-0-2017-08-10.log-Results [1]
/opt/kaltura/log/batch/dispatcheventnotification-0-2017-08-10.log:2017-08-10 02:36:07 [0.000102] [20672734] [246] [BATCH] [KJobHandlerWorker->closeJob] ERR: exception 'Exception' with message 'Err                                                                                                                                    or: Sending HTTP request failed [0]: ' in /opt/kaltura/app/infra/log/KalturaLog.php:83
/opt/kaltura/log/batch/dispatcheventnotification-0-2017-08-10.log:Stack trace:
/opt/kaltura/log/batch/dispatcheventnotification-0-2017-08-10.log-#0 /opt/kaltura/app/batch/batches/KJobHandlerWorker.class.php(364): KalturaLog::err('Error: Sending ...')
/opt/kaltura/log/batch/dispatcheventnotification-0-2017-08-10.log-Results [1]
/opt/kaltura/log/batch/dispatcheventnotification-0-2017-08-10.log:2017-08-10 02:37:02 [0.000105] [577345084] [66] [BATCH] [KJobHandlerWorker->closeJob] ERR: exception 'Exception' with message 'Err                                                                                                                                    or: Sending HTTP request failed [0]: ' in /opt/kaltura/app/infra/log/KalturaLog.php:83
/opt/kaltura/log/batch/dispatcheventnotification-0-2017-08-10.log:Stack trace:
/opt/kaltura/log/batch/dispatcheventnotification-0-2017-08-10.log-#0 /opt/kaltura/app/batch/batches/KJobHandlerWorker.class.php(364): KalturaLog::err('Error: Sending ...')
--
/opt/kaltura/log/batch/dispatcheventnotification-0-2017-08-10.log-Results [1]
/opt/kaltura/log/batch/dispatcheventnotification-0-2017-08-10.log:2017-08-10 02:37:02 [0.000147] [577345084] [102] [BATCH] [KJobHandlerWorker->closeJob] ERR: exception 'Exception' with message 'Er                                                                                                                                    ror: Sending HTTP request failed [0]: ' in /opt/kaltura/app/infra/log/KalturaLog.php:83
/opt/kaltura/log/batch/dispatcheventnotification-0-2017-08-10.log:Stack trace:
/opt/kaltura/log/batch/dispatcheventnotification-0-2017-08-10.log-#0 /opt/kaltura/app/batch/batches/KJobHandlerWorker.class.php(364): KalturaLog::err('Error: Sending ...')
--
/opt/kaltura/log/batch/dispatcheventnotification-0-2017-08-10.log-Results [1]
/opt/kaltura/log/batch/dispatcheventnotification-0-2017-08-10.log:2017-08-10 02:37:03 [0.000109] [577345084] [138] [BATCH] [KJobHandlerWorker->closeJob] ERR: exception 'Exception' with message 'Er                                                                                                                                    ror: Sending HTTP request failed [0]: ' in /opt/kaltura/app/infra/log/KalturaLog.php:83
/opt/kaltura/log/batch/dispatcheventnotification-0-2017-08-10.log:Stack trace:
/opt/kaltura/log/batch/dispatcheventnotification-0-2017-08-10.log-#0 /opt/kaltura/app/batch/batches/KJobHandlerWorker.class.php(364): KalturaLog::err('Error: Sending ...')
--
/opt/kaltura/log/batch/dispatcheventnotification-0-2017-08-10.log-Results [1]
/opt/kaltura/log/batch/dispatcheventnotification-0-2017-08-10.log:2017-08-10 02:37:03 [0.000103] [577345084] [174] [BATCH] [KJobHandlerWorker->closeJob] ERR: exception 'Exception' with message 'Er                                                                                                                                    ror: Sending HTTP request failed [0]: ' in /opt/kaltura/app/infra/log/KalturaLog.php:83
/opt/kaltura/log/batch/dispatcheventnotification-0-2017-08-10.log:Stack trace:
/opt/kaltura/log/batch/dispatcheventnotification-0-2017-08-10.log-#0 /opt/kaltura/app/batch/batches/KJobHandlerWorker.class.php(364): KalturaLog::err('Error: Sending ...')
--
/opt/kaltura/log/batch/dispatcheventnotification-0-2017-08-10.log-Results [1]
/opt/kaltura/log/batch/dispatcheventnotification-0-2017-08-10.log:2017-08-10 02:37:05 [0.000262] [1329946726] [30] [BATCH] [KJobHandlerWorker->closeJob] ERR: exception 'Exception' with message 'Er                                                                                                                                    ror: Sending HTTP request failed [0]: ' in /opt/kaltura/app/infra/log/KalturaLog.php:83
/opt/kaltura/log/batch/dispatcheventnotification-0-2017-08-10.log:Stack trace:
/opt/kaltura/log/batch/dispatcheventnotification-0-2017-08-10.log-#0 /opt/kaltura/app/batch/batches/KJobHandlerWorker.class.php(364): KalturaLog::err('Error: Sending ...')
--
/opt/kaltura/log/batch/dispatcheventnotification-0-2017-08-10.log-Results [1]
/opt/kaltura/log/batch/dispatcheventnotification-0-2017-08-10.log:2017-08-10 02:37:05 [0.000149] [1329946726] [66] [BATCH] [KJobHandlerWorker->closeJob] ERR: exception 'Exception' with message 'Er                                                                                                                                    ror: Sending HTTP request failed [0]: ' in /opt/kaltura/app/infra/log/KalturaLog.php:83
/opt/kaltura/log/batch/dispatcheventnotification-0-2017-08-10.log:Stack trace:
/opt/kaltura/log/batch/dispatcheventnotification-0-2017-08-10.log-#0 /opt/kaltura/app/batch/batches/KJobHandlerWorker.class.php(364): KalturaLog::err('Error: Sending ...')
--
/opt/kaltura/log/batch/dispatcheventnotification-0-2017-08-10.log-Results [1]
/opt/kaltura/log/batch/dispatcheventnotification-0-2017-08-10.log:2017-08-10 02:37:07 [0.000277] [1635415609] [30] [BATCH] [KJobHandlerWorker->closeJob] ERR: exception 'Exception' with message 'Er                                                                                                                                    ror: Sending HTTP request failed [0]: ' in /opt/kaltura/app/infra/log/KalturaLog.php:83
/opt/kaltura/log/batch/dispatcheventnotification-0-2017-08-10.log:Stack trace:
/opt/kaltura/log/batch/dispatcheventnotification-0-2017-08-10.log-#0 /opt/kaltura/app/batch/batches/KJobHandlerWorker.class.php(364): KalturaLog::err('Error: Sending ...')
--
/opt/kaltura/log/batch/dispatcheventnotification-0-2017-08-10.log-Results [1]
/opt/kaltura/log/batch/dispatcheventnotification-0-2017-08-10.log:2017-08-10 02:37:07 [0.000173] [1635415609] [66] [BATCH] [KJobHandlerWorker->closeJob] ERR: exception 'Exception' with message 'Er                                                                                                                                    ror: Sending HTTP request failed [0]: ' in /opt/kaltura/app/infra/log/KalturaLog.php:83
/opt/kaltura/log/batch/dispatcheventnotification-0-2017-08-10.log:Stack trace:
/opt/kaltura/log/batch/dispatcheventnotification-0-2017-08-10.log-#0 /opt/kaltura/app/batch/batches/KJobHandlerWorker.class.php(364): KalturaLog::err('Error: Sending ...')
--
/opt/kaltura/log/batch/dispatcheventnotification-0-2017-08-10.log-Results [1]
/opt/kaltura/log/batch/dispatcheventnotification-0-2017-08-10.log:2017-08-10 02:37:07 [0.000144] [1635415609] [102] [BATCH] [KJobHandlerWorker->closeJob] ERR: exception 'Exception' with message 'E                                                                                                                                    rror: Sending HTTP request failed [0]: ' in /opt/kaltura/app/infra/log/KalturaLog.php:83
/opt/kaltura/log/batch/dispatcheventnotification-0-2017-08-10.log:Stack trace:
/opt/kaltura/log/batch/dispatcheventnotification-0-2017-08-10.log-#0 /opt/kaltura/app/batch/batches/KJobHandlerWorker.class.php(364): KalturaLog::err('Error: Sending ...')
--
/opt/kaltura/log/batch/dispatcheventnotification-0-2017-08-10.log-Results [1]
/opt/kaltura/log/batch/dispatcheventnotification-0-2017-08-10.log:2017-08-10 02:37:08 [0.000107] [1635415609] [138] [BATCH] [KJobHandlerWorker->closeJob] ERR: exception 'Exception' with message 'E                                                                                                                                    rror: Sending HTTP request failed [0]: ' in /opt/kaltura/app/infra/log/KalturaLog.php:83
/opt/kaltura/log/batch/dispatcheventnotification-0-2017-08-10.log:Stack trace:
/opt/kaltura/log/batch/dispatcheventnotification-0-2017-08-10.log-#0 /opt/kaltura/app/batch/batches/KJobHandlerWorker.class.php(364): KalturaLog::err('Error: Sending ...')
--
/opt/kaltura/log/batch/dispatcheventnotification-0-2017-08-10.log-Results [1]
/opt/kaltura/log/batch/dispatcheventnotification-0-2017-08-10.log:2017-08-10 02:37:08 [0.000114] [1635415609] [174] [BATCH] [KJobHandlerWorker->closeJob] ERR: exception 'Exception' with message 'E                                                                                                                                    rror: Sending HTTP request failed [0]: ' in /opt/kaltura/app/infra/log/KalturaLog.php:83
/opt/kaltura/log/batch/dispatcheventnotification-0-2017-08-10.log:Stack trace:
/opt/kaltura/log/batch/dispatcheventnotification-0-2017-08-10.log-#0 /opt/kaltura/app/batch/batches/KJobHandlerWorker.class.php(364): KalturaLog::err('Error: Sending ...')
--
/opt/kaltura/log/batch/dispatcheventnotification-0-2017-08-10.log-Results [1]
/opt/kaltura/log/batch/dispatcheventnotification-0-2017-08-10.log:2017-08-10 02:37:13 [0.000260] [435852490] [30] [BATCH] [KJobHandlerWorker->closeJob] ERR: exception 'Exception' with message 'Err                                                                                                                                    or: Sending HTTP request failed [0]: ' in /opt/kaltura/app/infra/log/KalturaLog.php:83
/opt/kaltura/log/batch/dispatcheventnotification-0-2017-08-10.log:Stack trace:
/opt/kaltura/log/batch/dispatcheventnotification-0-2017-08-10.log-#0 /opt/kaltura/app/batch/batches/KJobHandlerWorker.class.php(364): KalturaLog::err('Error: Sending ...')
--
/opt/kaltura/log/batch/dispatcheventnotification-0-2017-08-10.log-Results [1]
/opt/kaltura/log/batch/dispatcheventnotification-0-2017-08-10.log:2017-08-10 02:37:13 [0.000114] [435852490] [66] [BATCH] [KJobHandlerWorker->closeJob] ERR: exception 'Exception' with message 'Err                                                                                                                                    or: Sending HTTP request failed [0]: ' in /opt/kaltura/app/infra/log/KalturaLog.php:83
/opt/kaltura/log/batch/dispatcheventnotification-0-2017-08-10.log:Stack trace:
/opt/kaltura/log/batch/dispatcheventnotification-0-2017-08-10.log-#0 /opt/kaltura/app/batch/batches/KJobHandlerWorker.class.php(364): KalturaLog::err('Error: Sending ...')
--
/opt/kaltura/log/batch/dispatcheventnotification-0-2017-08-10.log-Results [1]
/opt/kaltura/log/batch/dispatcheventnotification-0-2017-08-10.log:2017-08-10 02:38:06 [0.000117] [258523225] [102] [BATCH] [KJobHandlerWorker->closeJob] ERR: exception 'Exception' with message 'Er                                                                                                                                    ror: Sending HTTP request failed [0]: ' in /opt/kaltura/app/infra/log/KalturaLog.php:83
/opt/kaltura/log/batch/dispatcheventnotification-0-2017-08-10.log:Stack trace:
/opt/kaltura/log/batch/dispatcheventnotification-0-2017-08-10.log-#0 /opt/kaltura/app/batch/batches/KJobHandlerWorker.class.php(364): KalturaLog::err('Error: Sending ...')
--
/opt/kaltura/log/batch/dispatcheventnotification-0-2017-08-10.log-Results [1]
/opt/kaltura/log/batch/dispatcheventnotification-0-2017-08-10.log:2017-08-10 02:38:07 [0.000119] [258523225] [138] [BATCH] [KJobHandlerWorker->closeJob] ERR: exception 'Exception' with message 'Er                                                                                                                                    ror: Sending HTTP request failed [0]: ' in /opt/kaltura/app/infra/log/KalturaLog.php:83
/opt/kaltura/log/batch/dispatcheventnotification-0-2017-08-10.log:Stack trace:
/opt/kaltura/log/batch/dispatcheventnotification-0-2017-08-10.log-#0 /opt/kaltura/app/batch/batches/KJobHandlerWorker.class.php(364): KalturaLog::err('Error: Sending ...')
--
/opt/kaltura/log/batch/dispatcheventnotification-0-2017-08-10.log-Results [1]
/opt/kaltura/log/batch/dispatcheventnotification-0-2017-08-10.log:2017-08-10 02:38:07 [0.000121] [258523225] [174] [BATCH] [KJobHandlerWorker->closeJob] ERR: exception 'Exception' with message 'Er                                                                                                                                    ror: Sending HTTP request failed [0]: ' in /opt/kaltura/app/infra/log/KalturaLog.php:83
/opt/kaltura/log/batch/dispatcheventnotification-0-2017-08-10.log:Stack trace:
/opt/kaltura/log/batch/dispatcheventnotification-0-2017-08-10.log-#0 /opt/kaltura/app/batch/batches/KJobHandlerWorker.class.php(364): KalturaLog::err('Error: Sending ...')
--
/opt/kaltura/log/batch/dispatcheventnotification-0-2017-08-10.log-Results [1]
/opt/kaltura/log/batch/dispatcheventnotification-0-2017-08-10.log:2017-08-10 02:38:07 [0.000155] [258523225] [210] [BATCH] [KJobHandlerWorker->closeJob] ERR: exception 'Exception' with message 'Er                                                                                                                                    ror: Sending HTTP request failed [0]: ' in /opt/kaltura/app/infra/log/KalturaLog.php:83
/opt/kaltura/log/batch/dispatcheventnotification-0-2017-08-10.log:Stack trace:
/opt/kaltura/log/batch/dispatcheventnotification-0-2017-08-10.log-#0 /opt/kaltura/app/batch/batches/KJobHandlerWorker.class.php(364): KalturaLog::err('Error: Sending ...')
--
/opt/kaltura/log/batch/dispatcheventnotification-0-2017-08-10.log-Results [1]
/opt/kaltura/log/batch/dispatcheventnotification-0-2017-08-10.log:2017-08-10 02:38:08 [0.000113] [258523225] [246] [BATCH] [KJobHandlerWorker->closeJob] ERR: exception 'Exception' with message 'Er                                                                                                                                    
/opt/kaltura/log/batch/dispatcheventnotification-0-2017-08-10.log-Results [1]
/opt/kaltura/log/batch/dispatcheventnotification-0-2017-08-10.log:2017-08-10 02:38:10 [0.000110] [1542676521] [66] [BATCH] [KJobHandlerWorker->closeJob] ERR: exception 'Exception' with message 'Error: Sending HTTP request failed [0]: ' in /opt/kaltura/app/infra/log/KalturaLog.php:83
/opt/kaltura/log/batch/dispatcheventnotification-0-2017-08-10.log:Stack trace:
/opt/kaltura/log/batch/dispatcheventnotification-0-2017-08-10.log-#0 /opt/kaltura/app/batch/batches/KJobHandlerWorker.class.php(364): KalturaLog::err('Error: Sending ...')
--
/opt/kaltura/log/batch/dispatcheventnotification-0-2017-08-10.log-Results [1]
/opt/kaltura/log/batch/dispatcheventnotification-0-2017-08-10.log:2017-08-10 02:38:11 [0.000150] [1542676521] [174] [BATCH] [KJobHandlerWorker->closeJob] ERR: exception 'Exception' with message 'E                                                                                                                                    rror: Sending HTTP request failed [0]: ' in /opt/kaltura/app/infra/log/KalturaLog.php:83
/opt/kaltura/log/batch/dispatcheventnotification-0-2017-08-10.log:Stack trace:
/opt/kaltura/log/batch/dispatcheventnotification-0-2017-08-10.log-#0 /opt/kaltura/app/batch/batches/KJobHandlerWorker.class.php(364): KalturaLog::err('Error: Sending ...')

/opt/kaltura/log/batch/jobsuspender-0-2017-08-10.err.log:PHP Fatal error:  Uncaught exception 'KalturaClientException' with message 'Failed connect to cdn.hiphopservers.com:443; Connection refused                                                                                                                                    ' in /opt/kaltura/app/batch/client/KalturaClientBase.php:362
/opt/kaltura/log/batch/jobsuspender-0-2017-08-10.err.log:Stack trace:
/opt/kaltura/log/batch/jobsuspender-0-2017-08-10.err.log-#0 /opt/kaltura/app/batch/client/KalturaClient.php(1294): KalturaClientBase->doQueue()
--
/opt/kaltura/log/batch/kschedulehelper-0-2017-08-10.log-2017-08-10 02:30:33 [0.000066] [443491777] [7] [BATCH] [KalturaClientBase->doCurl] NOTICE: post: {"format":"3","ignoreNull":true,"clientTag"                                                                                                                                    :"batch: cdn.hiphopservers.com KScheduleHelper index: 0 sessionId: 443491777","apiVersion":"3.3.0","partnerId":"-1","ks":"[random_session_key_string}","kalsig":"[KAL_SIG_STRING]"}
/opt/kaltura/log/batch/kschedulehelper-0-2017-08-10.log:2017-08-10 02:30:33 [0.159703] [443491777] [8] [BATCH] [KScheduleHelper->run] ERR: exception 'Exception' with message 'System is not yet ready - ping failed' in /opt/kaltura/app/infra/log/KalturaLog.php:83
/opt/kaltura/log/batch/kschedulehelper-0-2017-08-10.log:Stack trace:
/opt/kaltura/log/batch/kschedulehelper-0-2017-08-10.log-#0 /opt/kaltura/app/batch/batches/KScheduleHelper.class.php(42): KalturaLog::err('System is not y...')
--
/opt/kaltura/log/batch/partnerloadcleanup-0-2017-08-10.err.log:PHP Fatal error:  Uncaught exception 'KalturaClientException' with message 'Failed connect to cdn.hiphopservers.com:443; Connection r                                                                                                                                    efused' in /opt/kaltura/app/batch/client/KalturaClientBase.php:362
/opt/kaltura/log/batch/partnerloadcleanup-0-2017-08-10.err.log:Stack trace:
/opt/kaltura/log/batch/partnerloadcleanup-0-2017-08-10.err.log-#0 /opt/kaltura/app/batch/client/KalturaClient.php(1380): KalturaClientBase->doQueue()

Now, I will note I was having an issue with my pem certificate but that should not affect Apache’s SSL handling. I have updated the pem to resolve the issue and restarted both processes that use the pem. Which are NGINX and MONIT. I did not bother to restart Apache as I didnt not see that being and issue.

jess · August 10, 2017, 4:17pm

Hi @hiphopservers,

See Converting process never ending & internal error on dashboard

hiphopservers · August 10, 2017, 4:45pm

Hi @jess

I ran the commands in the thread provided and still got the following errors during my sanity check. Specifically, I ran these commands.

and also these commands

Upload new video - stuck on convert profile step

# php /opt/kaltura/app/deployment/updates/update.php -i -d &gt;&gt; /opt/kaltura/log/kalt_up.log 2&gt;&1
# php /opt/kaltura/app/deployment/updates/update.php -i -s &gt;&gt; /opt/kaltura/log/kalt_up.log 2&gt;&1
# php /opt/kaltura/app/deployment/base/scripts/installPlugins.php &gt;&gt; /opt/kaltura/log/kalt_up.log 2&gt;&1

Then I hand another sanity check and got these results again.

[Delete flavor param] [PASSED, RC: 0] - [.279551765]
PHP Fatal error:  Uncaught exception 'KalturaClientException' with message 'failed to unserialize server result
<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
<html><head>
<title>500 Internal Server Error</title>
</head><body>
<h1>Internal Server Error</h1>
<p>The server encountered an internal error or
misconfiguration and was unable to complete
your request.</p>
<p>Please contact the server administrator at
 emailaddress@mydomain.com to inform them of the time this error occurred,
 and the actions you performed just before this error.</p>
<p>More information about this error may be available
in the server error log.</p>
</body></html>
' in /opt/kaltura/web/content/clientlibs/php5/KalturaClientBase.php:401
Stack trace:
#0 /opt/kaltura/web/content/clientlibs/php5/KalturaPlugins/KalturaEventNotificationClientPlugin.php(732): KalturaClientBase->doQueue()
#1 /opt/kaltura/bin/http_notifications.php(54): KalturaEventNotificationTemplateService->update(45, Object(KalturaHttpNotificationTemplate))
#2 {main}
  thrown in /opt/kaltura/web/content/clientlibs/php5/KalturaClientBase.php on line 401
[HTTP notification: ] [PASSED, RC: 0] - [.427463308]
PHP Fatal error:  Uncaught exception 'KalturaClientException' with message 'failed to unserialize server result
<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
<html><head>
<title>500 Internal Server Error</title>
</head><body>
<h1>Internal Server Error</h1>
<p>The server encountered an internal error or
misconfiguration and was unable to complete
your request.</p>
<p>Please contact the server administrator at
 emailaddress@mydomain.com to inform them of the time this error occurred,
 and the actions you performed just before this error.</p>
<p>More information about this error may be available
in the server error log.</p>
</body></html>
' in /opt/kaltura/web/content/clientlibs/php5/KalturaClientBase.php:401
Stack trace:
#0 /opt/kaltura/web/content/clientlibs/php5/KalturaPlugins/KalturaEventNotificationClientPlugin.php(732): KalturaClientBase->doQueue()
#1 /opt/kaltura/bin/mail_notifications.php(49): KalturaEventNotificationTemplateService->update(46, Object(KalturaEmailNotificationTemplate))
#2 {main}
  thrown in /opt/kaltura/web/content/clientlibs/php5/KalturaClientBase.php on line 401
[Mail notification: ] [PASSED, RC: 0] - [.427463308]
[Upload content kaltura_logo_animated_blue.flv] [PASSED, RC: 0] - [.851350663]
Napping 10 seconds to allow entry 0_e3ia6qc7 to digest..
Napping 10 seconds to allow entry 0_e3ia6qc7 to digest..
Napping 10 seconds to allow entry 0_e3ia6qc7 to digest..
[kaltura_logo_animated_blue.flv - 0_e3ia6qc7 status] [FAILED, RC: 1] - [-.529051638]
Napping 30 seconds to allow mail to be sent out..

hiphopservers · August 11, 2017, 8:09am

@jess

So I tried my sanity check after completing the commands. I am am starting to think because of the number of files in my folder and managed by the system. So all the permissions must not have been fix when I ran my last sanity check.

I ran it again a few minutes ago and it return all positive results except DWH which I will work on tomorrow now that the other issues are fixed. So the commands I ran worked but I had to wait a while for all the changes to complete before the errors stopped.

OCX · November 27, 2017, 2:51pm

Hello,

I am facing also a sanity check problem on a new kaltura debian 8 installation here are the logs when running the sanitycheck script:

./kaltura-sanity.sh
[Space on /opt/kaltura] [PASSED, RC: 0] - [.008711531]
[Space on /opt/kaltura/web] [PASSED, RC: 0] - [.008630322]
INFO: Downloading ready-made schema from: /opt/kaltura/web/content/clientlibs/KalturaClient.xml
INFO: Generating from api version: 3.3.0, generated at: 27-11-2017
INFO: Now generating: php5full using Php5ClientGenerator
INFO: Delete old files [/opt/kaltura/web/content/clientlibs/php5full, /opt/kaltura/app/clients/php5]
INFO: Generate client library [php5full]
INFO: Copy sources from [/opt/kaltura/clients-generator/sources/php5]
INFO: Copy tests from [/opt/kaltura/clients-generator/tests/ovp/php5]
INFO: Saving client library to [/opt/kaltura/web/content/clientlibs/php5full]
INFO: Trying to package
DEBUG: Changing dir to [/opt/kaltura/web/content/clientlibs/php5full]
INFO: Executing: tar -czf “…/php5full_27-11-2017.tar.gz” …/php5full
tar: Removing leading `…/’ from member names
INFO: Package created successfully: …/php5full_27-11-2017.tar.gz
DEBUG: Restoring dir to [/opt/kaltura/clients-generator]
INFO: php5full generated successfully
[Check apache2 daemon status] [PASSED, RC: 0] - [.010390466]
Socket error – Connection refused
Cannot connect to the monit daemon. Did you start it with http support?
Socket error – Connection refused
Cannot connect to the monit daemon. Did you start it with http support?
Socket error – Connection refused
Cannot connect to the monit daemon. Did you start it with http support?
Socket error – Connection refused
Cannot connect to the monit daemon. Did you start it with http support?

Any idea? Thank you in advance

OCX · November 27, 2017, 3:05pm

Solved the above error by modifying /etc/monit/monit.conf and restarting.

Now my sanity check script fails differently:

https://pastebin.com/afzgtt2X

jess · November 27, 2017, 3:20pm

Hi @OCX,

In regards to “Cannot connect to the monit daemon. Did you start it with http support?”, it’s important to note that this is not imperative for Kaltura CE to work. Enabling http support will simply allows you to communicate with Monit [used for monitoring and as a watchdog for the Kaltura daemons over HTTP]. It’s a nice feature but since this is a sensitive interface [it allows you to control the various daemons from a web I/F], I suggest you only allow it from a small subset of IPs and of course, use a strong passwd as well.

In regards to your second issue, which IS critical, take a look at /opt/kaltura/log/kaltura_api_v3.log looking for erroneous patterns, like so:

# grep -A 1 -B 1 --color "ERR:\|PHP\|trace\|CRIT\|\[error\]" /opt/kaltura/log/kaltura_api_v3.log

It might be easier to locate the relevant error by first truncating the existing log and re-running the script.
Note that you can run only the failing test manually by calling:

# /opt/kaltura/bin/upload_test.php <service_url> <partnerid> <secret> </path/to/asset>

The partner can be any valid non-system partner you have [partner_id >= 100].
You can get the partner user secret, which the script expects as the third argument, from KMC->Settings->Integration Settings or with this query:

mysql> select secret from partner where id=$PARTNER_ID;

OCX · November 27, 2017, 3:32pm

Hi Jess, the upload_test.php works fine with no errors reported

jess · November 27, 2017, 4:39pm

In that case, I suggest you truncate the api_v3.log again [do not delete the file, only truncate, if you delete it, you’ll need to reload Apache before continuing] and re-run kaltura-sanity.sh with bash -x for more debug info, like so:

# bash -x /opt/kaltura/bin/kaltura-sanity.sh

This will output more debug info to STDERR, including the params passed to upload_test.php from the kaltura-sanity.sh shell wrapper. Then check the log for errors and try to understand why it fails.