HTTPD startup Error :: Admin Page Endless loop

Community Forums Backend Server Support
3

Jess:

Even with the upgrade instructions. The main issue is that the configuration scripts do not properly swap out the tokens for the correct information. So when you get to the final steps if aways fails. I my instance I have to update:

/etc/httpd/conf.d/zzzkaltura.ssl.conf

This has to bee done before accepting which SSL port Vhost should listen too because the script overights or creates the file just before that point. It does not set the properly SSL CA file token. Then I have to update the the base confg file too after the configuration fails just to get ‘httpd’ to even run because the script does not set the path tokens in the file.

/etc/httpd/conf.d/zzzkaltura.conf

This misconfiguration doesn’t have a work around that I can see like the ssl configuration file because it does not pause before running the next step and crash the configuraiton process with the following errors.

Running Batch config…

kaltura-batch-11.4.0-1.noarch
base-config completed successfully, if you ever want to re-configure your system (e.g. change DB hostname) run the following script:

rm /opt/kaltura/app/base-config.lock

/opt/kaltura/bin/kaltura-base-config.sh

Reloading httpd: not reloading due to configuration syntax error
[FAILED]

kaltura-batch-config.sh FAILED with: 16435 on line 98

Archving logs to /opt/kaltura/log/log_23_12_15_16_27.tar.gz…
ERROR: /opt/kaltura/bin/kaltura-batch-config.sh failed:( You can re-run it when the issue is fixed.

Even after fixing the paths in the config files I still get the caption errors when starting ‘httpd’.

service httpd start
Starting httpd:

Basic no output saying ‘failed’ or ‘ok’. However, when I restart the process I get the following error.

service httpd restart
Stopping httpd: [ OK ]
Starting httpd: [Wed Dec 23 16:52:20 2015] [error] (EAI 2)Name or service not known: Could not resolve host name *:@KALTURA_VIRTUAL_HOST_PORT@ – ignoring!
[ OK ]

I have no idea where this token is set to change it manually to allow the process to run properly and when I run the following command to update the database as shown in the instructions I get this folowing message.

/opt/kaltura/bin/kaltura-db-update.sh
Error occurred during DB update:

I am finding many issue with properly configuring Kaltura with out any errors. Even with a fresh install of CentOS 6.7 with no other script install except Kaltura.

4

Jess:

I have even re-ran the update process again to download the latest changes since my last update and now I am getting more errors than before. Kaltura cannot get ‘httpd’ to start, the db error still exists and this is the message I get now.

/opt/kaltura/bin/kaltura-config-all.sh
Running base config…

kaltura-base-11.4.0-11.noarch
Welcome to Kaltura Server 11.4.0 post install setup.

Vhost port to listen on [80]:

There was a guy in another thread that was trying to explain the issue in this section saying that if you answer this question with the default port [80] it breaks SSL and you have to enter port [443} here to get SSL to work.


Here is the error message you get after completing the SSL configuration for Kaltura.

========================================================================================================================
Kaltura install answer file written to /tmp/kaltura_23_12_18_08.ans - Please save it!
This answers file can be used to silently-install re-install this machine or deploy other hosts in your cluster.

Configuration of 11.4.0 finished successfully!
Running FrontEnd config…

base-config completed successfully, if you ever want to re-configure your system (e.g. change DB hostname) run the following script:

rm /opt/kaltura/app/base-config.lock

/opt/kaltura/bin/kaltura-base-config.sh

kaltura-front-11.4.0-3.noarch
Is your Apache working with SSL?[Y/n]
Y
Please input path to your SSL certificate[/etc/ssl/certs/localhost.crt]:
/etc/ssl/certs/server.crt
Please input path to your SSL key[/etc/pki/tls/private/localhost.key]:
/etc/ssl/certs/server.key
Please input path to your SSL CA file or leave empty in case you have none:
/etc/ssl/certs/server.ca
Which port will this Vhost listen on? [443]

Please select one of the following options [0]:
0. All web interfaces

  1. Kaltura Management Console [KMC], Hosted Apps, HTML5 lib and ClipApp
  2. KAC - Kaltura Admin Console
    0
    Enabling Apache config - apps.conf
    Enabling Apache config - var.conf
    Enabling Apache config - admin.conf

========================================================================================================================
Kaltura install answer file written to /tmp/kaltura_23_12_18_08.ans - Please save it!
This answers file can be used to silently-install re-install this machine or deploy other hosts in your cluster.

Stopping httpd: [FAILED]
Starting httpd: [Wed Dec 23 18:10:20 2015] [warn] default VirtualHost overlap on port 443, the first has precedence
[FAILED]

kaltura-front-config.sh FAILED with: 9853 on line 305

Archving logs to /opt/kaltura/log/log_23_12_15_18_10.tar.gz…
ERROR: /opt/kaltura/bin/kaltura-front-config.sh failed:( You can re-run it when the issue is fixed.

This seems to be an on going issue not just for me but for other users as well.

5

Hello,

I am currently unaware of other complaints, can you share a link?
Also, I’d suggest you make sure you have symlinks like:
/etc/httpd/conf.d/zzzkaltura.conf
and rerun the config. If it still fails, please provide an answer file, it is auto generated under /tmp/kaltura_*.ans each time you run the scripts. Make sure to mask the passwds in it and paste it here. I’ll try to reproduce locally.

6

Jess:

I have previously provided the link to the other thread describing the same exact issue.

You did not follow what the user was attempting to described in this thread and I simply explained on the issue with more details. I am unclear about where to create a link to the described files. Please be more specific as to where to where I am linking to and from with the symbolic links to the files mention.

TIME_ZONE="America/New_York"
KALTURA_FULL_VIRTUAL_HOST_NAME="***HIDDEN**"
KALTURA_VIRTUAL_HOST_NAME="***HIDDEN**"
DB1_HOST="127.0.0.1"
DB1_PORT="3306"
DB1_PASS="***HIDDEN**"
DB1_NAME="kaltura"
DB1_USER="kaltura"
SERVICE_URL="https://***HIDDEN**"
SPHINX_SERVER1="127.0.0.1"
SPHINX_SERVER2="127.0.0.1"
DWH_HOST="127.0.0.1"
DWH_PORT="3306"
ADMIN_CONSOLE_ADMIN_MAIL="USER@***HIDDEN**.COM"
ADMIN_CONSOLE_PASSWORD="***HIDDEN**"
CDN_HOST="***HIDDEN**"
KALTURA_VIRTUAL_HOST_PORT="443"
SUPER_USER="root"
SUPER_USER_PASSWD="***HIDDEN**"
ENVIRONMENT_NAME="HHS Video Platerform"
DWH_PASS="***HIDDEN**"
RED5_HOST="***HIDDEN**"
USER_CONSENT="0"
CONTACT_MAIL="NO"
VOD_PACKAGER_HOST="***HIDDEN**"
VOD_PACKAGER_PORT="88"
IP_RANGE="###.###.###.###-###.###.###.###"
WWW_HOST="***HIDDEN**"
CONFIG_CHOICE="0"
IS_SSL="Y"
CRT_FILE="/etc/ssl/certs/server.crt"
KEY_FILE="/etc/ssl/certs/server.key"
CHAIN_FILE="/etc/ssl/certs/server.ca"
CA_FILE="NO_CA"

I have provided the answer file as you requested. How do I do a quite re-install this machine using this file? Let me know if you are able to duplicate this issue.

7

Hello,

Thank you for your report.
As with a lot of things, this was caused due to a sort of typo:)
Please obtain the fix by updating to kaltura-postinst-1.0.30-9 which I just released to the latest stable repo.
Since we use CDN, it might take a while for your edge node to sync, if you come across an issue, replace “installrepo.kaltura.org” with “installrepo.origin.kaltura.org” in /etc/yum.repos.d/kaltura.repo.
This will cause YUM to retrieve the package from the origin server instead of going to CDN.

Please remember to change it back after the upgrade though, since ideally, you should always go to CDN.

Thanks,

8

When I following the upgrade instructions:

I still get the caption error message when running the update database functions.

/opt/kaltura/bin/kaltura-db-update.sh
Error occurred during DB update:

I can confirm that the script is now properly writing the ‘/etc/httpd/conf.d/zzzkaltura.ssl.conf’ file and does not require me to manually edit the file to get it working properly.

However, HTTPD still returns an error when it tries to restart as shown here. My answers have not changed. Then tells me to rerun the frontend script once I fix the issue.

========================================================================================================================
Kaltura install answer file written to /tmp/kaltura_31_12_13_19.ans - Please save it!
This answers file can be used to silently-install re-install this machine or deploy other hosts in your cluster.

Stopping httpd: [FAILED]
Starting httpd: [FAILED]

kaltura-front-config.sh FAILED with: 9714 on line 305

Archving logs to /opt/kaltura/log/log_31_12_15_13_20.tar.gz…
ERROR: /opt/kaltura/bin/kaltura-front-config.sh failed:( You can re-run it when the issue is fixed.

I am not sure what to fix to resolve this issue. Nor why I am still getting the error when running the database update script.

Thanks,
Richard

9

Hi Richard,

In this particular case, there is no need to run /opt/kaltura/bin/kaltura-db-update.sh so we can ignore that part.
Now our issue is with why restarting Apache fails.
Please:
# service httpd restart
from the shell and then look at the error logs under /opt/kaltura/log/kaltura_apache_errlog
and /var/log/httpd/errorlog

lets see what it says…

10

okay… I check the error logs and there are no entries in the logs for kaltura but I did find the following errors in the '/var/log/httpd/error*log

[Thu Dec 31 18:21:26 2015] [notice] SELinux policy enabled; httpd running as context unconfined_u:system_r:httpd_t:s0
[Thu Dec 31 18:21:26 2015] [notice] suEXEC mechanism enabled (wrapper: /usr/sbin/suexec)
[Thu Dec 31 18:21:26 2015] [error] (28)No space left on device: Cannot create SSLMutex
Configuration Failed

First issue is I know that my SELinux is disabled.

SELINUX=disabled

I don’t know much about the suEXEC error and I will have to take a look into why the SSLMutex is happening because this system is on a 10TB system. I will contact my ISP support as they are the ones that setup the box from the start and I always allocate more than enough space to tmp folder and other dynamic resources to avoid this type of issue.

If you have any input on the other warning let me know.

Thanks,
Richard

11

Jess:

I Google the error previously mention and found one of three solution. First, was to reboot the server. and the alternate was run one of the following commands depending on your system.

on sh system

ipcs -s | grep apache | awk ’ { print $2 } ’ | xargs ipcrm sem

Or

ipcs -s | grep apache | sudo perl -alne ‘qx(ipcrm -s $F[1])’

So after I ran the command I found HTTPD was able to start again. However, even after fixing this error I still got the following errors.

Reloading httpd: not reloading due to configuration syntax error
[FAILED]

kaltura-batch-config.sh FAILED with: 14187 on line 98

Archving logs to /opt/kaltura/log/log_31_12_15_19_22.tar.gz…
ERROR: /opt/kaltura/bin/kaltura-batch-config.sh failed:( You can re-run it when the issue is fixed.

I remember from before this was a result of the install script not properly creating one of the HTTPD configuration files for Kaltura. Specifically, the required tokens are not being updated like I mention in the SSL configuration file. Here is the error you get after attempting a restart of HTTPD.

Starting httpd: httpd: Syntax error on line 226 of /etc/httpd/conf/httpd.conf: Syntax error on line 7 of /etc/httpd/conf.d/zzzkaltura.conf: Include directory ‘/etc/httpd/@APP_DIR@/configurations/apache/conf.d’ not found

Unlike the SSL configuration file this cannot be fixed manually prior to the script attempting to restart Apache so it breaks the install. You would need to find the error in the configuration script to allow it to properly update the ‘/etc/httpd/conf.d/zzzkaltura.conf’ configuration file prior to restart.

However, after editing the file manually and replacing the tokens with valid information then restarting HTTPD (Apache), I was able to get the host running again.

Thanks,
Richard

12

Hey @jess

After getting what seem to be everything working, I getting errors that I am missing files required for the Kaltura All In One plugin to work properly. I open a new thread to address this issue.

I am not sure if this is a plugin issue or an installation issue. It would appear to be an installation issue since files are missing from the Kaltura complete installation package required for this plugin to work. Your assistance is appreciated.

Thanks,
@hiphopservers

13

Hello @jess

I updated to the latest release of Kaltura CE you released six hours ago. However, I am still getting the following error during the configuration of the script.

kaltura-batch-11.5.0-1.noarch
base-config completed successfully, if you ever want to re-configure your system (e.g. change DB hostname) run the following script:
# rm /opt/kaltura/app/base-config.lock
# /opt/kaltura/bin/kaltura-base-config.sh


Reloading httpd: not reloading due to configuration syntax error
                                                           [FAILED]


kaltura-batch-config.sh FAILED with: 1731 on line 98

Archving logs to /opt/kaltura/log/log_04_01_16_11_41.tar.gz...
ERROR: /opt/kaltura/bin/kaltura-batch-config.sh failed:( You can re-run it when the issue is fixed.

To get this install running again I still had to edit the:

after updating the tokens in this file with the correct path information and making other manual configuration changes previously described, I was able to restart HTTPD and access Kaltura KMC and Admin Console, Red5 and run secure Kaltura Monit without errors.

However, the issues described in the following thread still exist.

14

Hi @hiphopservers,

the only way I am able to get httpd to work is by deleting file zzzkaltura.conf

15

However, when I login to Admin Console I get:

An error occurred
(error code: API:-1)

16

Hello @mark

I had that happen and I had to rerun the configuration script. Can you post a copy of your answer file and delete or hide all passwords IPs and URLs? It is related to your SSL configuraiton.

BTW, post the content of your ‘/etc/yum.repos.d/kaltura.repo’ file so I can see were you are pulling the RPMs from. Sometimes you get old files or other don’t update cause the main server has not updated with the latest RPMs. Some of this was mention by @jess in this thread.

17

@hiphopservers, Going for install #4 on clean CentOS 6.7. Will keep you posted. Thank you.

18

New error on install:

[root@player ~]# yum install mysql mysql-server

http://installrepo.kaltura.org/releases/latest/RPMS/x86_64/repodata/4962aed3c4ef2555d29a0210153e9775d74da7bf297b33ba029517b833b2cf51-primary.sqlite.bz2: [Errno 14] PYCURL ERROR 22 - "The requested URL returned error: 404 Not Found"
Trying other mirror.
Error: failure: repodata/4962aed3c4ef2555d29a0210153e9775d74da7bf297b33ba029517b833b2cf51-primary.sqlite.bz2 from Kaltura: [Errno 256] No more mirrors to try.

19

My guess is that your CDN edge has not fully updated from the origin server.
If so, you can change installrepo.kaltura.org to installrepo.origin.kaltura.org in /etc/yum.repos.d/kaltura.repo and retry.
Remember to change back later though since using CDN is naturally much faster.

20

That worked. Thank you @jess.

21

Install failed with the following error:
Running Batch config…

kaltura-batch-11.5.0-1.noarch
base-config completed successfully, if you ever want to re-configure your system (e.g. change DB hostname) run the following script:

rm /opt/kaltura/app/base-config.lock

/opt/kaltura/bin/kaltura-base-config.sh

Reloading httpd: not reloading due to configuration syntax error
[FAILED]

kaltura-batch-config.sh FAILED with: 9235 on line 98

Archving logs to /opt/kaltura/log/log_04_01_16_22_58.tar.gz…
ERROR: /opt/kaltura/bin/kaltura-batch-config.sh failed:( You can re-run it when the issue is fixed.
[root@player conf.d]# rpm -q kaltura-postinst
kaltura-postinst-1.0.30-10.noarch

SSL certs have been added to /etc/httpd/conf.d/zzzkaltura.ssl.conf.

22

Hi @jess, I was able to install with nonSSL. Now that the nonSSL instance is working…is there any safe way to change to SSL? Please advise. Thank you.