Kaltura-sanity.sh reports Space on / failed due to code in kaltura-functions.rc check_space()

1

After installing CentOS 7.3 and then Kaltura 12.16 (RPM kaltura-base-12.16.0-16.noarch) and running kaltura-config-all.sh, kaltura-sanity.sh was run but it failed on the first test that checks space in the root file system. After looking at code in function check_space() in shell script kaltura-functions.rc, I noticed the following line of code:

SPACE=df -k $1 |sed '1,1d' |awk -F " " '{print $3}';if [ $SPACE -lt 700000 ];then

This code checks the amount of space used in a file system. Perhaps it should check the amount of free space. I suggest the $3 be changed to $4 to check free space rather than used space.

Thanks.

2

Hi @stanrate,

Unfortunately, the output for df is a bit inconsistent… for example, on one of my CentOS machines:

# df -k 
Filesystem           1K-blocks     Used Available Use% Mounted on
/dev/mapper/VolGroup-lv_root
                      18102140  8578424   8604164  50% /
tmpfs                   961388        0    961388   0% /dev/shm
/dev/sda1               495844   125265    344979  27% /boot
ce-nfs.dev.kaltura.com:/opt/kaltura/web
                      18102272 16480000    702720  96% /opt/kaltura/web

And so:

[root@ce-front0 ~]#  df -k /|sed '1,1d'  |awk -F " " '{print $4}'
Will output:
Available

50%
df -k /opt/kaltura/web/|sed '1,1d'  |awk -F " " '{print $4}'
Will output:
Available

96%

Whereas the original code will output the correct results:

[root@ce-front0 ~]# df -k /|sed '1,1d'  |awk -F " " '{print $3}'

8589864
[root@ce-front0 ~]# df -k /opt/kaltura/web/|sed '1,1d'  |awk -F " " '{print $3}'

702720

The solution is to use the --direct flag, from the man page:

       --direct
              show statistics for a file instead of mount point

I tested this on several different machines and got the correct output with:

# df -k --direct /opt/kaltura/web/ |sed '1,1d' |awk -F " " '{print $4}'
# df -k --direct / |sed '1,1d' |awk -F " " '{print $4}'

Can you confirm it works correctly on your box[es] as well? if so, I’ll update the code.

Thanks for reporting,

3

Thanks Jess,

I tried the two commands on a CentOS7 and a RHEL6 host and they worked fine.

Can you please remove the space check for the root filesystem or else explain the purpose of the space check. The space check for filesystems checks for 700000KB free space. Our root file system has less than that. We have a /usr, /opt and a /var filesystem. I can’t imagine Kaltura writing files into the root filesystem during installation or operation. Last week I installed 12.16 on a CentOS7 host and after it had been running a few days, our 512MB root filesystem showed 12% utilization and 460152KB available which caused the sanity check to fail. I had to change the code to check for free space less than 460152KB so it would continue with the other checks. Perhaps a very old version of Kaltura used to put things into the root filesystem and that’s why the sanity check script expects 700000KB free space?

Thanks.

4

Hi @stanrate,

In your particular case, with /opt on a separate partition/disk it really does not make sense to require that much space on the root partition [/]. that said, most servers do not have a separate partition for /opt. /var on a different partition/disk is quite a common practice [and has a very good reason] but /opt, not so much:)

Since a UNIX FS can be partitioned in many different ways, there’s no “one size fits all” formula for this kind of test, especially since on an all in one server [as opposed to a cluster deployment], the contents of /opt/kaltura/web may not reside on remote storage and mounted over NFS.

What I can do is only emit a warning in the event check_space / fails, rather than exit with RC 11 as I do now.
I’ll make the change for the next release [12.18.0].

5

Since kaltura is installed in /opt/kaltura, I suggest the space check be done for /opt/kaltura rather than for /. Hopefully, this would be a better check for all kaltura installations at all sites.

Thanks.

6

Yes, that’s an option… even though, of course, a system where the root partition is full could never properly function, it is true that, depending on how partitioning was done, it may not require 700000KB of free space to operate.
I’ll change it to check /opt/kaltura/web and /opt/kaltura [may not be the same since, again, /opt/kaltura/web can, and in the event of a cluster, should, be mounted from a remote storage over NFS].

Thanks,