Hi @jess kaltura is eating up all the resources while transcoding videos which are large in size and it is utilizing all the cpu I uploaded 3 videos of around 500 mb each while transcoding it utilizes all the cpu we are increasing the cpu then also it is taking up all the resources. We have an ec2 instance of c52x.large with 8 cpus and 16 gig of memory and when the videos are uploaded parallely by multiple users it utilizes all the cpu.
Basically the FFmpeg job uses all the cpu below I am sharing an image.
Hello @himanshu_bhatt,
For production ENVs, it is ill-advised to have only one instance, regardless of its HW resources. This is also important from a redundancy standpoint.
You should have designated batch nodes for transcoding (preferably at least two). In /opt/kaltura/app/configurations/batch/batch.ini, set enabledWorkers.KAsyncConvert according to your number of cores.
It is setted to 1 if I will enable this then it will only use the specified number of cores and the resource utilization will not be high? Because I have seen the sever resources goes up when it is transcoding the video at that time it goes upto 90%+ and after it has finished the utilisation comes back to normal this happens only when the it is transcoding.
Hello,
You should not increase the number of concurrent transcoding jobs in your case, since you say even as it is now, you are having issues.
As I said, you need to have dedicated machines for transcoding purposes.
Regardless, one transcoding job should not consume all the CPU resources.
No when there are 4-5 videos each of around 1 gig to 500 MB then it goes higher so should I increase the enabledWorkers.KAsyncConvert to the number of cpu cores.
Actually, no. That directive determines how many of these you should be able to process in parallel, so you do not wish to increase it now.
Did you actually establish that it is set to 1? You can see which entries the conversion jobs are for via admin console->Batch Control Process->Entry investigation or by looking at the arguments of the ffmpeg procs in question (since the target file name includes the entry ID).
Yes it is set to 1 Iooked at it and I looked at the entry id in the admin console under entry investigation so what I have to look there are many things associated to the entry id.