Batch processes

Community Forums Backend Server Support
1

Hi all,

I am new to the Kaltura platform and I am a bit puzzled on the way batch processes work.

I would like to add a list of processes (something like audio extraction, subtitles creation, etc.) to run automatically whenever a user uploads a video. Unfortunately I couldn’t find specific documentation describing the process.

Can anyone share the related documentation or briefly explain the procedure?

Thanks a lot!

2

Hello @Fasteno,

I’m afraid there’s no elaborate documentation on creating additional batch workers but as a starting point, take a look at an existing worker. For example, consider KAsyncExtractMedia. The code is under /opt/kaltura/app/batch/batches/ExtractMedia (and its source repo is here: https://github.com/kaltura/server/tree/Orion-15.20.0/batch/batches/ExtractMedia).

So, if you wish to create a new batch worker, you will need to create a directory under batch/batches/$WORKER_NAME and, at a bare minimum, you will need to create the parallel of:

  • batch/batches/ExtractMedia/KAsyncExtractMedia.class.php
  • batch/batches/ExtractMedia/KAsyncExtractMediaExe.php

The workers that are meant to be run by the batch scheduler are configured in /opt/kaltura/app/configurations/batch/batch.ini

For KAsyncExtractMedia, we have the directive that determines how many workers should be allowed to run concurrently (per node):

enabledWorkers.KAsyncExtractMedia

and then a section with the needed directives:

[KAsyncExtractMedia : JobHandlerWorker]
id                                                  = 60
friendlyName                                        = Extract Media
type                                                = KAsyncExtractMedia
scriptPath                                          = batches/ExtractMedia/KAsyncExtractMediaExe.php
params.mediaInfoCmd=/opt/kaltura/bin/mediainfo
params.ffprobeCmd                                   = /opt/kaltura/bin/ffprobe
params.ffmpegCmd                                    = /opt/kaltura/bin/ffmpeg
params.localTempPath                                = /opt/kaltura/tmp/convert

Each worker should have its own unique id value in the INI fil.

The main batch daemon log is under /opt/kaltura/log/kaltura_batch.log and each worker has its own designated log and error log under /opt/kaltura/log/batch.

These workers all run as PHP CLI processes and communicate with the Kaltura API endpoint to perform all their CRUD operations.

The server repo is licensed under AGPLv3 and we welcome contributions so if you do end up creating useful workers, be sure to submit a pull request.

Cheers,