Drop folder not ingesting

Community Forums Backend Server Support
1

I need some help of where to look further.

I’ve created a local drop folder at /kaltura_droppof. Permission drwsrwsrwx and kaltura as group.
Content Ingestion - Drop Folder/s enabled for that publisher.
Batch is running and content can be uploaded using http and is then ingested.
kaltura-batch version 12.11.0-1 on debian
Nothing happens when I put content in that directory.

Although I have increased the log level to 5 and also deliberately created a non existing local drop folder, all I see in the logs looks like this.

2017-03-13 00:49:04 [0.000000] [1508029922] [1] [BATCH] [KBatchBase->__construct] DEBUG: ___________________________________________________________________________________
2017-03-13 00:49:04 [0.000135] [1508029922] [2] [BATCH] [KBatchBase->__construct] INFO: 2.0.2
2017-03-13 00:49:04 [0.000060] [1508029922] [3] [BATCH] [KBatchBase->__construct] DEBUG: set_time_limit({3000})
2017-03-13 00:49:04 [0.000066] [1508029922] [4] [BATCH] [KBatchBase->__construct] INFO: Batch index [0] session key [sess58c5de70ac27b]
2017-03-13 00:49:04 [0.026524] [1508029922] [5] [BATCH] [KAsyncDropFolderWatcher->run] NOTICE: Start running to watch folders
2017-03-13 00:49:04 [0.000089] [1508029922] [6] [BATCH] [KBatchBase->done] INFO: Done after [0.02872109413147] seconds
2017-03-13 00:50:05 [0.000000] [36188987] [1] [BATCH] [KBatchBase->__construct] DEBUG: ___________________________________________________________________________________
2017-03-13 00:50:05 [0.000147] [36188987] [2] [BATCH] [KBatchBase->__construct] INFO: 2.0.2
2017-03-13 00:50:05 [0.000060] [36188987] [3] [BATCH] [KBatchBase->__construct] DEBUG: set_time_limit({3000})
2017-03-13 00:50:05 [0.000079] [36188987] [4] [BATCH] [KBatchBase->__construct] INFO: Batch index [0] session key [sess58c5deadde6b7]
2017-03-13 00:50:05 [0.026838] [36188987] [5] [BATCH] [KAsyncDropFolderWatcher->run] NOTICE: Start running to watch folders
2017-03-13 00:50:05 [0.000091] [36188987] [6] [BATCH] [KBatchBase->done] INFO: Done after [0.02905797958374] seconds

What could be wrong?

2

No other experiencing this ?

3

I am also experiencing the same problem. I have tried everything in my knowledge to fix it but no success. I have searched every nook and cranny of this forum but no solution. I have reinstalled Kaltura on multiple distros but problem persists. I think it started from Release 12.11.0 in which there was a drop folder optimization. Well it was more like drop folder destruction. I have resulted to using the Bulk Upload feature instead until the drop folder is repaired. @jess please take note and help with solution.

4

Hi @arnebajse, @ukadikestanley,

I will try to reproduce it shortly and let you know as to my findings.

Thanks for reporting,

5

Hi all,

This was due to missing configuration in batch.ini.
I’ve updated it and new packages for kaltura-base and kaltrua-batch were released to the nightly repo.
Tomorrow, 12.13.0 will become the stable branch so the easiest solution is to wait for that and upgrade as usual. If you do not wish to wait, below are manual patching instructions.

in /opt/kaltura//app/configurations/batch/batch.ini, immediately after:

enabledWorkers.KAsyncDropFolderWatcher              = 1

add:

enabledWorkers.KAsyncDropFolderWatcherLocalBase     = 1

Under:

[KAsyncDropFolderWatcher : PeriodicWorker]

Add:

numberOfFoldersEachRun				    = 10
params.maxTimeForFolder         = 30000

then, right after that section, add the following two sections:

[KAsyncDropFolderWatcherLocal : KAsyncDropFolderWatcher]
sleepBetweenStopStart		= 900
params.maxTimeForFolder         = 30000
numberOfFoldersEachRun          = 50

[KAsyncDropFolderWatcherLocalBase : KAsyncDropFolderWatcherLocal]
id                              = 20009
friendlyName                    = Local Base Drop Folder Watcher
params.tags                     = localBase

If this works correctly, please apply the same changes to /opt/kaltura//app/configurations/batch/batch.ini.template so that they’ll reach /opt/kaltura//app/configurations/batch/batch.ini in the event you run the reconfiguration scripts.
If it still does not work, please provide the following info:

  • deb or RPM
  • Output for:
mysql> select * from drop_folder where id=$YOUR_DROPFOLDER_ID\G

NOTE: be sure to mask any sensitive data in the result

  • the contents of /opt/kaltura/log/batch/dropfolderwatcherlocal*-%Y-%m-%d.log
6

Sorry for such a late reply.

The upgrade didn’t do any difference. Applying the patch above did solve the problem though.