Kaltura_populate.log filling /opt

Hey all

I’ve noticed that on my Kaltura 13 CE installation that the kaltura_populate.log file is growing unchecked and eventually filling the /opt partition on our front ends.

I see that this may have been configured to rotate and some point but is not being rotated anymore. I don’t see it in the rotation configurations in /etc/logrotate.d. Would you expect it to be?

Is there a way to turn down the logging done for this script? It seems to be logging this sort of data over and over, about once a second. It doesn’t seem like very useful information, honestly.

[2017-09-12 10:23:14]/* omapi-kltfe01[746568314][sphinx_log] */ SELECT sphinx_log.ID, sphinx_log.EXECUTED_SERVER_ID, sphinx_log.OBJECT_TYPE, sphinx_log.OBJECT_ID, sphinx_log.ENTRY_ID, sphinx_log.PARTNER_ID, sphinx_log.DC, sphinx_log.SQL, sphinx_log.CREATED_AT FROM `sphinx_log` WHERE (sphinx_log.ID IS NULL  OR ((sphinx_log.ID>'810' AND sphinx_log.DC='0') AND sphinx_log.ID NOT IN ('811','812','813','814','815','816','817','818','819','820','821','822','823','824','825','826','827','828','829','830','831','832','833','834','835','836','837','838','839','840','841','842','843','844','845','846','847','848','849','850','851','852','853','854','855','856','857','858','859','860','861','862','863','864','865','866','867','868','869','870','871','872','873','874','875','876','877','878','879','880','881','882','883','884','885','886','887','888','889','890','891','892','893','894','895','896','897','898','899','900','901','902','903','904','905','906','907','908','909','910','911','912','913','914','915','916','917','918','919','920','921','922','923','924','925','926','927','928','929','930','931','932','933','934','935','936','937','938','939','940','941','942','943','944','945','946','947','948','949','950','951','952','953','954','955','956','957','958','959','960','961','962','963','964','965','966','967','968','969','970','971','972','973','974','975','976','977','978','979','980','981','982','983','984','985','986','987','988','989','990','991','992','993','994','995','996','997','998','999','1000','1001','1002','1003','1004','1005','1006','1007','1008','1009','1010','1011','1012','1013','1014','1015','1016','1017','1018','1019','1020','1021','1022','1023','1024','1025','1026','1027','1028','1029','1030','1031','1032','1033','1034','1035','1036','1037','1038','1039','1040','1041','1042','1043','1044','1045','1046','1047','1048','1049','1050','1051','1052','1053','1054','1055','1056','1057','1058','1059','1060','1061','1062','1063','1064','1065','1066','1067','1068','1069','1070','1071','1072','1073','1074','1075','1076','1077','1078','1079','1080','1081','1082','1083','1084','1085','1086','1087','1088','1089','1090','1091','1092','1093','1094','1095','1096','1097','1098','1099','1100','1101','1102','1103','1104','1105','1106','1107','1108','1109','1110','1111','1112','1113','1114','1115','1116','1117','1118','1119','1120','1121','1122','1123','1124','1125','1126','1127','1128','1129','1130','1131','1132','1133','1134','1135','1136','1137','1138','1139','1140','1141','1142','1143','1144','1145','1146','1147','1148','1149','1150','1151','1152','1153','1154','1155','1156','1157','1158','1159','1160','1161','1162','1163','1164','1165','1166','1167','1168','1169','1170','1171','1172','1173','1174','1175','1176','1177','1178','1179','1180','1181','1182','1183','1184','1185','1186','1187','1188','1189','1190','1191','1192','1193','1194','1195','1196','1197','1198','1199','1200','1201','1202','1203','1204','1205','1206','1207','1208','1209','1210','1211','1212','1213','1214','1215','1216','1217','1218','1219','1220','1221','1222','1223','1224','1225','1226','1227','1228','1229','1230','1231','1232','1233','1234','1235','1236','1237','1238','1239','1240','1241','1242','1243','1244','1245','1246','1247','1248','1249','1250','1251','1252','1253','1254','1255','1256','1257','1258','1259','1260','1261','1262','1263','1264','1265','1266','1267','1268','1269','1270','1271','1272','1273','1274','1275','1276','1277','1278','1279','1280','1281','1282','1283','1284','1285','1286','1287','1288','1289','1290','1291','1292','1293','1294','1295','1296','1297','1298','1299','1300','1301','1302','1303','1304','1305','1306','1307','1308','1309','1310'))) ORDER BY sphinx_log.ID ASC LIMIT 1000
[2017-09-12 10:23:14]Sql took - 0.004551887512207 seconds

So, two questions

  1. should kaltura_populate.log be part of the logrotate configuration and if so, how do I change the config to make it so? Preferably it would be great if I could change the configuration and not have it changed by an upgrade.
  2. Is it possible to turn down the level of logging in the populateFromLog.php script ?

Thanks!

Hi Dave,

/opt/kaltura/log/kaltura_populate.log is configured for rotation here:
/etc/logrotate.d/kaltura_populate -> /opt/kaltura/app/configurations/logrotate/kaltura_populate
Do you not have this file?

As for the amount of output, if you look at /opt/kaltura/app/plugins/search/providers/sphinx_search/scripts/populateFromLog.php, you’ll see that each iteration really doesn’t log that much but of course, there are A LOT of daily iterations so it becomes huge very quickly…

Hey Jess,

I do not have this symlink defined for some reason. I do have the file at /opt/kaltura/app/configurations/logrotate/kaltura_populate

I can define it and that should start it rotating again. I’m not sure how I don’t have it if it’s supposed to be done via set up.

Hi Dave,

The symlink is created here:

/opt/kaltura/bin/kaltura-sphinx-config.sh:
ln -sf $BASE_DIR/app/configurations/logrotate/kaltura_populate /etc/logrotate.d/

So all your Sphinx machines should have it. Not sure why it wasn’t created but certainly, go ahead and create it manually.

1 Like