DIRAC icon indicating copy to clipboard operation
DIRAC copied to clipboard

Integration tests issue with logging

Open aldbr opened this issue 1 year ago • 0 comments

Reported in https://github.com/DIRACGrid/DIRAC/pull/7924

As I said, I am not sure to fully understand, but it looks like it is due to the weakref. When we are in __deleteTQIfEmpty, it's like the streams have already been cleaned up, and the logger, even if it's still alive, cannot emit the messages anymore.

Quick and easy but a bit ugly fixes would consist in:

  • removing the log statements from __deleteTQIfEmpty, not a good idea
  • wrap the log statements in try/except blocks

Should we take the time to investigate the issue? I am not so sure as it doesn't seem to happen in production (probably just in the tests).

--- Logging error ---
Traceback (most recent call last):
  File "/home/dirac/ServerInstallDIR/diracos/lib/python3.11/logging/__init__.py", line 1113, in emit
    stream.write(msg + self.terminator)
ValueError: I/O operation on closed file.
Call stack:
  File "/home/dirac/ServerInstallDIR/diracos/lib/python3.11/weakref.py", line 666, in _exitfunc
    f()
  File "/home/dirac/ServerInstallDIR/diracos/lib/python3.11/weakref.py", line 590, in __call__
    return info.func(*info.args, **(info.kwargs or {}))
  File "/home/dirac/ServerInstallDIR/diracos/lib/python3.11/site-packages/DIRAC/Core/Utilities/DictCache.py", line 246, in _purgeAll
    deleteFunction(cache[cKey]["value"])
  File "/home/dirac/ServerInstallDIR/diracos/lib/python3.11/site-packages/DIRAC/WorkloadManagementSystem/DB/TaskQueueDB.py", line 950, in __deleteTQIfEmpty
    result = self.deleteTaskQueueIfEmpty(tqId, tqOwner, tqOwnerGroup)
  File "/home/dirac/ServerInstallDIR/diracos/lib/python3.11/site-packages/DIRAC/WorkloadManagementSystem/DB/TaskQueueDB.py", line 976, in deleteTaskQueueIfEmpty
    retVal = self._query(sqlCmd, conn=connObj)
  File "/home/dirac/ServerInstallDIR/diracos/lib/python3.11/site-packages/DIRAC/Core/Utilities/MySQL.py", line 718, in _query
    self.log.debug(f"_query: {self._safeCmd(cmd)}")
  File "/home/dirac/ServerInstallDIR/diracos/lib/python3.11/site-packages/DIRAC/FrameworkSystem/private/standardLogging/Logging.py", line [340](https://github.com/DIRACGrid/DIRAC/actions/runs/11820090778/job/32931577624#step:12:341), in debug
    return self._createLogRecord(LogLevels.DEBUG, sMsg, sVarMsg)
  File "/home/dirac/ServerInstallDIR/diracos/lib/python3.11/site-packages/DIRAC/FrameworkSystem/private/standardLogging/Logging.py", line 407, in _createLogRecord
    self._logger.log(level, "%s", sMsg, exc_info=exc_info, extra=extra)
Message: '%s'
Arguments: ('_query: SELECT TQId FROM `tq_TaskQueues` WHERE Enabled >= 1 AND `tq_TaskQueues`.TQId = 2 AND `tq_TaskQueues`.TQId not in ( SELECT DISTINCT TQId from `tq_Jobs` )',)

aldbr avatar Dec 04 '24 14:12 aldbr