python-swat icon indicating copy to clipboard operation
python-swat copied to clipboard

Redirect swat log to custom logger

Open gygabyte017 opened this issue 4 years ago • 3 comments

I have a custom logger (from standard logging module) in my code. I would like that all the swat outputs like:

[table.partition]
   table.name = "XXX" (string)
   casout.name    = "YYY" (string)
   casout.replace = true (boolean)
NOTE: Executing action 'table.partition'.
NOTE: Action 'table.partition' used (Total process time):
NOTE:       real time               0,078051 seconds
NOTE:       cpu time                0,268939 seconds (344,57%)
NOTE:       data movement time      0,001960 seconds
NOTE:       total nodes             7 (672 cores)
NOTE:       total memory            5,11T
NOTE:       memory                  639,94M (0,01%)
NOTE:       bytes moved             140,18K

are not printed in the console, but instead logged using my logger (which can still write to the console, or to a file, or whatever).

Is that possible?

gygabyte017 avatar Jun 18 '20 09:06 gygabyte017

I can't think of a good way to do this at the moment. You could try to replace the sys.stdout with a proxy object that writes to your logger (and the real sys.stdout as well if you want), but that's kind of a hack.

There is a hook to add post-processing steps to specific actions, but there isn't a way to add processing to all actions at once. That would at least allow you to log the action messages, but not the tracing messages.

I'd have to think about it some more to see how difficult it would be to add something like this. The biggest problem is that some messages come from the C extension and passing an alternative logger to that might be tricky.

kesmit13 avatar Jun 18 '20 14:06 kesmit13

I understand, thank you for the answer, for now I'll try patching sys.stdout.

However it would be a nice feature, the details about what actions are run and with what parameters are very very useful in logs for quick debugging. Until now I used to just redirect the stdout to a file, but now I'm required to write a true log file with specific formatting and specific rules, so in this way I'm loosing the precious information coming from cas.

gygabyte017 avatar Jun 18 '20 16:06 gygabyte017

I'll leave this issue open so that the idea doesn't get lost.

kesmit13 avatar Jun 18 '20 16:06 kesmit13