exception handling in celery and formatting log.info()

[radzia2]

logs are arriving in kibana, all is working! thanks again for helping me out earlier.

Still two probably simple to cover questions on the code.

I have celery tasks running, related to users. these turn into results.

I now wanted to add results and user details also in kibana, including exceptions (to prevent time-out) when things go bad inside my taks.

I tried the: log.info('info to log') and it puts the "info to log" inside the "raw message" inside kibana.

My two questions:

• If I wanted an output that returns a "structured" by kibana interpretable output, so that the message.raw is transformed in message.request.id, message.request.name, etc. How do I do that in an easy way? Or do I manually format my log.info()-input in json-style?

• For exception handling, i noticed your "ErrorLogObject"-class. How does it work? As I am in the celery task, and requests are not serialisable, it is impossible to pass it on, does it mean I will not be able to use this here?

EDIT: I figured out how ErrorLogObject works in views.py:

• request is needed - so my question is still valid for the case of a celery task
• I used except Exception as e: and used de "e" as exception input
• I set duration as an integer being "zero" which allows me to push error notifications to kibana.

EDIT 2: Tried minimalistic json output in the log.error, something like: log.error('{{ \'raw\' : {{ \'x\':{0}, \'y\':{1} }} }}'.format( x , y ) ) However, not splitting the "message.raw" into "message.x" and "message.y"

[cipriantarta]

Exceptions will be formatted if you use log.exception(<your exception>). However, other types of messages will not be formatted, but added as you’ve noticed in the raw key.

I do plan to change that and make a standard message format for this use case. When I started this library, it was only meant for request/response use case.

You can however, and please feel free to do so, add a new type of handler that meets your requirements and submit a PR, if you have the time to do so. :)

One thing that I should change meanwhile, is to not format a dict() message into “raw", which should lead the using of log.info <http://log.info/>({‘key’: ‘value’}) to a solution for your problem.

On 15 Nov 2017, at 16:19, radzia2 [email protected] wrote:

logs are arriving in kibana, all is working! thanks again for helping me out earlier.

Still two probably simple to cover questions on the code.

I have celery tasks running, related to users. these turn into results.

I now wanted to add results and user details also in kibana, including exceptions (to prevent time-out) when things go bad inside my taks.

I tried the: log.info('info to log') and it puts the "info to log" inside the "raw message" inside kibana.

My two questions:

If I wanted an output that returns a "structured" by kibana interpretable output, so that the message.raw is transformed in message.request.id, message.request.name, etc. How do I do that in an easy way? Or do I manually format my log.info()-input in json-style?

For exception handling, i noticed your "ErrorLogObject"-class. How does it work? As I am in the celery task, and requests are not serialisable, it is impossible to pass it on, does it mean I will not be able to use this here?

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/cipriantarta/django-logging/issues/18, or mute the thread https://github.com/notifications/unsubscribe-auth/AA2skbfqF7AvQcjQngcsV5zy31g-k9d-ks5s2vLZgaJpZM4Qe_qy.

[radzia2]

let's say time is not the issue here, but the skill ;)

in about 60% of the things I do, I don't really know what I am exactly doing.

If I come across relevant info, and I can help, I definitely will.

Thanks for the quick feedback!