daphne
daphne copied to clipboard
django
Update logging to use Python's logging framework.
Hi all,
I am using daphne in production environment for a chat server with around 300 concurrent users. My problem is that the daphne access log grows pretty fast (around 100MB per day) and it brings trouble when retriving log file from the remote server for checking. Is there any way to support a feature like the TimedRotatingFileHandler in python logging?
The dahpne is running under circus and the config is as below:
[watcher:daphne 1] cmd = daphne chat.asgi.prod:channel_layer --port 8081 -b 0.0.0.0 --access-log logs/daphne1.log numprocesses = 1 copy_env = True ...
There is not yet - I'll leave this open for implementation as a feature. Patches are, of course, always welcome.
Since the module is not using handlers for managing rotation of logs, should we add command line args for:
- Size based rotation
In this case file size have to be monitored each time while writing logs to file from
access.py - Time based rotation Here we have to check the time interval difference I presume.
Views?
I'd just go with whatever we can easily get from the Python logging module and that is relatively easy to let the user configure. What exactly that ends up being is a judgement call for the person who takes this on.
Is it enough to add a check method (in access.py) inside :
def write_entry(self, host, date, request, status=None, length=None, ident=None, user=None):
if the file size exceed the size constant then we create new file and we change the self.stream at least this what i done ...
logrotate can be found on pretty much any Linux system. I don't think Apache or Nginx rotate their own logs. Seems like adding needless complexity.
Here's what can be easily done:
- register a HUP signal listener
- close the Logger file handle on that signal
- rename the logfile from 'daphne.log' to 'daphne.log.1' (not sure about logrotate not already doing this)
- on the next logging event, Logger will automatically open a new file.
Then the logs can get rotated by sending a HUP signal to the daphne PID.
I have this working in another project.
Is there a special reason why daphne is not using python's logging framework for access/web logging or at least for emitting and formatting the logging entries? This may add some flexibility to daphne's logging without the need to reinvent the wheel for things like log-rotation or more sophisticated /flexible formatting of log entries.
@TraMZzz Well there's #260 which is possible.
I'm not entirely convinced Daphne needs this itself. Process managers generally handle log rotation, and several of the comments here support that take.
Moving to use Python logging might be a good step.
I agree, no need to add a specific log rotation feature. Process managers are the right layer to solve that at. Systemd, supervisor, etc. all do it for you.
Moving to using Python logging would be good though.
I would like to add that not using a logging framework affects more than just log rotation. In my use case we are using Docker containers in GKE. Not being able to format logs makes querying information difficult.
why not use config file to config logger?
The access log in the current implementation does not use the logging framework.
Hey is this issue still open? If yes I guess I would like to take it up
@turingnixstyx There's a couple of related PRs, that I haven't had bandwidth to get to. You could have an initial look at those to see where they're at, and leave a review. We could go from there.
Start with #473, linked a couple of comments up here. And then there's another one you can see if you click on the Pull Requests tab. Thanks!