fetch log

Open thraxil opened this issue 12 years ago • 1 comments

would be good to log everything that happens on each fetch for later debugging.

fetch time
response status
response headers
how many new entries found
any errors encountered along the way

Mar 18 '13 20:03 thraxil

I actually implemented this as another table in the database in a branch and it works fine.

Then I started doing the numbers in my head and realized that with 1000 feeds, that would mean 24,000 log entries per day, and I don't want to put anything that grows that fast in the database. Maybe It could cap it as well, trimming any log entries more than a week old, etc.

But all in all, I realized that it would be better to do most of this via Graphite/Statsd or similar or something like logstash for the error logging.

Mar 31 '13 01:03 thraxil