purl.obolibrary.org icon indicating copy to clipboard operation
purl.obolibrary.org copied to clipboard

Published 20 hours ago • verified publisher icon OBOFoundry

Document proper log analysis

Open kltm opened this issue 3 years ago • 1 comments

Now that there are rotating apache logs in an S3 bucket, how does the average operator get at them and use them?

  • [ ] Have proper AWS permissions, including API keys
  • [ ] Get AWS client tools
  • [ ] Get logs in obo-service-logs; currently obo/purl/355fea0db941/2022/03/ would be most useful for what we want
    For example: aws s3 sync s3://DESIRED_LOGS . --include "*" --exclude "*error*"
  • [ ] Decompress pigz -d purl.obolibrary.org.access.log.202203*.gz && cat purl.obolibrary.org.access.log.202203* > all-log.txt
  • [ ] Use your favorite apache2 log tool; for example goaccess all-log.txt --log-format=COMBINED

kltm avatar Apr 20 '22 15:04 kltm

Tagging @cmungall

kltm avatar Apr 20 '22 16:04 kltm