Linux Debugging Cheat Sheet

state: not finished, but helpful enough to be published. Merge Requests welcome!

This repo contains some useful commands and ideas for linux troubleshooting, performance analysis, ...

• How to solve hard (technical) problems
• Network debugging
• ~~Fighting~~ Debugging IPsec on linux (racoon + setkey)
• How to debug a core dump
• Performance Analysis
• How to build and debug linux kernel modules (easier than you think!)
• How to use bpftrace
• Some PHP stuff

Good Reads

• Brendan Gregg is great. Read and watch everything you catch: http://www.brendangregg.com/
• Linux Performance Analysis in 60s (video)
  • http://www.brendangregg.com/blog/2015-12-03/linux-perf-60s-video.html
  • https://netflixtechblog.com/linux-performance-analysis-in-60-000-milliseconds-accc10403c55
  • https://www.youtube.com/watch?v=FJW8nGV4jxY&list=PLhhdIMVi0o5RNrf8E2dUijvGpqKLB9TCR
  • https://www.slideshare.net/brendangregg/velocity-2015-linux-perf-tools
  • http://www.brendangregg.com/Slides/Velocity2015_LinuxPerfTools.pdf
• Networking
  • https://blog.cloudflare.com/syn-packet-handling-in-the-wild/
  • https://blog.nelhage.com/2010/08/write-yourself-an-strace-in-70-lines-of-code/
  • https://veithen.io/2014/01/01/how-tcp-backlog-works-in-linux.html
• eBPF
  • https://blog.pixielabs.ai/ebpf-http-tracing/
• else:
  • https://veithen.io/2013/11/18/iowait-linux.html
  • https://blog.heckel.io/2015/10/18/how-to-create-debian-package-and-debian-repository/