[Profiling] Profile Node Resources

[Olshansk]

Objective

Set a foundation for automatic, and easy-to-approach, performance profiling on the Pocket Network v1 protocol.

Origin Document

We took an initial step at profiling in docs/development/PROFILING.md but there is a lot more work to do to make it easy to use, access and have impact on our design decisions.

This task is required to better understand the performance of our Pocket Network v1 protocol node and to uncover any potential bottlenecks that might degrade our system's efficiency and responsiveness. This information will serve as a foundation for subsequent optimizations.

Goals

• Identify performance characteristics and bottlenecks of the node during bootup
• Understand how the node performs when Servicers receive lots of relays
• Evaluate the node's performance when validating a block
• Measure critical resource usage: disk usage, memory usage, and network bandwidth

Deliverable

• [ ] Report on node performance during bootup, including critical resource usage and bottlenecks
• [ ] Report on node performance during block validation, including resource usage and bottlenecks
• [ ] Create tickets required to do the work and reduce bottlenecks
• [ ] Optional: Report on node performance under high relay volume, including resource usage and bottlenecks

Non-goals / Non-deliverables

• Direct optimization or refactoring of the node codebase
• Profiling of any components or features not specifically mentioned

General issue deliverables

• [ ] Update the appropriate CHANGELOG(s)
• [ ] Update any relevant local/global README(s)
• [ ] Update relevant source code tree explanations
• [ ] Add or update any relevant or supporting mermaid diagrams

Testing Methodology

• [ ] Task specific tests or benchmarks: make ...
• [ ] New tests or benchmarks: make ...
• [ ] All tests: make test_all
• [ ] LocalNet: verify a LocalNet is still functioning correctly by following the instructions at docs/development/README.md
• [ ] k8s LocalNet: verify a k8s LocalNet is still functioning correctly by following the instructions here

---

Creator: @Olshansk Co-Owners: @okdas