Add docs for Observability of OpenDAL

[Xuanwo]

OpenDAL offers excellent observability support, allowing users to observe the behavior of OpenDAL through their preferred libraries for metrics, logging, and tracing.

We can provide a guide for users to observe the behavior of OpenDAL.

[zjregee]

I'd like to do this. This seems to be a work that needs to be done gradually and sorting out the layers provided by OpenDAL. I will advance this work step by step.

The first question is, does the document mentioned in this issue refer to adding a sub-section similar to Services called Layers in https://opendal.apache.org/docs/category/services, or does it mean providing a simple tutorial in https://github.com/apache/opendal/tree/main/examples, or both?

[Xuanwo]

The first question is, does the document mentioned in this issue refer to adding a sub-section similar to Services called Layers in https://opendal.apache.org/docs/category/services

I believe we're not prepared to maintain the Layers documentation. You might also notice that our service documentation is poorly maintained.

or does it mean providing a simple tutorial in https://github.com/apache/opendal/tree/main/examples

Yes, I think we should include some practical examples to begin with.

---

Hugo thanks for you to take those work :love_letter:

[zjregee]

Got this.

[zjregee]

Hi, @Xuanwo. OpenDAL can control the behavior more precisely through Reader, Writer, etc. This should be a commonly used interface, but I don’t seem to have seen the relevant introduction. How about adding a tutorial to the examples? I'd like to do this.

[Xuanwo]

Hi, @Xuanwo. OpenDAL can control the behavior more precisely through Reader, Writer, etc. This should be a commonly used interface, but I don’t seem to have seen the relevant introduction. How about adding a tutorial to the examples? I'd like to do this.

Feel free to do this, thanks in advance!

[Xuanwo]

Hi @zjregee! Thank you so much for your patience and willingness to contribute to OpenDAL's observability documentation. I apologize for the delayed response - it's been too long since we last discussed this.

I'd like to move forward with the examples directory approach as we discussed earlier. After reviewing our current observability layers, I think we can create comprehensive examples that would be very valuable for users.

Current Observability Layers Available

OpenDAL currently has these observability layers:

Core Observability:

• LoggingLayer - Operation logging with custom interceptors
• MetricsLayer - Basic metrics collection
• TracingLayer - Distributed tracing support

Metrics Export:

• PrometheusLayer - Prometheus metrics export (legacy)
• PrometheusClientLayer - Modern prometheus-client integration
• FastmetricsLayer - High-performance metrics collection

OpenTelemetry Integration:

• OtelMetricsLayer - OpenTelemetry metrics
• OtelTraceLayer - OpenTelemetry distributed tracing

Advanced Tracing:

• FastraceLayer - High-performance tracing with fastrace

Debugging Tools:

• AwaitTreeLayer - Async task debugging
• AsyncBacktraceLayer - Async stack traces
• DtraceLayer - Linux DTrace integration

Proposed Implementation Plan

I suggest we implement this in phases, starting with the most commonly used layers:

Phase 1 (Priority): Basic observability

• LoggingLayer example with custom interceptor
• MetricsLayer basic usage
• TracingLayer setup

Phase 2: Metrics export

• PrometheusClientLayer with HTTP server
• FastmetricsLayer usage

Phase 3: OpenTelemetry integration

• OtelMetricsLayer and OtelTraceLayer examples

Phase 4: Advanced debugging

• AwaitTreeLayer, AsyncBacktraceLayer examples

Each example should follow the existing pattern in core/examples/ with:

• Independent Cargo.toml with necessary dependencies
• README.md with clear usage instructions
• src/main.rs with practical demonstration

Would you be interested in starting with Phase 1? I can provide more detailed guidance on the specific implementation approach and help review the examples as you build them.

What are your thoughts on this plan? Happy to discuss and adjust based on your preferences!