Add comprehensive telemetry to all content providers

[Copilot]

Overview

This PR implements comprehensive logging and metrics reporting across all content providers (Bluesky, Twitter, Blazot, YouTube, YouTubeChat, AzureQueue, Mastodon, and TwitchChat) to improve observability and debugging capabilities.

Changes Made

Enhanced ProviderInstrumentation Class

Added new telemetry capabilities to TagzApp.Common.Telemetry.ProviderInstrumentation:

• ConnectionStatusChangesCounter: Tracks provider status transitions with provider and status tags
• ConnectionStatusGauge: Observable gauge showing real-time provider health (0=Disabled, 1=Unhealthy, 2=Degraded, 3=Healthy)
• Thread-safe status tracking with dictionary-based state management

Provider Updates

Updated all 8 content providers to include:

Logging:

• Connection lifecycle events (startup, shutdown, configuration changes)
• New message discovery with message counts
• Error conditions with detailed context
• Consistent log prefixes for filtering (e.g., "Bluesky:", "Twitter:", "TwitchChat:")

Metrics:

• Message received counts tagged by provider and author
• Connection status change events
• Real-time health status monitoring

Example logging output:

Bluesky: Starting connection to AT Protocol
Bluesky: Retrieved 5 new messages
Twitter: Retrieved 12 new tweets
TwitchChat: Client is not connected - check credentials

OpenTelemetry Integration

• Added tagzapp-provider-metrics meter to TagzApp.ServiceDefaults configuration
• Metrics automatically collected and exposed via OpenTelemetry exporters
• Compatible with existing tracing infrastructure

Documentation Updates

Added comprehensive telemetry requirements to guide future provider implementations:

• .github/copilot-instructions.md: Added "Telemetry Requirements" section with:

  • Required logging patterns with code examples
  • Required metrics with usage samples
  • OpenTelemetry integration details
  • Logging conventions and standards

• doc/Provider-Configuration-Pattern.md: Added detailed "Telemetry Requirements" section with:

  • Required dependencies (ILogger, ProviderInstrumentation)
  • Complete logging examples for all scenarios
  • Metrics documentation with code samples
  • References to existing providers as implementation examples
  • Updated migration checklist to include telemetry implementation

Benefits

1. Real-time Monitoring: Operators can track provider health and message throughput
2. Faster Debugging: Detailed logging helps identify connection issues quickly
3. Performance Insights: Metrics enable tracking of message volumes and provider performance
4. Consistency: All providers now follow the same telemetry pattern
5. Standard Guidelines: Future providers have clear telemetry implementation requirements

Testing

• ✅ All builds succeed
• ✅ Unit tests pass (30 passed, 2 expected failures due to external API dependencies, 2 skipped)
• ✅ Code formatted per project standards
• ✅ No breaking changes to existing functionality

Example Metrics

Operators can now query metrics like:

• messages-received{provider="twitter", author="@user"} - Message counts per provider/author
• connection-status-changes{provider="bluesky", status="Healthy"} - Status transition events
• connection-status{provider="mastodon"} - Current health status

Fixes FritzAndFriends/TagzApp#522

Original prompt

---

This section details on the original issue you should resolve

<issue_title>Update existing content providers with comprehensive telemetry</issue_title> <issue_description>All content providers need to be updated so that they provide comprehensive logging and metrics reporting. I'd like to see logging for new messages discovered, connection status changes. I'd like to see metrics reported for messages received as well</issue_description>

Comments on the Issue (you are @copilot in this section)

Fixes FritzAndFriends/TagzApp#522

---

💡 You can make Copilot smarter by setting up custom instructions, customizing its development environment and configuring Model Context Protocol (MCP) servers. Learn more Copilot coding agent tips in the docs.

[csharpfritz]

@copilot Let's also enhance our copilot instructions files to indicate that new content providers should also provide telemetry using this technique so that we have standard metrics

[Copilot]

@copilot Let's also enhance our copilot instructions files to indicate that new content providers should also provide telemetry using this technique so that we have standard metrics

Done! I've updated both documentation files to include comprehensive telemetry requirements:

.github/copilot-instructions.md - Added "Telemetry Requirements" section with:

• Required logging patterns with examples
• Required metrics with code samples
• Available metrics from ProviderInstrumentation
• OpenTelemetry integration details

doc/Provider-Configuration-Pattern.md - Added detailed "Telemetry Requirements" section with:

• Required dependencies (ILogger, ProviderInstrumentation)
• Logging conventions and examples for all scenarios
• Complete metrics documentation
• Reference to existing providers as examples

All future providers will now have clear guidance on implementing the standard telemetry pattern. Commit: c37b34a (documentation updates in subsequent commit)