FritzAndFriends
Add telemetry and logging to YouTube Chat provider for API quota tracking
Overview
This PR adds comprehensive telemetry and logging to the YouTube Chat provider to track YouTube Data API v3 quota usage and request limits, addressing issue #XXX.
Problem
YouTube Data API v3 has strict quota limits (default: 10,000 units per day), and different API operations have different quota costs. Without tracking, it's difficult to:
- Monitor API usage against daily quota limits
- Identify high-cost operations
- Debug quota exhaustion issues
- Plan capacity and optimize API usage
Solution
Added telemetry infrastructure following the same patterns used by TwitchChat and Mastodon providers:
1. Quota Tracking
Implemented quota cost tracking for all YouTube API operations:
LiveChatMessages.list- 5 quota units (main chat polling)Search.list- 100 quota units (broadcast/channel searches)Videos.list- 1 quota unit (video details)
2. Structured Logging
Added ILogger<YouTubeChatProvider> dependency and replaced Console.WriteLine with structured logging. Each API call now logs:
_Logger.LogInformation("YouTube API call: LiveChatMessages.list - Quota cost: {QuotaCost}, Total quota used: {QuotaUsed}, Messages retrieved: {MessageCount}",
QUOTA_LIVECHAT_LIST, _QuotaUsed, contents.Items.Count);
3. OpenTelemetry Metrics
Integrated with ProviderInstrumentation to emit counter metrics with dimensions:
provider- "YOUTUBE-CHAT"api_call- Operation name (e.g., "LiveChatMessages.list")quota_used- Cumulative quota consumed
These metrics can be exported to monitoring systems like Prometheus or Application Insights.
4. Health Status Enhancement
Updated health check to include quota usage:
Status: Healthy
Message: OK -- adding (42) messages for chatid 'xyz' at 2025-01-15T10:30:00Z | API Quota Used: 125 units
5. Message Author Tracking
Added message author tracking via ProviderInstrumentation.AddMessage() for consistency with other providers.
Changes
- YouTubeChatProvider.cs: Added ILogger and ProviderInstrumentation dependencies, quota tracking fields, telemetry instrumentation across all API calls
- TagzApp.Providers.YouTubeChat.csproj: Added
Microsoft.Extensions.Logging.Abstractionspackage reference - README.md (new): Comprehensive documentation on quota management, API costs, telemetry features, and monitoring recommendations
Testing
- ✅ All builds succeed
- ✅ Unit tests pass (expected results: 18 pass, 2 fail - Mastodon external API, 2 skip - YouTube API keys)
- ✅ Code formatting verified
- ✅ No breaking changes
Benefits
- Operational Visibility: Real-time monitoring of API quota consumption
- Cost Optimization: Identify expensive operations for optimization
- Proactive Alerting: Set up alerts before hitting quota limits
- Debugging: Structured logs help troubleshoot API issues
- Consistency: Aligns with telemetry patterns across all providers
Example Log Output
[Information] YouTube API call: LiveChatMessages.list - Quota cost: 5, Total quota used: 125, Messages retrieved: 42
[Information] YouTube API call: Search.list (Upcoming) - Quota cost: 100, Total quota used: 225, Results: 3
[Information] YouTube API call: Videos.list - Quota cost: 1, Total quota used: 226, VideoId: abc123xyz
Closes #XXX
[!WARNING]
Firewall rules blocked me from connecting to one or more addresses (expand for details)
I tried to connect to the following addresses, but was blocked by firewall rules:
mas.to
- Triggering command:
/usr/share/dotnet/dotnet exec --runtimeconfig /home/REDACTED/work/TagzApp/TagzApp/src/TagzApp.UnitTest/bin/Debug/net9.0/TagzApp.UnitTest.runtimeconfig.json --depsfile /home/REDACTED/work/TagzApp/TagzApp/src/TagzApp.UnitTest/bin/Debug/net9.0/TagzApp.UnitTest.deps.json /home/REDACTED/work/TagzApp/TagzApp/src/TagzApp.UnitTest/bin/Debug/net9.0/testhost.dll --port 41921 --endpoint 127.0.0.1:041921 --role client --parentprocessid 4391 --telemetryoptedin false(dns block)If you need me to access, download, or install something from one of these locations, you can either:
- Configure Actions setup steps to set up my environment, which run before the firewall is enabled
- Add the appropriate URLs or hosts to the custom allowlist in this repository's Copilot coding agent settings (admins only)
Original prompt
This section details on the original issue you should resolve
<issue_title>Add telemetry to YouTube chat provider to report usage and limits</issue_title> <issue_description>YouTube has a request limit and usage quota. Let's update the YouTubeChat provider so that it logs the current usage and adds a Counter meter (not sure if that's the right one to use) to show the usage</issue_description>
Comments on the Issue (you are @copilot in this section)
Fixes FritzAndFriends/TagzApp#526
✨ Let Copilot coding agent set things up for you — coding agent works faster and does higher quality work when set up for your repo.
Is there an API on the YouTube NuGet package we're using that will tell us how many tokens we've used and how many are remaining? This would be more accurate than us guessing based on the number of requests
Is there an API on the YouTube NuGet package we're using that will tell us how many tokens we've used and how many are remaining? This would be more accurate than us guessing based on the number of r...
Unfortunately, the YouTube Data API v3 and the Google.Apis.YouTube.v3 NuGet package do not provide quota information in API responses or response headers. The API responses only contain the data payload (e.g., LiveChatMessageListResponse) without any quota usage metadata.
The only way to get actual quota usage is through the Google Cloud Console's API Dashboard or by making separate calls to the Google Cloud Monitoring API to query the youtube.googleapis.com/quota/used metric, which would add significant complexity and additional API overhead.
The current approach of tracking quota by counting requests and multiplying by documented costs is the standard recommended practice for YouTube API developers and is what's documented in Google's official guidance. This gives us reliable estimates that match what YouTube actually charges.