[TRACKER] Telemetry and more telemetry

[daryllimyt]

Motivation

Add more telemetry to the stack. This could be anything from Sentry to our internal logging system.

Umbrellas

Business Intelligence

Having json logging here will help with collecting and wrangling this data.

Metrics

Usage statistics

Application

Frontend

• [ ] Need to decide what events should be tracked
• [ ] PostHog
  • [x] Basic level
• [ ] Sentry

Backend

• [ ] aiodebug

• [ ] Identifiers

  • [ ] prefix all IDs with a tag
  • [ ] perf: Convert IDs into UUIDs

• [ ] Loggers

  • [x] Evaluate loguru/structlog (outcome: selected loguru) #128
  • [x] Use ContextVar for threadlocal loggers #127 (NOTE: this is now replaced by loguru's logger.contextualize)
  • [ ] Contextualized loggers with user/task information
    • [x] #133
    • [ ] Replace standard_logger with custom loguru logger #134
  • [ ] Structured ndjson logging #134
  • [ ] Do we need a logging service? Verdict: use disk logger with rotating handler + batch write to S3, then just use Athena + SQL
  • [ ] hyperDX/sentry (needs more evaluation)
  • [ ] PII redaction (may help SOC2) #61

• [ ] API Endpoints

  • [ ] Logging middleware to contextualize endpoint logs with user information https://docs.sentry.io/platforms/python/integrations/asgi/

• [ ] JSON Audit Trails (for enterprise; HIPPA, ISO270001)

• [ ] Better log platform (aws UI isn't great): e.g. Highlight.io

• [ ] Sentry

  • [ ] Python integrations: see https://docs.sentry.io/platforms/python/integrations/
    • [ ] Asyncio
    • [ ] FastAPI
    • [ ] ASGI
    • [ ] HTTPX
    • [ ] Loguru

Infrastructure

AWS

• [x] Cloudwatch
• [x] Log groups

Supabase

Clerk

• [ ] Slack webhooks
  • [ ] User sign-up

Sensitive informaiton

PII

• Sentry can help with this