composer icon indicating copy to clipboard operation
composer copied to clipboard

Add analytics logging to `MosaicMLLogger`

Open angel-ruiz7 opened this issue 1 year ago • 3 comments
trafficstars

Add log_analytics function to MosaicMLLogger

This adds a log_analytics function to the MosaicMLLogger, which records these metrics about composer runs on the Mosaic platform:

  • fsdp_config
    • sharding_strategy
    • state_dict_type
    • activation_checkpointing
    • forward_prefetch
    • backward_prefetch
    • mixed_precision
    • device_mesh
  • autoresume
  • algorithms
  • cloud_providers
    • cloud_provider_data
    • cloud_provider_checkpoints
  • save_interval
  • loggers
  • precision
  • optimizer

By logging these fields from the MosaicMLLogger into a run's metadata, we can then parse the metadata in SQL and add these metrics into UC. This will enable anyone on the genai team to create dashboards / queries for analytical purposes and view the patterns in these metrics across runs on the platform. (see the genai_runs table for more info)

Output of the Quick Start Example

Screenshot 2024-03-21 at 1 40 20 PM

angel-ruiz7 avatar Mar 12 '24 16:03 angel-ruiz7