llm-foundry icon indicating copy to clipboard operation
llm-foundry copied to clipboard

Add EnvironmentLogger Callback

Open josejg opened this issue 1 year ago • 2 comments

This is a new callback to simplify logging environment metadata for reproducibility purposes:

  • Git Commits for packages under workspace_dir, useful for mcli integrations
  • Package versions for core dependencies, e.g. transformers
  • Nvidia versions, including cuda, cuddn and nccl
  • Docker image and tag
  • Distributed info
  • System info
  • Gpu info

Main change is that GitPython now becomes a dependency. It is already a composer[dev] dependency. I was not sure if it was worth gating under a additional dependency

josejg avatar Jul 12 '24 00:07 josejg