torchtune
torchtune copied to clipboard
Testing for Mistral models
I would have found it helpful to have tests for mistral while I was working on https://github.com/pytorch/torchtune/pull/840, and I noticed it was mentioned in https://github.com/pytorch/torchtune/issues/691, so I thought I'd work on it :)
Creating this issue to avoid overlap, and to give an opportunity to review my plan. The llama2 tests involve testing each component with reference code pulled from https://github.com/meta-llama/llama/blob/main. There's a similarly lightweight and minimal implementation of Mistral at https://github.com/mistralai/mistral-src/tree/main. I'm going to follow pretty much exactly the same process as each of the tests/torchtune/models/llama2
and create corresponding scripts for testing each Mistral component.