torchtune icon indicating copy to clipboard operation
torchtune copied to clipboard

Testing for Mistral models

Open SalmanMohammadi opened this issue 10 months ago • 0 comments

I would have found it helpful to have tests for mistral while I was working on https://github.com/pytorch/torchtune/pull/840, and I noticed it was mentioned in https://github.com/pytorch/torchtune/issues/691, so I thought I'd work on it :)

Creating this issue to avoid overlap, and to give an opportunity to review my plan. The llama2 tests involve testing each component with reference code pulled from https://github.com/meta-llama/llama/blob/main. There's a similarly lightweight and minimal implementation of Mistral at https://github.com/mistralai/mistral-src/tree/main. I'm going to follow pretty much exactly the same process as each of the tests/torchtune/models/llama2 and create corresponding scripts for testing each Mistral component.

SalmanMohammadi avatar Apr 23 '24 20:04 SalmanMohammadi