torchtune
torchtune copied to clipboard
Fix eval recipe for consecutive generation and non-generation tasks
Currently, if we specify multiple tasks for the eval recipe, and one of the tasks is a generation task which uses KV-cacheing, then the cache is still enabled for non-generation tasks. The recipe will then error out here.
We should try having some check when using cacheing in addition to caches_are_enabled
, which also checks that we're in e.g. inference mode.
EDIT: This is generally an issue for our models and will also become apparent for LoRA PPO.