llm-foundry
                                
                                 llm-foundry copied to clipboard
                                
                                    llm-foundry copied to clipboard
                            
                            
                            
                        Add big bench hard
Adding Big Bench Hard subset as a set of combined CoT tasks, formatted according to the specification in this repo.
These tasks are quite large and quite slow. I don't recommend including in all gauntlet runs, though it might be useful to include for some ad-hoc IFT runs.
We can calibrate them later to add to gauntlet.
Testing MPT7B: mpt-eval-cStFsb