Alex Mallen

icon location Berkeley, CA icon location I work on AI safety at Redwood Research. I've worked on scalable oversight, AI evaluations, eliciting latent knowledge, interpretability, and AI control.

Results 3 issues of Alex Mallen

Sharded safetensors are not loaded nor pushed to hub in Trainer

### System Info - `transformers` version: 4.37.2 - Platform: Linux-5.19.17-coreweave-x86_64-with-glibc2.31 - Python version: 3.11.7 - Huggingface_hub version: 0.20.3 - Safetensors version: 0.4.2 - Accelerate version: 0.27.0 - Accelerate config: not...

trainer
bug

Add `--no_balance` flag to not balance datasets

Support logging of accuracy for non-constant answer choices

Huggingface evaluation doesn't allow passing of the answer choices to the evaluation metric function, so we are currently asserting that the answer choices for all examples are the same.