Stas Bekman

Thank you for looking into it, Masahiro. No problem doing it elsewhere. Using torch mem profiler will be very helpful to see the reduction memory spikes https://pytorch.org/blog/understanding-gpu-memory-1/ - it's very...

> Good, I can an assertion to detect that torch.autocast is enabled outside of DeepSpeed but ds_config doesn't set torch_autocast's enabled. Or it might be better to automatically enable it....

if you want to skip cache lookup, which is what this error is about, set `FLASH_ATTENTION_FORCE_BUILD=TRUE`, e.g.: ``` FLASH_ATTENTION_FORCE_BUILD=TRUE pip install . --no-build-isolation --no-clean ``` this bypasses the online lookup.

And the suggested workaround doesn't work in HF Transformers @ main ``` $ uv pip install huggingface-hub==0.36.0 Using Python 3.12.11 environment at: /home/yak/miniconda3/envs/dev Resolved 13 packages in 5ms Uninstalled 1...

I tested the problem has been resolved in `evaluate@main` - please kindly make a new release asap! Thank you!

correction: `evaluate==0.4.6` works just fine, I'm looking at which package forced `evaluate==0.4.5` which is the culprit of this issue.

Perhaps it's transformers that need to check it's at least `evaluate==0.4.6` - let me make a PR https://github.com/huggingface/transformers/pull/41920