Kaizhao Liang

Not sure if it's related, but I am getting this on my M1 Macbook ``` OSError: [Errno 86] Bad CPU type in executable ```

did anyone get over this? Maybe there is memory leak somewhere, even if flash attention, I am getting OOM. This seems to be abnormal.

I did some initial probing and I found that somehow there is huge memory footprint with the megatron attention. Some of those extra and absolutely unnecessary shape transforms could be...

oh sorry, I think I commented on the wrong issue. Please ignore my comments.

I have the same issue too when working on pythia, it works with fp16, but when switched to bf16, it errors out with -7 without showing any other meaningful error....

Here is the ds config that doesn't work: ```{ "bf16": { "enabled": "auto" }, "optimizer": { "type": "AdamW", "params": { "lr": "auto", "betas": "auto", "eps": "auto", "weight_decay": "auto" } },...

same issue here. I believe I input all the correct information, still getting 403.

> o I see. Thank you so much! I just removed the user_agent and cf_clearance and it worked!!!!

> Try removing both of them from the config and use a session token from a chrome browser.

@acheong08 @JanYork you guys are doing God's work, I wish you the best of luck of figuring this out! ps: Even chatgpt doesn't know how to do this `response =...