Daniel Han

:)

Very interesting!

Interesting - will take a look at them!

Apologies on the delay - just relocated to SF so the delay! Sorry. It's definitely on our roadmap with our all model support feature - unsure exactly when though sorry

@Robinysh Thanks for the report - ill take a look - sorry on the delayed response!

@Robinysh @shizheng-rlfresh OOHHH I actually never tried PPO, but because it's generating on the fly as well, hence the inplace issue

Oh if any of you are willing to do a PR to fix the issue, that'll be awesome :) Thanks again!

Ye if possible - another way is to directly inject it via Unsloth

@jlin816 Oh thanks for that!! Hmm I might leave the folder as is then! I'll add a check to not randomnly delete the folder :)

@mxjyst Interesting on the loss not matching - would you be able to provide a reproducible example via Colab?