NanoCode012

Rebased! For those using this branch earlier, you will need to delete and repull due to rebase. Breaking change: We do not need the `type: sharegpt.load_multirole` anymore! How to use:...

Hey @JohanWork , would you be able to rebase or resolve merge conflicts? I will merge this PR after.

Thank you for the PR. Sorry that it took a while.

Does anyone still have this issue after trying casper's suggestion?

Hey! I think we're aware of this issue. It may be because of using tokenizer class name instead of the tokenizer https://github.com/OpenAccess-AI-Collective/axolotl/blob/5a5d47458d9aaf7ead798d15291ba3d9bef785c5/src/axolotl/utils/data.py#L137-L158 I've made a PR #1298 which fixes this

Test failing due to not normalizing config, I believe.

Hey! I sometimes get this behavior. From my experience, it will Always push the last epoch/final model. However, it sometimes push the intermediate checkpoints despite setting it to always do...

@JohanWork , cool! That would be appreciated. One thing I want to point out is that: perhaps, we can make it point to one of the configs within example folder...

@JohanWork , ah it's alright. This is a good start. @Vostredamus , could you try the colab that Johan has added?

Are you able to test with a newer GPU? I do not remember if bnb works well with V100.