mrcabbage972
mrcabbage972
@andreaskoepf I think you may have run an incorrect query, try: https://github.com/LAION-AI/Open-Assistant/pulls?q=is%3Apr+author%3Amrcabbage972+
Either tokenize from scratch or convert the arrow files to megatron format Mayank has a script for converting to megatron, whill share
@ontocord Can you please review the description of this issue? This is an important one, so I'd like to make sure we're aligned on the details.
Thanks @Stillerman! Let's close this issue?
Scripts from Sampo: [startup_scripts.zip](https://github.com/ontocord/MDEL/files/11215377/startup_scripts.zip)
@kenhktsui Great, please assign the ticket to yourself! Regarding lm-evaluation-harness, can you please create a separate issue for that and add the details (e.g. on which tasks we are going...
@kenhktsui Let's keep this ticket as element-wise averaging. I created a separate one for c-BTM.
@kenhktsui The version of Concedo's script that I saw only merges two experts, we need a solution to merge N. To close the ticket, I think what is needed is...
May be able to load the models layer by layer
@NourFahmy @kenhktsui Check out [Minho's adapation](https://colab.research.google.com/drive/1998n4b5S1Gw5qoG8DWX3LxjYv1ayZNZ5?usp=sharing) of the clustering step from the cBTM repo.