Enrico Shippole

> Will there be bigger models than 2B? That is largely dependent on whether CarperAI and StabilityAI want to pursue larger training runs for PaLM. I can say that there...

I do not have access to any gpus. So no models will be trained.

I will have to convert it to cpp at some time in the near future.

> Thanks for quick reply. Will be waiting and also will be looking to contribute to that. :) You can map the model to the CPU as well by doing:...

I think this is an issue related to the use of the Flash Attention kernel in PyTorch. Can you try setting Flash Attention to false?

> i want to train my model，can you guide me? I will be releasing a fine-tuning script once the 3B model completes. Showing how the models can be trained on...

> Hi [@conceptofmind](https://github.com/conceptofmind), > > Slurm clusters often have some specific configuration like that. In this case, you should be able to use `job_directives_skip` kwarg. > > See https://jobqueue.dask.org/en/latest/clusters-advanced-tips-and-tricks.html#skipping-unrecognised-line-in-submission-script-with-job-directives-skip Hi...

> Hi [@conceptofmind](https://github.com/conceptofmind), > > Slurm clusters often have some specific configuration like that. In this case, you should be able to use `job_directives_skip` kwarg. > > See https://jobqueue.dask.org/en/latest/clusters-advanced-tips-and-tricks.html#skipping-unrecognised-line-in-submission-script-with-job-directives-skip I...

Would it make sense to just default these hero sections to: ```javascript ```

Would it make sense to set up GitHub sponsors for this project? That way, people can donate to you for maintenance.