VatsaDev

Hmm, thats really interesting. thinking about it, repeated data could have distorted the data patterns, making the model look at data in ways it shouldn't have, which could have affected...

Looks like its been fixed with train run 2? was the previous val data something it might have trained on? Whats the deduplication of the data?

@eminorhan I had similar thoughts, but its also possible that in 35% of new data, slipajama might be more uniform, or it could have been code data, which also has...

Image Example:  Overall UI: 

Also, be sure to enter your Ngrok auth token for colab, or it wont work

@gabrielgrant thanks, I've incorporated all the changes, it works again!

Not really, its pure llama, so vllm should be fine with it, but future versions of this also need to move to chatML, which I believe the newer finetunes have...

This project was meant to be trained and inferenced on a GPU. It does extend to CPU via GGUF, but there is no real support for CoreML/Metal, beyond the standard...

Can you provide more details? How did you get the error, what were you doing, whats your platform/device? Can you give a stack trace. You're github issue right now gives...

GPT-2 is glorified auto complete with the ability to make sentences, If you want better sentences, fine tune it. I have personally had pretty [good success](https://github.com/VatsaDev/nanoChatGPT) with finetuning gpt-2-medium into...