Gary Mulder

icon indicating a website link https://www.linkedin.com/in/garymulder

icon company Perficientur Ltd. icon location UK icon location I enjoy analysing and building complex systems, including s/w, h/w, and everything in between. Currently data mining time series using neural networks.

Results 154 comments of Gary Mulder

tokenization issue for code

Probably a duplicate of #40?

Loss reaches 0 when finetuning 7B model using 1xA100 80G

1. Are you talking about eval set loss or training loss? 2. Plot both as a function of epoch similar to #63 to see whether you are overfitting or underfitting...

Loss reaches 0 when finetuning 7B model using 1xA100 80G

Without a plot it is difficult to say for certain, but you are probably overfitting. Don't train for more than one epoch.

Has anyone tried the 30b model?

@myeolinmalchi do you have a torrent for the 30B or 65B weights?