cgbur
read checkpoint in as float data
Sorry I had missed that this was opened, thank you for the fix. Did you run any benchmarks with this change? I’ll have to do some checks but I think the reason for page alignment was to have perfect divisions of the memory so that all the simd and usage of weights wouldn’t cross/straddle the page boundaries. I’d bet in practice it makes minimal to no difference but I’d want to check before merging. That may have been only a thing when mmaping but it’s been awhile since I’ve looked into the details.
Of course, thanks for your excellent work. I didn't do anything more extensive than running both versions several times with the same input. The token generation speed was basically identical