Farook (EDev) Al-Sammarraie
Farook (EDev) Al-Sammarraie
@anhappdev There's a bunch of migration work I did in #1040 to get TF 2.18.0 to work, I suggest waiting until that one's merged since it'll resolve quite a few...
- [x] Datasets should provide token limits as an input
> [@farook-edev](https://github.com/farook-edev) Can you please share the dataset? So I can upload it to our CDN so that the app can download it automatically. Sorry for the delay, here are...
@freedomtan I found a way to use different dataset files per `normal`, `quick`, and `rapid` runs, but they all have to use the same dataset (AKA `mlperf::QuerySampleLibrary`). Please let me...
@anhappdev could you please upload this model to the CDN? https://drive.google.com/file/d/1cd06WLDIW7QSZVI4_9o7iAjmNl0VdHM7/view?usp=sharing Thanks!
Compare current implementation against google's example. @farook-edev
Thanks!
I've managed to compare the result evaluation done by our implementation against google's implementation. I found and fixed multiple bugs (I'll submit a PR with the details later). I also...
I've tested the tiny33 results for 1b and 8b. Here's what I noticed from the test: - discrepancy with (1127) did not reoccur, however, (1219) (in 8b) and (1129) (in...
@freedomtan That's very curious.. I redownloaded the [python code](https://github.com/google-research/google-research/tree/master/instruction_following_eval) and re-ran the strict comparison with python3.10 and python3.13 The difference I got from my original google test was this: 3.10...