Lei Chen

example/tensorflow_example.py had an exception (about dir making) when nb_workers > 1

1

I tried the `tensorflow_example.py` to test the function of using multiple GPUs. When setting up `nb_workers` more than 1, I met an exception as follows. As a result, I just...

Improvements to MGSM

9

https://github.com/EleutherAI/lm-evaluation-harness/blob/7411947112117e0339fe207fb620a70bcec22690/lm_eval/tasks/mgsm/direct/mgsm_direct_en.yaml#L3 According to my understanding, for `target`, we need to obtain a sub string of `answer` from the location 6+1. The existing jiajia-2, however, represent the char on location 7....

Tried both `OPT-1.3B` and `OPT-6.7B` models but met a lengthy warning and `accuray = 0` when using smoothquanted model. Details can be found at #62. Any clue on solving this...

Can we have an example showing how to use in-memory object, e.g., TF Dataset

1

Thanks for creating this project. It will be very useful to allow TF users to have something like PyTorch Lighting. I already have my codes that convert text files into...

On A100 card, speed-up effect does not show up.

2

First, thanks very much for creating this cool technology. On one A100 GPU w/ 80GB VRAM, I tried benchmarking `sq-vicuna-7b-v1.3-w3-s0 ` and its base. It is a bit strange that...