kwrobel.eth

Support OpenAI's Batch API

1

How complex would be adding OpenAI's Batch API? For now, GPT-4 benchmark of Polish tasks would cost me $1500 :(

Does it support async?

2

Rocksdb allows for `async db.get`. Can it be used with rocksdict or this should be a feature request?

Be able to generate random samples in `gr.Examples` or `examples` in `gr.ChatInterface`

2

- [x] I have searched to see if a similar issue already exists. **Is your feature request related to a problem? Please describe.** I am using ChatInterface (https://huggingface.co/spaces/speakleash/Bielik-7B-Instruct-v0.1-test/blob/main/app.py) and want...

Something like this: ```python import shelve class ShelveBenchmark(BaseBenchmark): def __init__(self, db_tpl): super().__init__(db_tpl, "shelve") def open(self): return shelve.open(self.path) ```

E.g. in https://github.com/ayulockin/debugNNwithWandB/blob/master/MNIST_pytorch_wandb_Vanishing_Gradient.ipynb Maybe you could add some screenshots.

Test nvidia-docker

Error when chat template is not a string

1

E.g. Mixtral has more than one template https://huggingface.co/mistralai/Mixtral-8x22B-Instruct-v0.1/blob/main/tokenizer_config.json ``` Traceback (most recent call last): File "/venv/bin/lm_eval", line 8, in sys.exit(cli_evaluate()) File "/lm-evaluation-harness/lm_eval/__main__.py", line 369, in cli_evaluate results = evaluator.simple_evaluate( File...

Long time testing Qwen2-72B

2

When testing Qwen/Qwen2-72B `parallelize=True,max_length=4096` on `generate_until` task I am getting warnings: ``` Both `max_new_tokens` (=2048) and `max_length`(=882) seem to have been set. `max_new_tokens` will take precedence. Please refer to the...

While working ENSRainbow (an app in ENSNode project) we have found an issue with the rainbow tables: https://github.com/namehash/ensnode/issues/140 There appears to be an invalid ENS record in the rainbow tables...

Where the list of names comes from?

1

How the list of names was prepared? What are ens-files/preimages?