Vanessa-Taing

[BUG] Storm with claude sonnet did not use up the maximum token (8192) in its output.

1

**Describe the bug** Tried to generate a 5000 words article with claude haiku and claude sonnet. Settings for the token: ``` conv_simulator_lm = ClaudeModel(model='claude-3-haiku-20240307', max_tokens=4096, **claude_kwargs) question_asker_lm = ClaudeModel(model='claude-3-haiku-20240307', max_tokens=4096,...

## Describe the bug Encounter attribute error when running narrativeqa evaluation, ## To Reproduce 1. Cloned the repo, ran the following command: ``` lighteval accelerate \ --model_config_path="/mnt/c/Users/Documents/lighteval/lighteval/examples/model_configs/base_model.yaml" \ --tasks "helm|narrativeqa|5|0"...

json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0) when running vllm

3

### System Info / 系統信息 CUDA Version: 12.2 transformers Version: 4.44.2 Python: 3.12.4 Operating system: Windows Subsystem for Linux (WSL) in VS Code ### Who can help? / 谁可以帮助到您？ _No...