chuangzhidan

### Describe the bug C:\Users\Gavin>interpreter Traceback (most recent call last): File "D:\Users\Gavin\anaconda3\lib\runpy.py", line 197, in _run_module_as_main return _run_code(code, main_globals, None, File "D:\Users\Gavin\anaconda3\lib\runpy.py", line 87, in _run_code exec(code, run_globals) File "D:\Users\Gavin\anaconda3\Scripts\interpreter.exe\__main__.py",...

--context_window & --max_tokens issue

6

### Describe the bug **run into this problem after "interpreter --model openai/gpt-4-1106-preview --api_key sk-j5axpBIXNNHI3WTSXXXXXXXXXXX016B046"** ▌ Model set to openai/gpt-4-1106-preview Open Interpreter will require approval before running code. Use interpreter -y...

0.6 req /s is kinda low ,for real?

2

we have one A100 can support 2 requests ,throughoutput about 10 tokens /s，with just kv cache technique. your configuration with 4 *100 can achieve only 0.6 req /s under vllm...

模型不支持中文？还有批量跑多个图片预测时会缓存前面的结果？

5

遍历图片时总是复制前面的输出，是因为跑多个图片预测时会缓存前面的输出结果？ 而单独对每个图片进行推断，中文ocr效果又很差。是不支持中文ocr识别吗

**Describe the bug** root@ac6edc15b00f:/workspace/quantization# python test_gptq.py Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained. Loading checkpoint shards: 53%|██████████████████████████████████████████████▉ | 16/30...

(swift) root@gptai:/media/data/xp# python airllm_test.py **>>>> cache_utils installed** **found index file... found_layers:{'model.embed_tokens.': True, 'model.layers.0.': True, 'model.layers.1.': True, 'model.layers.2.': True, 'model.layers.3.': True**, 'model.layers.4.': True, 'model.layers.5.': True, 'model.layers.6.': True, 'model.layers.7.': True, 'model.layers.8.': True,...

[BUG] <title>微调vl的全部参数出现错误 terminate called after throwing an instance of 'c10::Error' what(): CUDA error: an illegal memory access was encountered Compile with `TORCH_USE_CUDA_DSA` to enable device-side assertions.

1

### 是否已有关于该错误的issue或讨论？ | Is there an existing issue / discussion for this? - [X] 我已经搜索过已有的issues和讨论 | I have searched the existing issues / discussions ### 该问题是否在FAQ中有解答？ | Is there an...

locally downloaded model files doesn't seem to be supported by the code, as it always requires online downloading

1

**def convnextv2_large(pretrained=False, kwargs) -> ConvNeXt: /root/autodl-tmp/hongan/NewFolder/LLaVA-HR/llava/model/multimodal_encoder/convnext.py:1091: UserWarning: Overwriting convnextv2_huge in registry with llava.model.multimodal_encoder.convnext.convnextv2_huge. This is because the name being registered conflicts with an existing name. Please check if this is...

demo doesn't work : Error while finding module specification for 'llava.serve.cli' (ModuleNotFoundError: No module named 'llava')

2

**python -m llava.serve.cli \ --model-path ./checkpoints/llava-hr-7b-sft-1024 \ --image-file "./assets/example.jpg" i have downloaded model from huggingface and didn't find any model card info ,sad . so i tried above command line...