Nuclear6 comments

Results 40 comments of


                                            Nuclear6

Load Flux base model from quantized?

@bghira https://huggingface.co/Comfy-Org/flux1-dev/blob/main/flux1-dev-fp8.safetensors Sorry, I still don't understand why others have already quantized the fp8 model, but we can't train lora directly. Why do we have to re-quantize it before we...

ValueError: The model's max seq len (4096) is larger than the maximum number of tokens that can be stored in KV cache (3664). Try increasing `gpu_memory_utilization` or decreasing `max_model_len` when initializing the engine.`

Is there a solution to this problem now? I still encounter this problem on gemma-7b.

ValueError: The model's max seq len (4096) is larger than the maximum number of tokens that can be stored in KV cache (3664). Try increasing `gpu_memory_utilization` or decreasing `max_model_len` when initializing the engine.`

> > Is there a solution to this problem now? I still encounter this problem on gemma-7b.现在这个问题有解决办法吗？我在 gemma-7b 上仍然遇到这个问题。 > > Maybe try a lower model length should be fine,just...

ValueError: The model's max seq len (4096) is larger than the maximum number of tokens that can be stored in KV cache (3664). Try increasing `gpu_memory_utilization` or decreasing `max_model_len` when initializing the engine.`

> > > > Is there a solution to this problem now? I still encounter this problem on gemma-7b.现在这个问题有解决办法吗？我在 gemma-7b 上仍然遇到这个问题。 > > > > > > > > >...

ValueError: The model's max seq len (4096) is larger than the maximum number of tokens that can be stored in KV cache (3664). Try increasing `gpu_memory_utilization` or decreasing `max_model_len` when initializing the engine.`

https://rahulschand.github.io/gpu_poor/

ValueError: The model's max seq len (4096) is larger than the maximum number of tokens that can be stored in KV cache (3664). Try increasing `gpu_memory_utilization` or decreasing `max_model_len` when initializing the engine.`

> gpu_memory_utilization not work, Can you post the modified files and code?

Nuclear6

Load Flux base model from quantized?

ValueError: The model's max seq len (4096) is larger than the maximum number of tokens that can be stored in KV cache (3664). Try increasing `gpu_memory_utilization` or decreasing `max_model_len` when initializing the engine.`

ValueError: The model's max seq len (4096) is larger than the maximum number of tokens that can be stored in KV cache (3664). Try increasing `gpu_memory_utilization` or decreasing `max_model_len` when initializing the engine.`

ValueError: The model's max seq len (4096) is larger than the maximum number of tokens that can be stored in KV cache (3664). Try increasing `gpu_memory_utilization` or decreasing `max_model_len` when initializing the engine.`

ValueError: The model's max seq len (4096) is larger than the maximum number of tokens that can be stored in KV cache (3664). Try increasing `gpu_memory_utilization` or decreasing `max_model_len` when initializing the engine.`

ValueError: The model's max seq len (4096) is larger than the maximum number of tokens that can be stored in KV cache (3664). Try increasing `gpu_memory_utilization` or decreasing `max_model_len` when initializing the engine.`

[Feature Request]: how to view artifacts

Error executing verb "cluster_graph" in create_base_entity_graph: Columns must be same length as key

Indexer Error

The entities extracted from Chinese manual documents are very messy