markfu909

Hi, could you please help with resolve below issue i have already install the serathi-serve, but still meet this error # python vidur/profiling/attention/main.py \ --models codellama/CodeLlama-34b-Instruct-hf \ --num_gpus 4 Traceback...

python /app/software1/vidur/vidur/profiling/collectives/main.py \ --num_workers_per_node_combinations 2 \ --collective send_recv 2025-02-14 07:59:21,469 INFO worker.py:1821 -- Started a local Ray instance. 0%| | 0/994 [00:00

hello; how to fix this error; Error importing optional module IPython.core.display

1

$python -m vidur.main \ > --replica_config_device a100 \ > --replica_config_model_name meta-llama/Meta-Llama-3-8B \ > --cluster_config_num_replicas 1 \ > --replica_config_tensor_parallel_size 1 \ > --replica_config_num_pipeline_stages 1 \ > --request_generator_config_type synthetic \ > --synthetic_request_generator_config_num_requests...

241218 2021

1

FILL IN THE PR DESCRIPTION HERE FIX #xxxx (*link existing issues this PR will resolve*) **BEFORE SUBMITTING, PLEASE READ THE CHECKLIST BELOW AND FILL IN THE DESCRIPTION ABOVE** --- PR...