Perling
Perling
``` INFO 09-26 11:50:12 api_server.py:650] args: Namespace(host='0.0.0.0', port=8082, allow_credentials=False, allowed_origins=['*'], allowed_methods=['*'], allowed_headers=['*'], served_model_name=None, model='Qwen/Qwen-14B-Chat-Int4', tokenizer=None, tokenizer_mode='auto', trust_remote_code=True, download_dir='/home/models/qwen/Qwen-14B-Chat-Int4', load_format='auto', dtype='auto', worker_use_ray=False, pipeline_parallel_size=1, tensor_parallel_size=1, block_size=16, seed=0, swap_space=4, gpu_memory_utilization=0.9, max_num_batched_tokens=2560, max_num_seqs=256, disable_log_stats=False,...