codernew007

Results 8 comments of codernew007

the same savedModel can be loaded by tf-serving 2.2, but can not be loaded by tf-serving 2.5.2 when using s3 storage

> @singhniraj08 but our model train by tf.estimator not using keras, is that ok to export model using tf.keras.model.save_model ? And our model can be loaded correctly by tf2.5.2 when...

the same savedModel can be loaded by tf-serving 2.2, but can not be loaded by tf-serving 2.5.2 when using s3 storage

> @codernew007, > > Could you please share the model code and the complete error stack trace so we can better understand the issue? Thank you! My model can be...

support multi lora at the same time? like s-lora

bellow is my code by huggingface/peft, all i need is that trt-llm could support running multiple LoRA adapters in a single batch or concurrent requests in a similar fashion to...

Colab 无法运行

俺也一样

Baichuan13B vllm 效果很差

需要根据api_server.py实现，在request中增加top_p、repetable_penality等参数，另外需要设置chat-template为baichuan

CMake Error at pull/CMakeLists.txt

I got this error utils I rebuild with the source code and make install. But, I got another Error, which is shown below, even though I had installed grpc successfully....

base url

mark

Run a container(ubuntu 20.04) with GPU accleration, the container use OpenGL renderer: llvmpipe (LLVM 15.0.7, 256 bits) but not NVIDIA GeForce RTX 3060/PCIe/SSE2

Same!! Expect your solution