Daekyoung Jung

ModuleNotFoundError: No module named 'tinyllava.model.builder'

1

Hi, guys When I tried to use your model for inference, I faced ModuleNotFoundError. I did install TinyLLaVa as `README.md` tells, with the following command at the root directory of...

[Question] It takes too much time for the first token to be returned after a reqeust issued.

5

## ❓ General Questions Hi guys, I got a finetuned Llava model run on the device with MLC-LLM. MLCChat generated text as expected, but what perplexed me was that it...

__Changes:__ - Added CUDA kernel implementation for addition operation (`addition_cuda.cu`, `addition_cuda.h`) - Created CUDA interface layer for BLAS operations (`cuda_interface.cpp`, `cuda_interface.h`) - Updated meson build configuration to include new CUDA...

- Introduce `CudaContextManager` and `CudaBufferManager` to encapsulate CUDA context and buffer operations. - Integrate `CudaContext` retrieval in `MemoryPool` for robust device memory allocation and dealloc, addressing potential SEH exceptions. -...