Jason Dai comments

Results 106 comments of


                                            Jason Dai

inference problem with baichuan 13b

> I just download the baichuan2-13b model from HF and run model.chat. This is what I mean default Does `model.chat` use BigDL?

ModuleNotFoundError: No module named 'transformers_modules.Qwen-7B-Chat'

Looks like `transformers` version mismatch

Add a custom layer at the end of a trained Model

Can we add the model and additional layers to `Sequential`? Or something like ```python in = Input(...) out = model(in) ... ```

Add a custom layer at the end of a trained Model

> I could add more examples using the functional api in the doc. I usually use stackoverflow, because there are a lot of spark community and it will help for...

[Question]: How to use int8 API (nano or ggml?) in LLAMA inference?

> Hi, guys. I notice that BigDL utilizes BigDL nano and ggml to accelerate int8/int4 computations. I wonder how to invoke these APIs in LLMs like LLAMA. Specifically, I want...

Compatibility with Spark 3.2 + Scala 2.13

> @emartinezs44 Bad news, I tried to upgrade dllib to scala 2.13, but only one of dllib's dependency xgboost4j didn't support scala 2.13. [dmlc/xgboost#6596](https://github.com/dmlc/xgboost/issues/6596) Maybe we can release an experimental...

Using GGUF with BIGDL?

See https://github.com/intel-analytics/BigDL/tree/main/python/llm/example/CPU/HF-Transformers-AutoModels/Advanced-Quantizations/GGUF and https://github.com/intel-analytics/BigDL/tree/main/python/llm/example/GPU/HF-Transformers-AutoModels/Advanced-Quantizations/GGUF

Jason Dai

inference problem with baichuan 13b

ModuleNotFoundError: No module named 'transformers_modules.Qwen-7B-Chat'

Add a custom layer at the end of a trained Model

Add a custom layer at the end of a trained Model

[Question]: How to use int8 API (nano or ggml?) in LLAMA inference?

Compatibility with Spark 3.2 + Scala 2.13

Using GGUF with BIGDL?

How can we enable GPU training on chronos

[LLM] Add attention_sinks for CodeShell example to optimize multi-turn chat

A personal review of BigDL (installation problems)