zhangkaihuo
zhangkaihuo
Hi! I run the example: ``` sh examples/training/huggingface/run_ner.sh ``` but get the error: ``` RuntimeError: CUDA out of memory. Tried to allocate 50.00 MiB (GPU 0; 15.78 GiB total capacity;...
when I set per_device_train_batch_size=1, get the error: ``` File "/usr/local/python3.7.0/lib/python3.7/site-packages/lightseq/training/ops/pytorch/transformer_encoder_layer.py", line 288, in forward assert bs == encoder_padding_mask.size(0) and sl == encoder_padding_mask.size(1) AssertionError ```
When I set max_batch_tokens=1024 in [ls_hf_transformer_encoder_layer.py](https://github.com/bytedance/lightseq/blob/master/examples/training/huggingface/ls_hf_transformer_encoder_layer.py#L21), I still get the following error: ``` f"Batch token numbers {bs * sl} exceeds the limit {self.config.max_batch_tokens}." ValueError: Batch token numbers 1344 exceeds the...
Hello @jazelly, Running minicpm on Mac can use [LLMFarm-MiniCPM](https://github.com/OpenBMB/LLMFarm-MiniCPM).
Please confirm that you have downloaded the [hf model](https://huggingface.co/openbmb/MiniCPM-2B-sft-bf16/tree/main) to `./cache/MiniCPM-V/`
I added --copt=ENABLE_ODML_CONVERTER in setup.py but it didn't work.
@ayushgdev Unable to find the odml package for local compilation. How can I pull the odml package?
Firstly, compile and install the source code through `python3 setup.py install -- link opencv`, and then follow this [tutorial](https://developers.google.com/mediapipe/solutions/genai/llm_inference/android#convert-model) to convert the model. The error message `has no attribute' GenerateCpuTfLite...
> Hi @zkh2016, > > After internal discussion, we have concluded that this scenario cannot be supported. This functionality is designed and built specifically for internal use and is not...