Jersey
Jersey
The inference module only works on Linux.
What's your CUDA driver version? Maybe the driver doesn't support these two cards. Or you can try a different CUDA version.
If the model is trained by huggingface, you can refer to [inference example](https://github.com/bytedance/lightseq/blob/master/examples/inference/python/README.md) for inference speed up.
You may try to downgrade the driver version or change a different CUDA version.
It's on the plan. For now, you can use Quantization Training and Inference by [building from source code](https://github.com/bytedance/lightseq/blob/master/docs/inference/build.md) and running the quant examples.
For now, lightseq inference cannot support structure modification. You can describe your network and we will evaluate if it can be supported.
Thanks for your affirmation.
We have tested the consistency of VIT, you can refer to the [infer example](https://github.com/bytedance/lightseq/blob/master/examples/inference/python/test/ls_vit.py) to check your usage is correct.
Yes, it's HuggingFace's modeling_vit. vit and bert have the same structure except the embedding layer.
Swin Transformer is not supported for now