djl

djl copied to clipboard

## Description I'm getting the following exception when I'm trying to use a basic `TabularRegression`: ``` Exception in thread "main" ai.djl.translate.TranslateException: java.lang.ArrayIndexOutOfBoundsException: Index 0 out of bounds for length 0...

starzu-sentio

pytorch and djl

1

if the return value of forwards function in pytorch is a tuple of some tensors with different dimensions,what should i do to deal with such a problem in DJL lib...

yufengsjtu

Paddle YoloV3 用DJL推断很慢

5

当输入图片尺寸为416 * 416时，检测一张图片大约需要7秒。尺寸调整为224 * 224时，一张图片大约为2秒。均远低于Yolo模型在Python中的性能，请问有没有可以提高推断性能的方法？ PS：推断结果是正确的 我的推断代码如下： public static void main(String[] args) throws IOException, ModelNotFoundException, MalformedModelException, TranslateException { List images = new ArrayList(); for(int i = 1;i < 30; i++){...

JiMa98

Torch NDArray .max(int, boolean) should return tuple.

2

Currently NDArray.max(int, boolean) only returns the max values. However it should also give the indexes in a tuple as per pytorch docs. https://pytorch.org/docs/stable/generated/torch.max.html "Returns a namedtuple (values, indices) where values...

brr53

ai.djl.engine.EngineException: Failed to load PyTorch native library

4

Caused by: java.lang.UnsatisfiedLinkError: C:\Users\80313223\.djl.ai\pytorch\1.12.1-20220818-cpu-win-x86_64\asmjit.dll: Can't find dependent libraries ai.djl.engine.EngineException: Failed to load PyTorch native library ai.djl.pytorch pytorch-jni 1.13.0-0.20.0 ai.djl.pytorch pytorch-native-cpu win-x86_64 runtime 1.13.0

Cyu1

Support for FP8 quantization with TensorRT-LLM

2

DJL does not support (or has not documented support) for FP8 quantization ([docs](https://demodocs.djl.ai/docs/serving/serving/docs/lmi/user_guides/trt_llm_user_guide.html#quantization-support)). FP8 is currently TensorRT-LLM's [recommended quantization technique](https://github.com/NVIDIA/TensorRT-LLM/blob/main/docs/source/blogs/quantization-in-TRT-LLM.md#best-practices-to-choose-the-right-quantization-methods), with the lowest performance degradation with good speedup. It would...

nathan-az

Fixes #3141

frankfliu

Tokenizer.json compability with jni rust tokenizers - data did not match any variant of untagged enum

2

## Description The python/rust upstream transformer tokenizer save_pretrained function adds a new key on the model level in the tokenizer.json configuration. `model.byte_fallback` which causes an exception when calling the native...

jobergum

[pytorch] optimize memory copy cost for pytorch NDArray

7

## Description ## Now the `public double[] toDoubleArray() `method in pytorch Ndarray needs to copy the contents of the local memory to the heap memory, and then copy it from...

ewan0x79

Regression: EngineException: default_program(22): error: extra text after expected end of number with DJL 0.28.0 + intfloat/multilingual-e5-small on machine with GPU

31

## Description Trying to perform predictions using `intfloat/multilingual-e5-small` fails on a machine with a GPU. This used to work in DJL 0.26.0 using PY_TORCH 2.0.1 but now fails on 0.28.0...

david-sitsky