yolo-world VisionLanguageInput throws error when candidates count is not 2

Open geekwenjie opened this issue 3 months ago • 0 comments

When using the yolo-world example：https://github.com/deepjavalibrary/djl/blob/master/examples/src/main/java/ai/djl/examples/inference/cv/YoloWorld.java，the candidates parameter in VisionLanguageInput only works if exactly two labels are provided. If the candidates array contains only one label or more than two, the program throws an error.

`Exception in thread "main" ai.djl.translate.TranslateException: ai.djl.engine.EngineException: The following operation failed in the TorchScript interpreter. Traceback of TorchScript, serialized code (most recent call last): File "code/torch.py", line 77, in forward _36 = (_20).forward((_19).forward(act, _35, ), _29, ) _37 = (_21).forward(act, _36, txt_feats, ) _38 = (_22).forward(_33, input, _35, _37, ) ~~~~~~~~~~~~ <--- HERE return _38 File "code/torch/ultralytics/nn/modules/head.py", line 50, in forward _13 = int(_9) _14 = int(_9) _15 = [torch.view(xi, [int(_9), 66, -1]), torch.view(xi0, [_14, 66, -1]), torch.view(xi1, [_13, 66, -1])] ~~~~~~~~~~ <--- HERE _16 = torch.split_with_sizes(torch.cat(_15, 2), [64, 2], 1) x, cls, = _16

Traceback of TorchScript, original code (most recent call last): /home/ubuntu/source/venv/lib/python3.12/site-packages/ultralytics/nn/modules/head.py(326): forward /home/ubuntu/source/venv/lib/python3.12/site-packages/torch/nn/modules/module.py(1726): _slow_forward /home/ubuntu/source/venv/lib/python3.12/site-packages/torch/nn/modules/module.py(1747): _call_impl /home/ubuntu/source/venv/lib/python3.12/site-packages/torch/nn/modules/module.py(1736): _wrapped_call_impl /home/ubuntu/source/djl/examples/src/main/python/trace_yolo_worldv2.py(53): forward /home/ubuntu/source/venv/lib/python3.12/site-packages/torch/nn/modules/module.py(1726): _slow_forward /home/ubuntu/source/venv/lib/python3.12/site-packages/torch/nn/modules/module.py(1747): _call_impl /home/ubuntu/source/venv/lib/python3.12/site-packages/torch/nn/modules/module.py(1736): _wrapped_call_impl /home/ubuntu/source/venv/lib/python3.12/site-packages/torch/jit/_trace.py(1278): trace_module /home/ubuntu/source/venv/lib/python3.12/site-packages/torch/jit/_trace.py(698): _trace_impl /home/ubuntu/source/venv/lib/python3.12/site-packages/torch/jit/_trace.py(1002): trace /home/ubuntu/source/djl/examples/src/main/python/trace_yolo_worldv2.py(122): jit_trace /home/ubuntu/source/djl/examples/src/main/python/trace_yolo_worldv2.py(134): RuntimeError: shape '[1, 66, -1]' is invalid for input of size 312000

at ai.djl.inference.Predictor.batchPredict(Predictor.java:200)
at ai.djl.inference.Predictor.predict(Predictor.java:133)
at ai.djl.examples.inference.cv.YoloWorld.predict(YoloWorld.java:68)
at ai.djl.examples.inference.cv.YoloWorld.main(YoloWorld.java:43)

Caused by: ai.djl.engine.EngineException: The following operation failed in the TorchScript interpreter. Traceback of TorchScript, serialized code (most recent call last): File "code/torch.py", line 77, in forward _36 = (_20).forward((_19).forward(act, _35, ), _29, ) _37 = (_21).forward(act, _36, txt_feats, ) _38 = (_22).forward(_33, input, _35, _37, ) ~~~~~~~~~~~~ <--- HERE return _38 File "code/torch/ultralytics/nn/modules/head.py", line 50, in forward _13 = int(_9) _14 = int(_9) _15 = [torch.view(xi, [int(_9), 66, -1]), torch.view(xi0, [_14, 66, -1]), torch.view(xi1, [_13, 66, -1])] ~~~~~~~~~~ <--- HERE _16 = torch.split_with_sizes(torch.cat(_15, 2), [64, 2], 1) x, cls, = _16

at ai.djl.pytorch.jni.PyTorchLibrary.moduleRunMethod(Native Method)
at ai.djl.pytorch.jni.IValueUtils.forward(IValueUtils.java:57)
at ai.djl.pytorch.engine.PtSymbolBlock.forwardInternal(PtSymbolBlock.java:146)
at ai.djl.nn.AbstractBaseBlock.forward(AbstractBaseBlock.java:79)
at ai.djl.nn.Block.forward(Block.java:129)
at ai.djl.inference.Predictor.predictInternal(Predictor.java:150)
at ai.djl.inference.Predictor.batchPredict(Predictor.java:175)
... 3 more`

Sep 04 '25 02:09 geekwenjie