Could not find any similar ops to torchvision::nms

[xuexl]

I get a model from detectron2 of mask-rcnn . And it is panicked when i load the model by tch::CModule . And the panicked infomation is :

thread 'main' panicked at 'called Result::unwrap() on an Err value: Torch("\nUnknown builtin op: torchvision::nms.\nCould not find any similar ops to torchvision::nms. This op may not exist or may not be currently supported in TorchScript.\n:\n File "/home/lin/anaconda3/envs/tailings_pond/lib/python3.9/site-packages/torchvision/ops/boxes.py", line 35\n """\n _assert_has_ops()\n return torch.ops.torchvision.nms(boxes, scores, iou_threshold)\n ~~~~~~~~~~~~~~~~~~~~~~~~~ <--- HERE\nSerialized File "code/torch/torchvision/ops/boxes.py", line 24\n _6 = torch.torchvision.extension._assert_has_ops\n _7 = _6()\n _8 = ops.torchvision.nms(boxes, scores, iou_threshold)\n ~~~~~~~~~~~~~~~~~~~ <--- HERE\n return _8\n'nms' is being compiled since it was called from '_batched_nms_coordinate_trick'\n File "/home/lin/anaconda3/envs/tailings_pond/lib/python3.9/site-packages/torchvision/ops/boxes.py", line 87\n offsets = idxs.to(boxes) * (max_coordinate + torch.tensor(1).to(boxes))\n boxes_for_nms = boxes + offsets[:, None]\n keep = nms(boxes_for_nms, scores, iou_threshold)\n ~~~ <--- HERE\n return keep\nSerialized File "code/torch/torchvision/ops/boxes.py", line 16\n _5 = torch.unsqueeze(torch.slice(offsets), 1)\n boxes_for_nms = torch.add(boxes, _5)\n keep = torch.torchvision.ops.boxes.nms(boxes_for_nms, scores, iou_threshold, )\n ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ <--- HERE\n _0 = keep\n return _0\n'_batched_nms_coordinate_trick' is being compiled since it was called from 'RPN.forward'\nSerialized File "code/torch/detectron2/modeling/proposal_generator/rpn.py", line 19\n argument_9: Tensor,\n image_size: Tensor) -> Tensor:\n _0 = torch.torchvision.ops.boxes._batched_nms_coordinate_trick\n ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ <---

---

I had loaded this model in a c++ program, and need to include "torchvison" . I know this , but how to use "torchvision" in tch ?

Anyone can help me ? thanks a lot . (wait on line)

[LaurentMazare]

I don't think there is a very good way to do this at the moment. What you can try doing is editing tch-rs/build.rs so as to link the torchvision shared library (if you already have it). That said, there is a risk it will be "optimized out", so you should run ldd on your binary just to check that it's still depending on torchvision.so, if that's not the case you may want to edit some c++ code, e.g. torch_api.cpp to add a dependency to one of the function from the shared library (we have such hacks in place to force link cuda). If you end up getting it to work, we could try improving this a bit, e.g. by having some environment variable to trigger the link of torchvision in build.rs for example.

[xuexl]

@LaurentMazare Thanks your reply. It is complex.

And I have an idea that if i have a function for "NMS" of rust, i put this function somewhere of tch, can i solve this problem ? or put somewhere of my code. And I have a question that why not import torchvision to tch, Is it complex ?

I know the torchvison not more. Especially the source code of torchvision.

[TBonnin]

The solution seems to be adding println!("cargo:rustc-link-lib=torchvision"); in torch-sys/build.rs main function and #include <torchvision/vision.h> in torch-sys/libtch/torch_api.cpp (for instance) for the torchvision operators to be registered.

It would be nice indeed to optionally link and include torchvision if one already has it

[nathan-brainkey]

This worked for me but I also had to modify the CMakeLists.txt in torch-sys/libtch to add TorchVision. Also, I found that I had to set linker flags in target_link_libraries to include the whole archive:

cmake_minimum_required(VERSION 3.5 FATAL_ERROR)
project(tch)

list(APPEND CMAKE_PREFIX_PATH "path/to/vision/build")
find_package(Torch REQUIRED)
find_package(TorchVision REQUIRED)

add_library(tch STATIC torch_api.cpp)
target_link_libraries(tch "${TORCH_LIBRARIES}")
target_link_libraries(tch TorchVision::TorchVision "-Wl,--whole-archive")

set_property(TARGET tch PROPERTY CXX_STANDARD 11)
install(TARGETS tch DESTINATION .)

[Nic-Gould]

I know this is old, but nathan-brainkey's suggestion is the upstream fix hinted at in the related pytorch github issue.

Will you close this or would you like a patch?

[vesuvisian]

@nathan-brainkey, can you expand on where "path/to/vision/build" might be? I'm running into a libtch/torch_api.cpp:16:9: fatal error: 'torchvision/vision.h' file not found error.