error while parsing .d file

[boyang9602]

Describe the bug I want to test the traffic light detection module on Apollo v7.0.0. I noticed that this module had been resumed, as shown by https://github.com/ApolloAuto/apollo/pull/13325. However, the tests under perception/camera are still disabled, as indicated by the BUILD_ file. I created a new folder containing a BUILD file and the camera_app_traffic_light_camera_perception_test.cc, but the build is failed, the output is shown below

[bo@in-dev-docker:/apollo]$ bazel test //modules/perception/camera/bo_test:camera_app_traffic_light_camera_perception_test
Starting local Bazel server and connecting to it...
(00:01:17) INFO: Invocation ID: 3d760505-b608-4453-9df7-c2f3dcf242a9
(00:01:17) INFO: Current date is 2022-04-13
(00:01:19) INFO: Analyzed target //modules/perception/camera/bo_test:camera_app_traffic_light_camera_perception_test (124 packages loaded, 2466 targets configured).
(00:01:19) INFO: Found 1 test target...
(00:01:41) ERROR: /apollo/modules/perception/inference/tensorrt/plugins/BUILD:47:13: error while parsing .d file: /apollo/.cache/bazel/540135163923dd7d5820f3ee4b306b32/execroot/apollo/bazel-out/k8-fastbuild/bin/modules/perception/inference/tensorrt/plugins/_objs/dfmb_psroi_align_plugin/dfmb_psroi_align_plugin.pic.d (No such file or directory)
gcc: warning: modules/perception/inference/tensorrt/plugins/dfmb_psroi_align_plugin.cu: linker input file unused because linking not done
Target //modules/perception/camera/bo_test:camera_app_traffic_light_camera_perception_test failed to build
Use --verbose_failures to see the command lines of failed build steps.
(00:01:42) INFO: Elapsed time: 27.860s, Critical Path: 12.76s
(00:01:42) INFO: 410 processes: 2 remote cache hit, 64 internal, 344 local.
(00:01:42) FAILED: Build did NOT complete successfully
//modules/perception/camera/bo_test:camera_app_traffic_light_camera_perception_test FAILED TO BUILD

(00:01:42) FAILED: Build did NOT complete successfully

To Reproduce Steps to reproduce the behavior:

1. Create a new folder under perception/camera/, for example perception/camera/bo_test
2. Create a BUILD file with the following contents

load("@rules_cc//cc:defs.bzl", "cc_library", "cc_test")
load("//tools:cpplint.bzl", "cpplint")

package(default_visibility = ["//visibility:public"])

cc_test(
   name = "camera_app_traffic_light_camera_perception_test",
   srcs = ["camera_app_traffic_light_camera_perception_test.cc"],
   deps = [
       "//modules/perception/camera/app:traffic_light_camera_perception",
       "//modules/perception/inference:inference_factory",
       "//modules/perception/inference/operators:perception_inference_operators",
       "@com_github_jbeder_yaml_cpp//:yaml-cpp",
       "@com_google_googletest//:gtest_main",
       "@local_config_tensorrt//:tensorrt",
   ],
)

cpplint()

3. Copy the camera_app_traffic_light_camera_perception_test.cc from camera/tests/ to the new folder
4. run bazel test //modules/perception/camera/<folder name>:camera_app_traffic_light_camera_perception_test. Then the error should occur. Different run will have different detailed output. But the error is always error while parsing .d file: /apollo/.cache/..../tensorrt/plugins/<xxx.pic.d> (No such file or directory)

NVIDIA driver

+-----------------------------------------------------------------------------+
| NVIDIA-SMI 510.47.03    Driver Version: 510.47.03    CUDA Version: 11.6     |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|                               |                      |               MIG M. |
|===============================+======================+======================|
|   0  Quadro RTX 4000     Off  | 00000000:01:00.0 Off |                  N/A |
| 30%   37C    P8     9W / 125W |   1729MiB /  8192MiB |      3%      Default |
|                               |                      |                  N/A |

BTW, why the tests under perception/camera are still disabled in apollo v7.0.0?

[boyang9602]

In addition, I could build the entire apollo by bash apollo.sh build_opt_gpu. I cannot build things under tensorrt/plugins. For example,

[bo@in-dev-docker:/apollo]$ bazel build //modules/perception/inference/tensorrt/plugins:kernels_cuda
(01:28:16) INFO: Invocation ID: 2429bb91-3de6-4297-b972-7af1834e6550
(01:28:16) INFO: Current date is 2022-04-13
(01:28:16) INFO: Analyzed target //modules/perception/inference/tensorrt/plugins:kernels_cuda (0 packages loaded, 0 targets configured).
(01:28:16) INFO: Found 1 target...
(01:28:16) ERROR: /apollo/modules/perception/inference/tensorrt/plugins/BUILD:33:13: error while parsing .d file: /apollo/.cache/bazel/540135163923dd7d5820f3ee4b306b32/execroot/apollo/bazel-out/k8-fastbuild/bin/modules/perception/inference/tensorrt/plugins/_objs/kernels_cuda/kernels.pic.d (No such file or directory)
gcc: warning: modules/perception/inference/tensorrt/plugins/kernels.cu: linker input file unused because linking not done
Target //modules/perception/inference/tensorrt/plugins:kernels_cuda failed to build
Use --verbose_failures to see the command lines of failed build steps.
(01:28:16) INFO: Elapsed time: 0.140s, Critical Path: 0.00s
(01:28:16) INFO: 5 processes: 5 internal.
(01:28:16) FAILED: Build did NOT complete successfully

Same error as above.

[Peeta586]

I have the similar problem, the error is also error while parsing .d file, @boyang9602 have you fixed it? thank you!

[boyang9602]

No, I have not solved it yet.

[WildBeast114514]

Sorry for taking so long to reply.

In you posted information, you're trying to compile a target of perception is getting this error.

perception module relies on a predefined config of bazel. When you use build_opt_gpu, it equals to

bazel build --config=gpu --config=opt

so, just add --config=gpu can solve this problem.

[boyang9602]

It works! Thank you @WildBeast114514 !

[Hgsil]

I have the similar problem. I also use bazel build --config=gpu //modules/perception/inference/tensorrt/plugins:kernels_cuda to solve it， but it does not work. The output is shown below

root@nvidianvidia:/home/nvidia/apollo/apollo8.0# bazel build --config=gpu //modules/perception/inference/tensorrt/plugins:kernels_cuda Starting local Bazel server and connecting to it... (14:56:50) INFO: Invocation ID: a5847dea-92de-45f9-9d8b-d2746ed23326 (14:56:50) INFO: Current date is 2023-04-27 (14:56:54) INFO: Analyzed target //modules/perception/inference/tensorrt/plugins:kernels_cuda (31 packages loaded, 178 targets configured). (14:56:54) INFO: Found 1 target... (14:56:54) ERROR: /home/nvidia/apollo/apollo8.0/modules/perception/inference/tensorrt/plugins/BUILD:32:12: error while parsing .d file: /home/nvidia/apollo/apollo8.0/.cache/bazel/f7798cdc6095c390995a653fb3f598ec/e,ecroot/apollo/bazel-out/aarch64-fastbuild/bin/modules/perception/inference/tensorrt/plugins/_objs/kernels_cuda/nms_cuda.pic.d (?????????) gcc: warning: modules/perception/inference/tensorrt/plugins/nms_cuda.cu: linker input file unused because linking not done Target //modules/perception/inference/tensorrt/plugins:kernels_cuda failed to build Use --verbose_failures to see the command lines of failed build steps. (14:56:54) INFO: Elapsed time: 6.816s, Critical Path: 0.23 (14:56:54) INFO: 3 processes: 3 internal. (14:56:54) FAILED: Build did NOT complete successfully

how can i solve it. @WildBeast114514 ，thank you.

[cherishTMYY]

eption module relies on a predefined config of bazel. When you use build_opt_gpu, it equals to

Hello, I have also encountered the same problem. Have you resolved it? What is the reason? Can you explain it？

[boyang9602]

so, just add --config=gpu can solve this problem.

You need to config it to use gpu by this option.