DPFT icon indicating copy to clipboard operation
DPFT copied to clipboard

Published 20 hours ago • verified publisher icon TUMFTM
→

Metadata

Dual Perspective Fusion Transformer for Camera-Radar-based Object Detection

Linux Docker Python

Dual Perspective Fusion Transformer

Prerequisites

  • ubuntu
  • docker
  • nvidia-docker

Dataset

  1. Get the dataset from https://github.com/kaist-avelab/K-Radar

  2. Structure the dataset according to

    K-Radar Data Structure 
    .
|
+---data/
|   |
|   +---kradar/
|   |   |
|   |   +---raw/
|   |   |   |
|   |   |   +---1/
|   |   |   |   |
|   |   |   |   +---cam-front/
|   |   |   |   |
|   |   |   |   +---cam-left/
|   |   |   |   |
|   |   |   |   +---cam-rear/
|   |   |   |   |
|   |   |   |   +---cam-right/
|   |   |   |   |
|   |   |   |   +---description.txt
|   |   |   |   |
|   |   |   |   +---info_calib/
|   |   |   |   |
|   |   |   |   +---info_frames/
|   |   |   |   |
|   |   |   |   +---info_label/
|   |   |   |   |
|   |   |   |   +---info_label_v2/
|   |   |   |   |
|   |   |   |   +---info_matching/
|   |   |   |   |
|   |   |   |   +---os1-128/
|   |   |   |   |
|   |   |   |   +---os2-64/
|   |   |   |   |
|   |   |   |   +---radar_tesseract/
|   |   |   |   |
|   |   |   |   +---...
|   |   |   |
|   |   |   +---2, 3...

Install

docker build -t dprt:0.0.1 .

Run

docker run \
    --name dprt \
    -it \
    --gpus all \
    -e DISPLAY \
    -v /tmp/.X11-unix:/tmp/.X11-unix \
    -v <path to repository>:/app \
    -v <path to data>:/data \
    dprt:0.0.1 bash

Prepare

python -m dprt.prepare --src /data/kradar/raw/ --cfg /app/config/kradar.json --dst /data/kradar/processed/

Train

python -m dprt.train --src /data/kradar/processed/ --cfg /app/config/kradar.json

Evaluate

python -m dprt.evaluate --src /data/kradar/processed/ --cfg /app/config/kradar.json --checkpoint /app/log/<path to checkpoint>

Citation

If DPFT is useful or relevant to your research, please kindly recognize our contributions by citing our paper:

@article{fent2024dpft,
    title={DPFT: Dual Perspective Fusion Transformer for Camera-Radar-based Object Detection}, 
    author={Felix Fent and Andras Palffy and Holger Caesar},
    journal={arXiv preprint arXiv:2404.03015},
    year={2024}
}

FAQ

No CUDA runtime is found

  1. Install nvidia-container-runtime

    sudo apt-get install nvidia-container-runtime

  2. Edit/create the /etc/docker/daemon.json with content:

    {
    "runtimes": {
        "nvidia": {
            "path": "/usr/bin/nvidia-container-runtime",
            "runtimeArgs": []
        } 
    },
    "default-runtime": "nvidia" 
}

  3. Restart docker daemon:

    sudo systemctl restart docker

    Reference: https://stackoverflow.com/questions/59691207/docker-build-with-nvidia-runtime

    If this doesn't solve the problem, try:

    DOCKER_BUILDKIT=0 docker build -t dprt:0.0.1 .

    Reference: https://stackoverflow.com/questions/59691207/docker-build-with-nvidia-runtime

fatal error: cusolverDn.h: No such file or directory

  1. Export CUDA path:

    export PATH=/usr/local/cuda/bin:$PATH

    Reference: https://github.com/microsoft/DeepSpeed/issues/2684

← Metadata

15
Stars
1
Forks
Watchers

Owner

verified publisher icon TUMFTM

Metadata

Dual Perspective Fusion Transformer for Camera-Radar-based Object Detection

Back