SiT-Dataset icon indicating copy to clipboard operation
SiT-Dataset copied to clipboard

Published 20 hours ago • verified publisher icon SPALaboratory
→

Metadata

SiT Dataset: Socially Interactive Pedestrian Trajectory Dataset for Social Navigation Robots

SiT Dataset: Socially Interactive Pedestrian Trajectory Dataset for Social Navigation Robots

SPA Logo

YouTube Video Data set OpenReview Github.io

Example of SiT dataset

Updates

  • [2023-09] Our paper is accepted to NeurIPS 2023 Dataset and Benchmark Track! Paper Link
  • [2023-07] SiT benchmark and devkit for each perception task release on Github.
  • [2023-07] Dockerfiles for each perception task release on Dockerhub.
  • [2023-07] SiT Mini-Dataset release on public.
  • [2023-06] SiT mini Dataset released on Google Drive.
  • [2023-06] Semantic map data of SiT Dataset released on Github.
  • [2023-05] We opened SiT Dataset Github.
  • [2023-05] We opened SiT Dataset Website.

Upcomings

  • [2023] Weights of trained models for 3D object detection and Trajectory prediction release on public.
  • [2023] SiT Full dataset with rosbag files release on public.
  • [2024] SiT End-to-End pedestrain trajectory prediction challenge starts on Eval AI.

Overview

Our Social Interactive Trajectory (SiT) dataset is a unique collection of pedestrian trajectories for designing advanced social navigation robots. It includes a range of sensor data, annotations, and offers a unique perspective from a robot navigating crowded environments, capturing dynamic human-robot interactions. It's meticulously organized for training and evaluating models across tasks like 3D detection, 3D multi-object tracking, and trajectory prediction, providing an end-to-end modular approach. It includes a comprehensive benchmark and exhibits the performance of several baseline models. This dataset is a valuable resource for future pedestrian trajectory prediction research, supporting the development of safe and agile social navigation robots.

Robot Platform & Sensor Setup

Sensor Setup Illustration

  • Ubuntu 18.04 LTS
  • ROS Melodic
  • Clearpath Husky UGV
  • Velodyne VLP-16 * 2
  • RGB Camera Basler a2A1920-51gc PRO GigE * 5
  • MTi-680G IMU & GPS * 1
  • VectorNAV VN-100 IMU * 1

Ground Truth

We provide GT boxes for 2D and 3D data as below.

  • 2D: Class name, Track ID, Camera number, Top left X coordinate, Top left Y coordinate, Width (w), and Height (h)
  • 3D: Class name, Track ID, Height (h), Length (l), Width (w), X, Y, and Z coordinates, and rotation (rot).

Benchmarks

We provide benchmarks and weights of trained models for 3D pedestrian detection, 3D Multi-Object Tracking, Pedestrian Trajectory Prediction and End-to-End Motion Forecasting.

3D Object Detection

Methods Modality mAP ↑ AP(0.25) ↑ AP(0.5) ↑ AP(1.0) ↑ AP(2.0) ↑ Trained
FCOS3D Camera 0.131 0.054 0.147 0.162 0.162 TBD
PointPillars LiDAR 0.319 0.202 0.316 0.346 0.414 TBD
CenterPoint-P LiDAR 0.382 0.233 0.388 0.424 0.482 TBD
CenterPoint-V LiDAR 0.514 0.352 0.522 0.556 0.620 TBD
Transfusion-P Fusion 0.396 0.213 0.371 0.451 0.551 TBD
Transfusion-V Fusion 0.533 0.360 0.512 0.587 0.672 TBD

3D Multi-Object Trajectory Tracking

Method sAMOTA↑ AMOTA↑ AMOTP(m)↓ MOTA↑ MOTP(m)↓ IDS↓
PointPillars + AB3DMOT 0.3679 0.0826 0.5125 0.2073 0.9702 1048
Centerpoint Detector + AB3DMOT 0.4626 0.1159 0.3757 0.3438 0.8360 554
Centerpoint Tracker 0.7244 0.2793 0.2611 0.5150 0.4274 1136

Pedestrian Trajectory Prediction

Name Map ADE5 ↓ FDE5 ↓ ADE20 ↓ FDE20 ↓ Trained
Social-LSTM X 1.336 2.554 1.319 2.519 TBD
Y-NET X 1.188 2.427 0.640 1.547 TBD
Y-NET O 1.036 2.306 0.596 1.370 TBD
NSP-SFM X 1.036 1.947 0.529 0.936 TBD
NSP-SFM O 0.808 1.549 0.443 0.807 TBD

End-to-End Pedestrian Motion Forecasting

Method mAP ↑ mAPf ↑ ADE5 ↓ FDE5 ↓ Trained
Fast and Furious 0.490 0.079 1.915 3.273 TBD
FutureDet-P 0.209 0.037 2.532 4.537 TBD
FutureDet-V 0.408 0.053 2.416 4.409 TBD

Download Dataset

  • Download SiT Mini dataset from below Google Drive link.
    Click Download link.

  • Full dataset and Rosbag files will be uploaded(TBD).

ROS Bag Raw Data

ROS bagfiles include below sensor data:

Topic Name Message Type Message Descriptison
/29_camera/pylon_camera_node/
image_raw/compressed		 sensor_msgs/CompressedImage Compressed Bayer Image by Basler a2A1920-51gv PRO GigE
/41_camera/pylon_camera_node/
image_raw/compressed		 sensor_msgs/CompressedImage Compressed Bayer Image by Basler a2A1920-51gv PRO GigE
/46_camera/pylon_camera_node/
image_raw/compressed		 sensor_msgs/CompressedImage Compressed Bayer Image by Basler a2A1920-51gv PRO GigE
/47_camera/pylon_camera_node/
image_raw/compressed		 sensor_msgs/CompressedImage Compressed Bayer Image by Basler a2A1920-51gv PRO GigE
/65_camera/pylon_camera_node/
image_raw/compressed		 sensor_msgs/CompressedImage Compressed Bayer Image by Basler a2A1920-51gv PRO GigE
/bottom/velodyne_points sensor_msgs/PointCloud2 Pointcloud by Velodyne VLP-16
/top/velodyne_points sensor_msgs/PointCloud2 Pointcloud by Velodyne VLP-16
/vn100/vectornav/IMU sensor_msgs/Imu VN-100 IMU
/xsens/filter/position_interpolated geometry_msgs/Vector3Stamped Interpolated GNSS data to the timestep of top velodyne
/xsens/filter/positionlla geometry_msgs/Vector3Stamped GNSS by MTi-680
/xsens/imu/data sensor_msgs/Imu IMU by MTi-680
/xsens/imu_interpolated sensor_msgs/Imu Interpolated IMU data to the timestep of top velodyne

License by-nc-nd_4.0 apache_2.0

The SiT dataset is published under the CC BY-NC-ND License 4.0, and all codes are published under the Apache License 2.0.

Acknowledgement

The SiT dataset is contributed by Jongwook Bae, Jungho Kim, Junyong Yun, Changwon Kang, Junho Lee, Jeongseon Choi, Chanhyeok Kim, Jungwook Choi, advised by Jun-Won Choi.

We thank the maintainers of the following projects that enable us to develop SiT Dataset: MMDetection by MMLAB

← Metadata

36
Stars
2
Forks
Watchers

Owner

verified publisher icon SPALaboratory

Metadata

SiT Dataset: Socially Interactive Pedestrian Trajectory Dataset for Social Navigation Robots

Back