PENet_ICRA2021 icon indicating copy to clipboard operation
PENet_ICRA2021 copied to clipboard

Can I reduce kitti_Raw data for training ?

Open Laihu08 opened this issue 3 years ago • 12 comments

Hi, recently I've been through your great work, I would like to train the network but with less Kitti_Raw data, is that possible? if so please guide me on how to reduce Kitt-raw data for training. Thanks in advance!

Laihu08 avatar Jun 25 '21 08:06 Laihu08

Hi Laihu: Thanks for your interest! The files are loaded in kitti_loader.py. And the paths are written in function get_paths_and_transform(split, args) . You could explicitly maintain a path list and only load files in the list, i.e. replace the *_sync in line51 with 2011_09_26_drive_0001_sync or other directories. I hope this might help!

JUGGHM avatar Jun 25 '21 13:06 JUGGHM

Thanks, JUGGHM for the quick reply, I think you misunderstand my question as I request about kitti_raw data, not kitti_depth. Kindly guide me through this issue, as all know Kitti_raw dataset is very huge, and out of 5 dates of data, I have 3 dates. like 2011_09_26 , 2011_09_28 and 2011_09_30.

Laihu08 avatar Jun 25 '21 15:06 Laihu08

The paths of corresponding RGB images are obtained via the function get_rgb_paths(p). So I think they are actually the same.

JUGGHM avatar Jun 25 '21 17:06 JUGGHM

Thank you very much, JUGGHM. As you said path is the same.

Laihu08 avatar Jun 26 '21 03:06 Laihu08

Hello, thank you very much for your research. How to get_ rgb_ Paths (P) is modified to reduce the data set. Thank you for your patience

liushuqi123 avatar Sep 23 '21 15:09 liushuqi123

Hello, thank you very much for your research. How to get_ rgb_ Paths (P) is modified to reduce the data set. Thank you for your patience

For example, you could include fewer images by modifying the * in the following code (involving all files/directories) into elements in a pre-defined list corresponding to some subset. glob_d = os.path.join( args.data_folder, 'data_depth_velodyne/train/*_sync/proj_depth/velodyne_raw/image_0[2,3]/*.png' )

JUGGHM avatar Sep 23 '21 22:09 JUGGHM

您好,非常感谢您的回复。麻烦再打扰问您一下,如果我想训练数据时,只用2011_9_26的raw  data来训练,不是kitti depth来训练,如何进行修改,可以详细说一下吗,很抱歉打扰到您了,希望得到您的回复

---Original--- From: @.> Date: Fri, Sep 24, 2021 06:34 AM To: @.>; Cc: @.@.>; Subject: Re: [JUGGHM/PENet_ICRA2021] Can I reduce kitti_Raw data for training ? (#19)

Hello, thank you very much for your research. How to get_ rgb_ Paths (P) is modified to reduce the data set. Thank you for your patience

For example, you could include fewer images by modifying the * in the following code (involving all files/directories) into elements in a pre-defined list corresponding to some subset. glob_d = os.path.join( args.data_folder, 'data_depth_velodyne/train/_sync/proj_depth/velodyne_raw/image_0[2,3]/.png' )

— You are receiving this because you commented. Reply to this email directly, view it on GitHub, or unsubscribe. Triage notifications on the go with GitHub Mobile for iOS or Android.

liushuqi123 avatar Sep 24 '21 06:09 liushuqi123

您好,非常感谢您的回复。麻烦再打扰问您一下,如果我想训练数据时,只用2011_9_26的raw  data来训练,不是kitti depth来训练,如何进行修改,可以详细说一下吗,很抱歉打扰到您了,希望得到您的回复 ---Original--- From: @.> Date: Fri, Sep 24, 2021 06:34 AM To: @.>; Cc: @.@.>; Subject: Re: [JUGGHM/PENet_ICRA2021] Can I reduce kitti_Raw data for training ? (#19) Hello, thank you very much for your research. How to get_ rgb_ Paths (P) is modified to reduce the data set. Thank you for your patience For example, you could include fewer images by modifying the * in the following code (involving all files/directories) into elements in a pre-defined list corresponding to some subset. glob_d = os.path.join( args.data_folder, 'data_depth_velodyne/train/_sync/proj_depth/velodyne_raw/image_0[2,3]/.png' ) — You are receiving this because you commented. Reply to this email directly, view it on GitHub, or unsubscribe. Triage notifications on the go with GitHub Mobile for iOS or Android.

您好,KITTI上深度图补全中稀疏深度图都在kitti_depth中,raw里面只有彩色图,所以我不是很理解如何只用彩色图进行补全;如果要做单目深度估计的话,这部分代码需要重新组织。原始代码的逻辑是先从kitti_depth中读取深度图,再根据深度图文件名去raw中获取对应的彩色图像。

JUGGHM avatar Sep 26 '21 01:09 JUGGHM

好的,非常感谢您

---Original--- From: @.> Date: Sun, Sep 26, 2021 09:09 AM To: @.>; Cc: @.@.>; Subject: Re: [JUGGHM/PENet_ICRA2021] Can I reduce kitti_Raw data for training ? (#19)

您好,非常感谢您的回复。麻烦再打扰问您一下,如果我想训练数据时,只用2011_9_26的raw  data来训练,不是kitti depth来训练,如何进行修改,可以详细说一下吗,很抱歉打扰到您了,希望得到您的回复 … ---Original--- From: @.> Date: Fri, Sep 24, 2021 06:34 AM To: @.>; Cc: @.@.>; Subject: Re: [JUGGHM/PENet_ICRA2021] Can I reduce kitti_Raw data for training ? (#19) Hello, thank you very much for your research. How to get_ rgb_ Paths (P) is modified to reduce the data set. Thank you for your patience For example, you could include fewer images by modifying the * in the following code (involving all files/directories) into elements in a pre-defined list corresponding to some subset. glob_d = os.path.join( args.data_folder, 'data_depth_velodyne/train/_sync/proj_depth/velodyne_raw/image_0[2,3]/.png' ) — You are receiving this because you commented. Reply to this email directly, view it on GitHub, or unsubscribe. Triage notifications on the go with GitHub Mobile for iOS or Android.

您好,KITTI上深度图补全中稀疏深度图都在kitti_depth中,raw里面只有彩色图,所以我不是很理解如何只用彩色图进行补全;如果要做单目深度估计的话,这部分代码需要重新组织。原始代码的逻辑是先从kitti_depth中读取深度图,再根据深度图文件名去raw中获取对应的彩色图像。

— You are receiving this because you commented. Reply to this email directly, view it on GitHub, or unsubscribe. Triage notifications on the go with GitHub Mobile for iOS or Android.

liushuqi123 avatar Sep 26 '21 01:09 liushuqi123

The paths of corresponding RGB images are obtained via the function get_rgb_paths(p). So I think they are actually the same.

hello, I want to know how to change RGB images path if I have only 2011_09_26.

Danica-zc avatar Oct 22 '21 14:10 Danica-zc

The paths of corresponding RGB images are obtained via the function get_rgb_paths(p). So I think they are actually the same.

hello, I want to know how to change RGB images path if I have only 2011_09_26.

I think you could change as following: From 'data_depth_velodyne/train/*_sync/proj_depth/velodyne_raw/image_0[2,3]/*.png' to 'data_depth_velodyne/train/2011_09_26_*_sync/proj_depth/velodyne_raw/image_0[2,3]/*.png' for all similar formatted paths in the dataloader.

JUGGHM avatar Oct 23 '21 02:10 JUGGHM

Hello, thank you very much for your research. The kitti data set is relatively large, because my computing power is not enough (only 2080ti*2), so I want to use part of the data set for training, for example, there is a five-day data set, can I only use a one-day data set, please guide me How to reduce the dataset for training, looking forward to your reply.Thank you for your patience.

wly-ysu avatar Jun 15 '22 01:06 wly-ysu