a question about data_time

[h1063135843]

trafficstars

When training, One line of log like this Iter [150/80000] lr: 5.949e-06, eta: 4:06:51, time: 0.137, data_time: 0.028 When I extend rgb to rgb-d segmentation, I find data_time increase a lot (0.1s for an iter, several hours finally) like below Iter [100/80000] lr: 3.955e-06, eta: 7:14:43, time: 0.254, data_time: 0.126 Does the data time mean the loading time for image? I print the data_time for each_iter

	RGB-D	RGB
data_time	0.168552	0.004988
data_time	0.001994	0.001995
data_time	0.179523	0.001995
data_time	0.000997	0.000998
data_time	0.302198	0.199458
data_time	0.000998	0.001994
data_time	0.426859	0.000997
data_time	0.000998	0.001997
data_time	0.216421	0.001994
data_time	0.002992	0.001995
data_time	0.196475	0.001995
data_time	0.001995	0.000998
data_time	0.039893	0.235371
data_time	0.032912	0.002992
data_time	0.339092	0.002992
data_time	0.001995	0.001994
data_time	0.295213	0.002991
data_time	0.000998	0.001995
data_time	0.161568	0.001994
data_time	0.001995	0.001995
data_time	0.246344	0.216421
data_time	0.000998	0.000997
data_time	0.462766	0.001995
data_time	0.000997	0.001994
data_time	0.140627	0.002992
data_time	0.000997	0.001993
data_time	0.118685	0.002992
data_time	0.000997	0.000997
data_time	0.141622	0.174533
data_time	0.001995	0.001994
data_time	0.388962	0.001995
data_time	0.002991	0.000997
data_time	0.148601	0.000998
data_time	0.001994	0.000998
data_time	0.272274	0.000998
data_time	0.001996	0.000998
data_time	0.226397	0.196476
data_time	0.001994	0.000998
data_time	0.427856	0.002993
data_time	0.000997	0.000997
data_time	0.208444	0.000998
data_time	0.001995	0.001995
data_time	0.389958	0.001995
data_time	0.003991	0.001994
data_time	0.095744	0.244349
data_time	0.00399	0.000998
data_time	0.376992	0.002992
data_time	0.001994	0.002992
data_time	0.208447	0.001996
data_time	0.002992	0.001995
Average	0.125166	0.027028

It is evident that every twice iter the data_time of rgbd increase, however the data_time of rgb increase every eighth iter . What is the reason of this inconsistency? IO? How to solve it?

[Junjun2016]

Try persistent_works. 发自 网易邮箱大师 ---- 回复的原邮件 ---- 发件人 @.> 日期 2021年09月03日 22:18 收件人 @.> 抄送至 @.***> 主题 [open-mmlab/mmsegmentation] a question about data_time (#852) When training, One line of log like this Iter [150/80000] lr: 5.949e-06, eta: 4:06:51, time: 0.137, data_time: 0.028 When I extend rgb to rgb-d segmentation, I find data_time increase a lot (0.1s for an iter, several hours finally) like below Iter [100/80000] lr: 3.955e-06, eta: 7:14:43, time: 0.254, data_time: 0.126 Does the data time mean the loading time for image? I print the data_time for each_iter   RGB-D RGB data_time 0.168552 0.004988 data_time 0.001994 0.001995 data_time 0.179523 0.001995 data_time 0.000997 0.000998 data_time 0.302198 0.199458 data_time 0.000998 0.001994 data_time 0.426859 0.000997 data_time 0.000998 0.001997 data_time 0.216421 0.001994 data_time 0.002992 0.001995 data_time 0.196475 0.001995 data_time 0.001995 0.000998 data_time 0.039893 0.235371 data_time 0.032912 0.002992 data_time 0.339092 0.002992 data_time 0.001995 0.001994 data_time 0.295213 0.002991 data_time 0.000998 0.001995 data_time 0.161568 0.001994 data_time 0.001995 0.001995 data_time 0.246344 0.216421 data_time 0.000998 0.000997 data_time 0.462766 0.001995 data_time 0.000997 0.001994 data_time 0.140627 0.002992 data_time 0.000997 0.001993 data_time 0.118685 0.002992 data_time 0.000997 0.000997 data_time 0.141622 0.174533 data_time 0.001995 0.001994 data_time 0.388962 0.001995 data_time 0.002991 0.000997 data_time 0.148601 0.000998 data_time 0.001994 0.000998 data_time 0.272274 0.000998 data_time 0.001996 0.000998 data_time 0.226397 0.196476 data_time 0.001994 0.000998 data_time 0.427856 0.002993 data_time 0.000997 0.000997 data_time 0.208444 0.000998 data_time 0.001995 0.001995 data_time 0.389958 0.001995 data_time 0.003991 0.001994 data_time 0.095744 0.244349 data_time 0.00399 0.000998 data_time 0.376992 0.002992 data_time 0.001994 0.002992 data_time 0.208447 0.001996 data_time 0.002992 0.001995 Average 0.125166 0.027028 It is evident that every twice iter the data_time of rgbd increase, however the data_time of rgb increase every eighth iter . What is the reason of this inconsistency? IO? How to solve it? — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub, or unsubscribe. Triage notifications on the go with GitHub Mobile for iOS or Android.

[h1063135843]

@Junjun2016 I am using the latest version of mmsegmentation. Persistent_works is default to be true. The data_time of rgb input is low enough . The problem lies in the rgb-d situation.

[MengzhangLI]

Hi, @h1063135843 what is channel number of your image? Seems like 4 channels, rgb-d?

[h1063135843]

yes, four channel, rgbd

---Original--- From: @.> Date: Fri, Dec 17, 2021 16:16 PM To: @.>; Cc: @.@.>; Subject: Re: [open-mmlab/mmsegmentation] a question about data_time (#852)

Hi, @h1063135843 what is channel number of your image? Seems like 4 channels, rgb-d?

— Reply to this email directly, view it on GitHub, or unsubscribe. Triage notifications on the go with GitHub Mobile for iOS or Android. You are receiving this because you were mentioned.Message ID: @.***>

[MengzhangLI]

Currently we do not support 4 channels segmentation.

What modifications you made? We consider support it in the near future.

[h1063135843]

I preprocessed all color and depth to four channel images.(RGBA, I use alpha channel to store depth) In config files, change some parameters. dict(type='LoadImageFromFile', color_type='unchanged') And in data augmentation, photo distortion is discarded as it change alpha channel. Then channge the network input channel from 3 to 4. I separate color and depth in the forward function mannualy. Here, it is an example network based on segformer to handle rgbd. To support RGB-D, it's better to provide some scripts to preprocess original datasets, and add some network specific for RGB-D handling. Also, there are many exiting scripts not considering four channel input such as get flops.py.

[MengzhangLI]

Very useful information for us, thank you so much.