Usage of image branch and possible scaling bug

[ggalan87]

I am interested in investigating results with and without the image branch of the network. As far as I am concerned you do not provide checkpoint which utilizes such a branch, so I trained the network enabling this branch, on my own.

First of all, I want to know if I am correct that corner_acc.npy, topdown_acc.npy which you provide within your example are the weights which correspond to pretraining of DRN and HG networks, thus we utilize them as-is for both training and validation/testing purposes.

Next I want to point out a possible bug related to image features which is the following. I realized that within RecordWriterTango.py, RGB values are scaled twice.

One time during loading: https://github.com/art-programmer/FloorNet/blob/e7bd879df7d7825b1badb09440c85b4f1107a6d6/RecordWriterTango.py#L99

Second time during processing: https://github.com/art-programmer/FloorNet/blob/e7bd879df7d7825b1badb09440c85b4f1107a6d6/RecordWriterTango.py#L334

This results to RGB values within a range (-0.5, -0.49607843)

It can be reproduced with the following snippet (similar for training records)

record_filepath ='Tango_val.tfrecords'
dataset = getDatasetVal([record_filepath], '', True, 1)
iterator = dataset.make_one_shot_iterator()
input_dict, gt_dict = iterator.get_next()

pt = input_dict['points'][0]
points = tf.Session().run([pt])[0]
RGB = points[:, 3:6]
print(np.amin(RGB), np.amax(RGB))

Could such issue affect how the values are interpreted by the network? Is this related to already pretrained weights?

Thanks in advance

[art-programmer]

You are right on both points. It seems that the scaling indeed happens twice, which is a bug. I am not sure how much will fixing the scaling issue benefit the network. Need to test later. The pretrained weights correspond to the data provided in the .tfrecords.