retinanet-examples icon indicating copy to clipboard operation
retinanet-examples copied to clipboard

got IndexError

Open jjxyai opened this issue 4 years ago • 1 comments

boxes = box_head[a, :, y, x]

IndexError: tensors used as indices must be long, byte or bool tensors

https://github.com/NVIDIA/retinanet-examples/blob/main/odtk/box.py#L299

i wonder why use '/' instead of '//' to get the index?

i modified the code to

`

    # classes = indices // width // height % num_classes  # modified

    # classes = classes.type(all_cls_head.type())
    
    # # Infer kept bboxes
    # x = indices % width  # torch.int64

    # y = indices // width % height  # modified

    # a = indices // width // height // num_classes  # modified

`

jjxyai avatar Sep 14 '21 10:09 jjxyai

I agree, I faced the same problem and it makes sense to modify the code as you have documented above. Single '/' in python can give float values, but '//' will give the integer value. After the modification, It's working fine.

ankushgupta7 avatar Feb 24 '22 00:02 ankushgupta7