KeyError: self.env.attention_radius[(VEHICLE, VEHICLE)] in OnlineMultimodalGenerativeCVAE.encode_edge

[j-scharrenbach]

Hi everybody,

I implemented a demo application using your repository (it was a pleasure using such a well-structured codebase, great job!).

When running the application I encountered the following problem:

Traceback (most recent call last):
  [...]
  File ".../trajectron/model/online/online_trajectron.py", line 199, in incremental_forward
    maps)
  File ".../trajectron/model/online/online_mgcvae.py", line 382, in encoder_forward
    maps)
  File ".../trajectron/model/online/online_mgcvae.py", line 192, in obtain_encoded_tensors
    edge_masks_batched)
  File ".../trajectron/model/online/online_mgcvae.py", line 298, in encode_edge
    std[0:2] = self.env.attention_radius[edge_type_tuple]
KeyError: (VEHICLE, VEHICLE)

After some debugging I found out that within OnlineMultimodalGenerativeCVAE.encode_edge (online_mgcvae.py#L286) calling isinstance(self.node_type, type(edge_type_tuple[0])) from the debugger returns False.

This causes that within the lookup in self.env.attention_radius (online_mgcvae.py#L297) the implicitly called NodeType.__eq__ (node_type.py#L9) fails due to the returned False of isinstance(...). Both the name and the value are equal for selfand other and from a semantic perspective both elements should be equal. Just the types of these elements seem not to be treated as equal (self.node_type is not an instance of the environment.node_type.NodeType somehow).

The node gets created by using node_type=env.NodeType.VEHICLE, I also tried to hand over a 'fresh' NodeTypeobject. Both ways should provide the correct type (as far as I undestand) but fail with the same error.

I tracked down the issue to online_mgcvae.py#L122 and fixed it by changing the functio to:

def _get_edge_type_from_str(self, edge_type_str):
    n1_type_str, n2_type_str = edge_type_str.split('->')
    return (self.env.NodeType[self.env.NodeType.node_type_list.index(n1_type_str)],
            self.env.NodeType[self.env.NodeType.node_type_list.index(n2_type_str)])

Instead of returning a newly created NodeType object I am using the one already existent within the Environment object.

Do you have any recommendations what my issue might be or is it possible thet there is a bug within the repository?

Thanks in advance! Jannik

[BorisIvanovic]

Hi Jannik!

Thank you for outlining such a thorough analysis, it's been a while since I looked at the online part of the codebase 😊

I'll be honest, it's completely possible that the online code has a bug in it (it's the least tested part of the codebase overall). All of the node type management in particular was a hassle even in the beginning... I'm sorry if this caused you some pain!

Please feel free to submit a pull-request with your recommended fix and then I'll take a closer look sometime later this month (I'm currently traveling).

Thanks! Boris

[j-scharrenbach]

Hi Boris,

Thank you for the reply 😊 I created a pull request #52 with minor changes to the code.

Honestly, I learned a lot while digging through your code, so time was well-spent on my side.

Enjoy the traveling! Jannik