interacting agents in interaction prediction dataset

[srikanthmalla]

Hi, How are the 2 interacting agents labeled in this interaction prediction dataset: https://waymo.com/open/challenges/2021/interaction-prediction/

Was there some criteria for selecting those interacting objects?

Best Regards, Srikanth

[YikangZhang1641]

I'm suffering from the same problem, lol. Only data in folder 'testing_interactive' and 'validation_interactive' have exact 2 labeled vehicles in 'state/tracks_to_predict' field, and we need to search for the pairs in training data by ourselves, I think.

Waymo recently proposed a mutual information method, which is a brilliant way to calculate the interaction score, but I don't quite understand the details(for example, in the paper, the road lane encoder follows VectorNet, but we don't have road connectivity in this dataset). It would be very helpful if some open source code or supplyment materials are provided.

[s-ettinger]

Hi Srikanth, The interacting objects were selected using an algorithm that mines for specific types of interactions (lane changes, merges, vehicle/pedestrian interactions, etc.). We don't specifically provide pairs for all scenarios in the training set, however for some training scenarios (slightly less than half) we do provide a field called "objects_of_interest". These objects are interacting objects selected with the same algorithm as those in the interactive validation and test sets. Be aware it is possible for this field to only have one object in some cases. Many scenarios do not have these objects labeled as those scenarios do not contain a specific mined interaction.

[srikanthmalla]

Waymo recently proposed a mutual information method, which is a brilliant way to calculate the interaction score.

Hi @YikangZhang1641, Can you point to the paper that you are referring to regarding mutual information method?

Best Regards, Srikanth

[srikanthmalla]

The interacting objects were selected using an algorithm that mines for specific types of interactions (lane changes, merges, vehicle/pedestrian interactions, etc.).

Hi @scott-ettinger , Which algorithm are you referring to? Is it publicly available or something internal for waymo?

Best Regards, Srikanth

[s-ettinger]

Hi Srikanth. The interacting object selection algorithm is not publicly available. It is part of an internal code base.

[YikangZhang1641]

"Identifying Driver Interactions via Conditional Behavior Prediction" @srikanthmalla

[ghost]

Hi @scott-ettinger, is there a label showing the interaction types, e.g., vehicle-bicycle, lane-changing, etc.

Thanks.

[s-ettinger]

There is not a separate label, but the types of the interacting objects can be determined by using the tracks_to_predict field (in Scenario protos) or the state/tracks_to_predict field (in tf.Example protos). From the indices in the tracks_to_predict field you can retrieve the interacting objects and get their types from either the object_type (within the Track message of the Scenario proto) or the state/type field (within the tf.Example proto). Please let me know if you have further questions.

-Scott Ettinger Waymo Research

On Sun, May 15, 2022 at 7:31 AM Jasonx @.***> wrote:

Hi @scott-ettinger https://github.com/scott-ettinger, is there a label showing the interaction types, e.g., vehicle-bicycle, vehicle-pedestrian, etc.

— Reply to this email directly, view it on GitHub https://github.com/waymo-research/waymo-open-dataset/issues/288#issuecomment-1126953536, or unsubscribe https://github.com/notifications/unsubscribe-auth/ATYRUO7Q7HCTD274CH7N5PDVKEDD5ANCNFSM4327AYNA . You are receiving this because you were mentioned.Message ID: @.***>

[ghost]

Hi, @scott-ettinger. Thanks for your reply. I have checked out the xxx_interactive datasets. It seems that few AVs are included in in 'state/tracks_to_predict' field.

I am interested in human-AV interactions, e.g., how people follow AV and how AV affects other drivers when making a lane-change. Is there any approach to mining these AV-involved car-following or lane-changing events?

Thanks.