Question about "current information"

[FedericoVasile1]

Hi, congrats for your work and great results! I read your paper and I did not understand what you intend for "current information input", what should I concretely provide as current information input to IDU? From what I understand, at each timestep IDU has 3 inputs: Ht-1, Xt, X0. Ht-1 is the hidden state of the previous timestep, Xt is the feature vector extracted at the current timestep, and what about X0(current information)?

Thank you :)

[hjeun]

As you mentioned, IDU has 3 inputs. Among them, X0 is the feature vector extracted at the timestep of interest, i.e., t0. For online action detection, what the network predicts is the action at t0 when action is happening now.

[FedericoVasile1]

So let's suppose I have T = 16 unrolling steps for the idu cell (as you said in the paper), that means I have a sample tensor of shape (T, feature_vector_dim). To clarify what should be the input to IDU at each step of T I write here a simple pseudo-code, to check if I understand correcly:

h_t = zeros
for step in range(T):
     h_t = IDU.forward(feat_vects[step], feat_vects[-1], h_t)

where feat_vects is the above mentioned tensor. Is this pseudo-code what you mean?

Furthermore, another question: the predictions are made at each step of T (using the current hidden state obtained at that step) or only at the end of the sequence T (hence using the last hidden state)

Thank you for your patience 👍

PS: I'm doing a personal pytorch implementation of your paper for an home-made application for online action detection purposes, if you want once is ended I can do a pull request :)