code to load 2D parameters (pretrained on imagenet) to 3D model

[emergencyd]

Could you please provide the code which loads 2D parameters (pretrained on imagenet) to I3D model？ (especially the processing of BN/GN).

Thank you very much!

[joaoluiscarreira]

Hi,

this is old tensorflow code (2017), but hopefully still helpful:

def assign_from_checkpoint_2d_to_3d_scale(model_path, var_list): """Creates an operation to assign specific variables from a checkpoint.

Args: model_path: The full path to the model checkpoint. To get latest checkpoint use model_path = tf.train.latest_checkpoint(checkpoint_dir) var_list: A list of Variable objects or a dictionary mapping names in the checkpoint to the corresponding variables to initialize. If empty or None, it would return no_op(), None.

Returns: the restore_op and the feed_dict that need to be run to restore var_list.

Raises: ValueError: If the checkpoint specified at model_path is missing one of the variables in var_list. """

reader = pywrap_tensorflow.NewCheckpointReader(model_path)

if isinstance(var_list, (tuple, list)): var_list = {var.op.name: var for var in var_list}

feed_dict = {} assign_ops = []

for checkpoint_var_name in var_list: var = var_list[checkpoint_var_name] if not reader.has_tensor(checkpoint_var_name): raise ValueError( 'Checkpoint is missing variable [%s]' % checkpoint_var_name)

var_value = reader.get_tensor(checkpoint_var_name)
placeholder_name = 'placeholder/' + var.op.name
placeholder_value = tf.placeholder(
    dtype=var.dtype.base_dtype,
    shape=var.get_shape(),
    name=placeholder_name)
assign_ops.append(var.assign(placeholder_value))

if var.get_shape() != var_value.shape:
  n = var_value.shape[0]
  feed_dict[placeholder_value] = np.tile(np.expand_dims(var_value/n,0), [n,1,1,1,1])
else:
  feed_dict[placeholder_value] = var_value.reshape(var.get_shape())

assign_op = tf.group(*assign_ops) return assign_op, feed_dict

[emergencyd]

Thank you very much! Is there bn/gn parameter in the pretrained 2D model? if so, do we need to ignore it or also load it to the I3D model?

[joaoluiscarreira]

There was batch norm in the pretrained 2d model, and i think that code loads the corresponding parameters. We also didn't freeze the batch norm parameters when finetuning on any of the datasets in the paper.

Joao

On Wed, Feb 20, 2019 at 2:04 PM emergencyd [email protected] wrote:

Thank you very much! Is there bn/gn parameter in the pretrained 2D model? if so, do we need to ignore it or also load it to the I3D model?

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/deepmind/kinetics-i3d/issues/52#issuecomment-465587915, or mute the thread https://github.com/notifications/unsubscribe-auth/AO6qau9nCYsIqeG2-ypPeT6T-u1IXTcLks5vPVYJgaJpZM4bFJV5 .