Failure with shared embedding layer

[iskandr]

I'm using a shared embedding layer between multiple lengths of sequence inputs and get the following error:

/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/site-packages/keras_diagram/diagram.py in ascii(model)
    167 
    168 def ascii(model):
--> 169     node = Node(model.layers[-1])
    170     return node.render()
    171 

/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/site-packages/keras_diagram/diagram.py in __init__(self, layer)
     11         self.layer = layer
     12         self.text = "%20s %-20s" %  (self._name(), layer.output_shape)
---> 13         self.children = self._calculate_children()
     14         self.node_width = len(self.text)
     15         self.family_width = self._calculate_family_width()

/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/site-packages/keras_diagram/diagram.py in _calculate_children(self)
     25         layers = list(_flatten([node.inbound_layers for node in self.layer.inbound_nodes]))
     26         layers = [l.layers[-1] if issubclass(type(l), Model) else l for l in layers]
---> 27         return [Node(l) for l in layers]
     28 
     29     def _calculate_family_width(self):

/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/site-packages/keras_diagram/diagram.py in <listcomp>(.0)
     25         layers = list(_flatten([node.inbound_layers for node in self.layer.inbound_nodes]))
     26         layers = [l.layers[-1] if issubclass(type(l), Model) else l for l in layers]
---> 27         return [Node(l) for l in layers]
     28 
     29     def _calculate_family_width(self):

/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/site-packages/keras_diagram/diagram.py in __init__(self, layer)
     11         self.layer = layer
     12         self.text = "%20s %-20s" %  (self._name(), layer.output_shape)
---> 13         self.children = self._calculate_children()
     14         self.node_width = len(self.text)
     15         self.family_width = self._calculate_family_width()

/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/site-packages/keras_diagram/diagram.py in _calculate_children(self)
     25         layers = list(_flatten([node.inbound_layers for node in self.layer.inbound_nodes]))
     26         layers = [l.layers[-1] if issubclass(type(l), Model) else l for l in layers]
---> 27         return [Node(l) for l in layers]
     28 
     29     def _calculate_family_width(self):

/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/site-packages/keras_diagram/diagram.py in <listcomp>(.0)
     25         layers = list(_flatten([node.inbound_layers for node in self.layer.inbound_nodes]))
     26         layers = [l.layers[-1] if issubclass(type(l), Model) else l for l in layers]
---> 27         return [Node(l) for l in layers]
     28 
     29     def _calculate_family_width(self):

/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/site-packages/keras_diagram/diagram.py in __init__(self, layer)
     11         self.layer = layer
     12         self.text = "%20s %-20s" %  (self._name(), layer.output_shape)
---> 13         self.children = self._calculate_children()
     14         self.node_width = len(self.text)
     15         self.family_width = self._calculate_family_width()

/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/site-packages/keras_diagram/diagram.py in _calculate_children(self)
     25         layers = list(_flatten([node.inbound_layers for node in self.layer.inbound_nodes]))
     26         layers = [l.layers[-1] if issubclass(type(l), Model) else l for l in layers]
---> 27         return [Node(l) for l in layers]
     28 
     29     def _calculate_family_width(self):

/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/site-packages/keras_diagram/diagram.py in <listcomp>(.0)
     25         layers = list(_flatten([node.inbound_layers for node in self.layer.inbound_nodes]))
     26         layers = [l.layers[-1] if issubclass(type(l), Model) else l for l in layers]
---> 27         return [Node(l) for l in layers]
     28 
     29     def _calculate_family_width(self):

/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/site-packages/keras_diagram/diagram.py in __init__(self, layer)
     11         self.layer = layer
     12         self.text = "%20s %-20s" %  (self._name(), layer.output_shape)
---> 13         self.children = self._calculate_children()
     14         self.node_width = len(self.text)
     15         self.family_width = self._calculate_family_width()

/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/site-packages/keras_diagram/diagram.py in _calculate_children(self)
     25         layers = list(_flatten([node.inbound_layers for node in self.layer.inbound_nodes]))
     26         layers = [l.layers[-1] if issubclass(type(l), Model) else l for l in layers]
---> 27         return [Node(l) for l in layers]
     28 
     29     def _calculate_family_width(self):

/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/site-packages/keras_diagram/diagram.py in <listcomp>(.0)
     25         layers = list(_flatten([node.inbound_layers for node in self.layer.inbound_nodes]))
     26         layers = [l.layers[-1] if issubclass(type(l), Model) else l for l in layers]
---> 27         return [Node(l) for l in layers]
     28 
     29     def _calculate_family_width(self):

/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/site-packages/keras_diagram/diagram.py in __init__(self, layer)
     11         self.layer = layer
     12         self.text = "%20s %-20s" %  (self._name(), layer.output_shape)
---> 13         self.children = self._calculate_children()
     14         self.node_width = len(self.text)
     15         self.family_width = self._calculate_family_width()

/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/site-packages/keras_diagram/diagram.py in _calculate_children(self)
     25         layers = list(_flatten([node.inbound_layers for node in self.layer.inbound_nodes]))
     26         layers = [l.layers[-1] if issubclass(type(l), Model) else l for l in layers]
---> 27         return [Node(l) for l in layers]
     28 
     29     def _calculate_family_width(self):

/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/site-packages/keras_diagram/diagram.py in <listcomp>(.0)
     25         layers = list(_flatten([node.inbound_layers for node in self.layer.inbound_nodes]))
     26         layers = [l.layers[-1] if issubclass(type(l), Model) else l for l in layers]
---> 27         return [Node(l) for l in layers]
     28 
     29     def _calculate_family_width(self):

/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/site-packages/keras_diagram/diagram.py in __init__(self, layer)
     10     def __init__(self, layer):
     11         self.layer = layer
---> 12         self.text = "%20s %-20s" %  (self._name(), layer.output_shape)
     13         self.children = self._calculate_children()
     14         self.node_width = len(self.text)

/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/site-packages/keras/engine/topology.py in output_shape(self)
    827                             'the notion of "output shape" is ' +
    828                             'ill-defined for the layer. ' +
--> 829                             'Use `get_output_shape_at(node_index)` instead.')
    830 
    831     def set_weights(self, weights):

Exception: The layer "amino_acid_embedding has multiple inbound nodes, with different output shapes. Hence the notion of "output shape" is ill-defined for the layer. Use `get_output_shape_at(node_index)` instead.

[brianlow]

Thanks @iskandr. Any chance you can post a compilable model?

[iskandr]

Hey @brianlow,

I tried to trim the code down to a manageable subset:

shared_embedding_layer = Embedding(
    input_dim=20,
    output_dim=64,
    name="amino_acid_embedding")

def make_length_specific_model(
        input_length,
        shared_embedding_layer,
        hidden_activation_function="tanh",
        hidden_layer_sizes=[]):
    input_layer = Input(
            shape=(input_length,), 
            dtype="int32", 
            name="input_for_length_%d" % input_length)
    x = Flatten()(shared_embedding_layer(input_layer))

    for i, hidden_layer_size in enumerate(hidden_layer_sizes):
        hidden_name = "hidden%d_for_length_%d" % (i + 1, input_length)
        print("Making %s (size=%d)" % (hidden_name, hidden_layer_size))
        x = Dense(size, name=name, activation=hidden_activation_function)(x)
    output = Dense(
        output_dim=1, 
        input_dim=hidden_layer_sizes[-1] if hidden_layer_sizes else shared_embedding_layer.output_dim * input_length,
        activation="sigmoid", 
        name="output")(x)
    model = Model(input=[input_layer], output=[output])
    model.compile(optimizer='adam',
              loss={'output': 'mse'},
              loss_weights={'output': 1.})
    return model

model_for_8mers = make_length_specific_model(8, shared_embedding_layer)
model_for_9mers = make_length_specific_model(9, shared_embedding_layer)

If you try to make a diagram for either of those models then you should get the same error.

[brianlow]

Thanks, working on it.

[brianlow]

I have a failing test but can't crack this one.

Not sure how to determine which input layer to include (currently it is trying to include both).

Does the the Keras visualization module handle this model? http://keras.io/visualization/

[iskandr]

Keras shows the inputs for just one model (even if a layer has inputs coming into it from other models, they're ignored). That seems like reasonable behavior. Alternatively, providing a way to visualize the combined graph of multiple models with shared structure would be good too.

On Thu, Jul 14, 2016 at 11:30 PM, brianlow [email protected] wrote:

I have a failing test but can't crack this one.

Not sure how to determine which input layer to include (currently it is trying to include both).

Does the the Keras visualization module handle this model? http://keras.io/visualization/

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/brianlow/keras_diagram/issues/2#issuecomment-232851773, or mute the thread https://github.com/notifications/unsubscribe-auth/AAC9OVzI7XDqCWTy84bvUMM0qI71bMpMks5qVv7LgaJpZM4JKye4 .