yaroslavvb
Linearize creating cyclic graph.
I tried to use the lineare.linearize() function, but it created cyclic graph.
I tried that to optimize CycleGAN, which has lots of gradients.
Without the usage of lineare.linearize() everything works, but after adding lineare.linearize() after the graph building, the sess.run(tf.global_variables_initializer()) fails with tensorflow.python.framework.errors_impl.InvalidArgumentError: Graph is invalid, contains a cycle with 107 nodes, including: Adam/real-anime_gen/Select/e, Adam/real-anime_gen/Select, Adam/real-anime_gen/learning_rate.
But the linearize() function itself worked and did not detect any cycles.
models.zip
I am adding original graph and linearized version as .pb files in zip (in this example generators and discriminators are only dummy one-layer networks to simplify the problem) and also html visualization of linearized graph (based on your visualizer from https://github.com/yaroslavvb/stuff/blob/master/simple_rewiring.ipynb which is really nice btw, thanks for that).
It creates cycles between individual generators, apparently it does not handle well the cycle loss.
I also modified the end of the linearize function and added
toposort(copy_graph(get_graph(tf.get_default_graph())))
toposort(copy_graph(reversed_graph(get_graph(tf.get_default_graph()))))
before the return statement. In case of cyclic graph, at least one of them should throw the CircularDependencyError, right? But no error was thrown there and still the cycle remains and tensorflow complains and throws exception.
