Graph Normalization and miniKanren Goals for TensorFlow Graphs

[brandonwillard]

After TensorFlow graph support is added (i.e. #4), we will need to adapt the existing miniKanren relations to TF[P] objects.

This might require a core set of linear/tensor-algebraic graph normalization/canonicalization relations, as well, since those were previously provided by Theano.

FYI: It will probably be easier to create new relations specific to TF[P]—compared to generalizing the meta objects enough to cover both. Regardless, we should keep both options in mind.

[josephwillard]

What would be the best way to approach this issue?

[brandonwillard]

We can start by creating a goal, normalizeo, and make it able to normalize according to all the standard scalar group axioms under a simple ordering like lexicographical path ordering.

For example, the standard, reduced normalization rewrite-rules for scalars:

inv(e) -> e
x * e -> x
e * x -> x
inv(inv(x)) -> x
x * inv(x) -> e
inv(x) * x -> e
x * (inv(x) * y) -> y
inv(x) * (x * y) -> y
(x * y) * z -> x * (y * z)
inv(x * y) -> inv(y) * inv(x)

Next, we'll need to add some rewrite rules for operator properties (e.g. linearity), tensors, and their interactions with scalars. We might be able to get away with some small combination of axioms.

After that, we can designate specific TF operators (e.g. RandomStandardNormal, Add, etc.) as having said properties using kanren.facts.