Lukasz Stafiniak

This might be as simple as configuring the compiler path, but probably there will be issues...

Consider: Refactor / rewrite gradient management from storing backprop computations to storing backprop tensors

1

This is high priority as it is big design impact. It removes the "differentiable / non-differentiable tensors" distinction. This does not get rid of global session state management with implicit...

https://siboehm.com/articles/22/Fast-MMM-on-CPU

Consider getting rid of the global-state solution to routine reus

1

tinygrad did this: https://x.com/__tinygrad__/status/1883459978566262803 It would be a major design change. E.g. we would probably get rid of the global-state solution to routine reuse.

Tensor nodes that are used in different routines must be materialized. Tensor nodes that are outputs/sinks, but not inputs in any routine, should be hosted; etc. Actually, check first if...

https://github.com/srush/Tensor-Puzzles

Deep dives into Petalisp and Caten

1

https://github.com/hikettei/Caten "Deep Learning Compiler based on Polyhedral Compiler, Light-weight IRs, and Optimizing Pattern Matcher." hikettei.github.io/Caten/ Young impressive project (started July/August 2024). https://github.com/marcoheisig/Petalisp "Elegant High-Performance Computing" Old and active project.

It's not needed, but cool and not much work.

https://iree.dev/ It builds on top of MLIR.

Implement the XLA backend, compiling directly from `Assignments.comp`. Relevant links: - https://openxla.org/xla/operation_semantics - https://github.com/LaurentMazare/ocaml-xla - https://github.com/openxla/xla/blob/main/xla/hlo/builder/xla_builder.h - https://github.com/openxla/xla/blob/main/xla/service/hlo.proto