Andrew Fitzgibbon

Thanks @leycec ! Just to clarify on "destroying runtime performance", do you mean making it worse than ```python query : LxDk = linear(head.query, t1) die_if_unbearable(query, LxDk) key : LxDk =...

Because the above seems acceptable to me, particularly under a jax-style define-by-run scheme. And if it's not I might wrap `die_if_unbearable` to have logic like: ```python die_if_unbearable(query, LxDk) if rand()...

Thanks both for this discussion - I implemented something quick based on @patrick-kidger's spike above, and it seems to work quite nicely. Next step is to integrate with jaxtyping, but...

I see: you're doing all your work at https://github.com/google/jaxtyping/blob/35201eb189cc004276925f96e0aa6bfc469e46be/jaxtyping/array_types.py#L102, and then typeguard says ``` elif not isinstance(value, expected_type): raise TypeError( 'type of {} must be {}; got {} instead'. format(argname,...

Exactly so. It might even be a case for, ugh, an environment variable, so a usage pattern might be ``` % python t.py ... Error message. % JAXTYPING=verbose python t.py...

Thanks. In the short run, even a hyperlink to the manual would be considerably more ergonomic.

I think we should remove replicate point mode. We should perhaps even just use random points, with a given seed.

Yes that is very odd, and doesn't ring a bell. Let's also make this issue track other LSTM oddnesses: diagonal weight matrix. Any more?

Specifically, we should not produce all graphs for all branches, but have a way in which a branch can initiate a benchmark run for that branch. For example, one could...

I just checked the document, https://github.com/awf/ADBench/blob/0a4f34e371deec4c45d3654bca4260f996730622/Documents/ms.tex#L201-L204 Which is also column-major, so this is a significant change.