Justin Bayer

Introduce targets for either generating a debug build or a production build.

Although importance was introduced for datasets, there are no tests so far.

Introduce a class "ErrorFunction" from which error functions like sum-of-squares, cross-entropy and the like can derive.

At a lot of places, errors are not handled very gracefully: - mdrnns with invalid sequence/blockshape sizes have undefined behaviour - networks with cycles just crash due to an assertion...

In order to increase performance (runtime wise), it makes sense to forward minibatches through a network. By this, multiple matrix/vector products can be turned into a single matrix/matrix product which...

LSTM gradient check fails

1

And has to be corrected: ``` [ RUN ] TestGradient.LstmLayer New Parametrized instance. Numeric/Analytical:-0 0 Numeric/Analytical:-0 0 Numeric/Analytical:-0 2.11224 Numeric/Analytical:-0 0 Expected: (0.001) > (gradient_check(*net_p, true)), actual: 0.001 vs 2.11224...

Make the tests run after each build and check whether they pass.

Instead of files, programmatically create gin config

2

In my use cases, I rarely set hyper parameters by hand. Instead I have an external logic that generates them (e.g. random search). Now, using gin it seems I would...

Differentiation wrt system parameters

6

Hey, I can see how brax can be used to differentiate with respect to the system state. I wonder if there is a nice way to also diff wrt, e.g.,...

This includes adding at least one line search that works on the gpu.