Avik Pal

Forward Mode for Neural ODEs / Structured Parameters broken

1

Test from Optimization.jl: https://github.com/SciML/Optimization.jl/blob/master/test/minibatch.jl Error: https://github.com/SciML/Optimization.jl/actions/runs/10550106197/job/29225903838#step:7:1169 The ComponentArrays parameters are being converted to regular arrays, so Lux doesn't know how to handle them. ReverseMode works fine. cc @Vaibhavdixit02 (who also...

How can we make KA fast on CPUs?

10

See https://github.com/LuxDL/LuxLib.jl/pull/136 for some background context. The main motivation for me is to avoid code duplication between CPU and GPU versions. However, if you take a look at the benchmark...

How (if possible) can we get rid of SparseArrays and StaticArrays?

3

```julia julia> @time_imports using KernelAbstractions 9.8 ms UnsafeAtomics 6.4 ms Atomix ┌ 0.8 ms SuiteSparse_jll.__init__() 116.4 ms SuiteSparse_jll 96.92% compilation time ┌ 8.2 ms SparseArrays.CHOLMOD.__init__() 95.65% compilation time 156.1 ms...

Currently several places in the codebase uses this pattern: ```julia adapt(eltypeθ, hcat(vec(map(points -> collect(points), Iterators.product(span...)))...)) ``` But, `adapt` doesn't work with eltype, for example: ```julia julia> adapt(Float32, rand(2, 3)) 2×3...

fixes #929 needs https://github.com/LuxDL/Boltz.jl/pull/54

fix: dependency management

2

- removes dependencies that aren't being used - loads DataInterpolations (fixes #935) - removes duplicated tests

[WIP] perf: common neural network workloads benchmarks

3

currently add CPU versions. should be easy to extend to GPU once SciMLBenchmarks has GPU runners available. ## TODOs - [ ] Restore Manifest to use the released versions of...

checking whether we need a full revert like https://github.com/SciML/DiffEqBase.jl/pull/1075 or just reverting the ext change will do

Check if function is being called inside `autodiff`

4

## Conversation from Slack ### `@avik-pal` In Enzyme can we check if a function is being called within an autodiff? Something equivalent to the following ChainRules version ```julia within_gradient(_) =...