Michael Abbott

I think this loop is exactly what we want in the quickstart: ``` for i in 1:10 losses = [] for (x, y) in loader ``` But I do not...

Yes I agree it's specialised to some uses. It just seems slightly weird to force people to define a function which is the just adjusting the signature to work, not...

> confused users who ported over a `loss(x, y)` function from some other library and don't understand the resulting MethodError Right now this is worse, `loss(x, y) = norm(x -...

OK, https://fluxml.ai/Flux.jl/previews/PR2114/training/training/ takes this view that we should just always make an anon. function. It emphasises gradient + update over `train!`, and for gradient you are always going to want...

Yes this is my concern too. I wonder a bit how useful these things really are, how often do people get NaN? Maybe typically through the loss becoming infinite etc?...

> Shouldn't it be faster? It need only call `log` N times, not N^2. Having such a method as a fast path for `logitcrossentropy(ŷ::AbstractArray, y::OneHotArray)` seems like half the point...

Haha no problem. For crossentropy this path does `logits = log.(ŷ .+ ϵ)` which is N^2 log, and could do N. The other path does `xlogy.(y, ŷ .+ ϵ)` which...

Re this fast path, is it just that gather has no bounds checks? That's what https://github.com/FluxML/NNlibCUDA.jl/pull/51 was going to fix, I think. API-wise, the question above is: Why should a...

There are a few which do accept labels, like [binarycrossentropy](http://fluxml.ai/Flux.jl/stable/models/losses/#Flux.Losses.binarycrossentropy). Almost all the others expect two arrays the same size, and could plausibly take a OneHotMatrix as the truth. They...

Sure, that's what I meant about binarycrossentropy. Labels and only labels. But note that it's a separate function, rather than being a method `crossentropy(ŷ::Matrix, y::Vector{Bool})` or `crossentropy(ŷ::Vector, y::Vector{Bool})`. (It takes...