Mike J Innes
Mike J Innes
We are actively working on NNPACK in #67. My main issue with MKL-DNN is that it seems to work best if you build its computational graph thing, rather than exposing...
Good idea. cc @pfitzseb
Yeah, that's a great idea.
Agreed! Sorry we missed hacktoberfest, but if you're still willing to give this a go we'd be happy to help.
I think that kind of demo would be great for the API work but I'm not sure it makes as much sense for the ideas discussed here. In order to...
Yes, that or some wrapper for an existing implementation. A generic kernel is probably a good idea either way though.
Yeah I'm pretty sure I got it working at some point.
Yeah, that's probably the way to do it, and would delay having to write a kernel for it. It's annoying because we can't dispatch on CPU-ness, so we have to...
@SimonDanisch perhaps [PyTorch's version](https://github.com/pytorch/pytorch/blob/1ff537ca7198ddef7e76d0f75c001d0b3f41d7fd/aten/src/THCUNN/SoftMaxCommon.cuh#L398) can be ported as a generic kernel?
Here's a basic implementation: ```julia function softmax!(y, x; dim=1) tmp = maximum(x, dim) y .= exp.(x .- tmp) sum!(tmp, y) y ./= sum(y, dim) end softmax(x; dim = 1) =...