kernl icon indicating copy to clipboard operation
kernl copied to clipboard

Published 20 hours ago • verified publisher icon ELS-RD

feat: add bw pass on layernorm/rmsnorm

Open pommedeterresautee opened this issue 2 years ago • 3 comments

pommedeterresautee avatar Nov 06 '22 19:11 pommedeterresautee

I notice this PR has been open for a while; what's its status?

samhavens avatar Mar 08 '23 22:03 samhavens

@samhavens hi, no real issue, just focusing on inference right now... Is it something you are interested into?

pommedeterresautee avatar Mar 21 '23 09:03 pommedeterresautee

Not having to deal with CUDA for an RMSnorm kernel is appealing, yeah 😄 it's not high priority currently, but I wanted to make sure to keep tabs on this PR 👀

samhavens avatar Mar 21 '23 17:03 samhavens