QIAOQIAN

I encounter the same issue, the train loss suddenly jump from ~3.0 to ~7.0, and then stay around 7.0 for a long time. Did you figure out the issue?

change from `[[Simd; B]; IN]` to `[[Simd; B]; IN/8]` can greatly reduce the stack usage and slightly enhance the reference speed.

i tried the `#![feature(generic_const_exprs)] #![allow(incomplete_features)]` but it generates a bunch of warnings. On my desktop, the speed changed from `achieved tok/s: 4.8163757` to `achieved tok/s: 5.0361977`

this proj is a great place to learn rust and llama and cuda(triton), very appreciated, hope to do something helpful to the proj

> Would love any contribution, I'm also learning Rust and Triton on the fly. > > What if we try this library? It seems pretty cool. https://docs.rs/typenum/latest/typenum/ import of the...

> Another idea would be to explore adding testing. Not sure how unit tests work in rust, but it would be nice to have these for small sizes. I am...

Settings -> Files -> Default File Type / Extension -> Custom(choose this) -> Custom Extension: mdx ------------------ set [Custom Extension] to mdx then typora will not change you extension after...

set [Custom Extension] to txt, typora will automatically rename to txt extention. it seems a bug when you set `mdx` make the rename operation failed, so it keep your extension,...