Replacing uniform with uniform_philox , and incrementing PRNG state
Before submitting
- [ ] Was this discussed/approved via a Github issue? (no need for typos and docs improvements)
- [ ] Did you read the contributor guideline, Pull Request section?
- [ ] Did you make sure to update the docs?
- [ ] Did you write any new necessary tests?
What does this PR do?
Fixes #114
- [x] Add the prim operators to query/update the default CUDA RNG state PR
- [ ] Fix #231
- [x] Add transformation to replacing uniform with uniform_philox (this PR)
- [ ] Make sure the rematerialization works properly for Dropout
TODO: fix #231 update the offset according to nvfuser properly
Let's split replace_uniform into a separate PR.
Primitives for getting and setting the state can be tested without replace_uniform and having https://github.com/Lightning-AI/lightning-thunder/issues/231 fixed is not required for these primitives.
Hi @mruberry @jjsjann123 @IvanYashchuk , I modified it according our design review discussion, could you take a look?
Yeah, I'll learn to read...
sorry for the trouble, I should get that draft status on