ray
ray copied to clipboard
[RLlib; Offline RL] Store episodes in state form.
Why are these changes needed?
Storing with ray.data
episodes as instances results in pickled instances that are maybe not compatible with later python versions. This PR tries to develop a stream that is compatible with later Python versions by simplifying objects in their get_state
methods and tries to reproduce them in their static from_state
method.
It uses mgspack
to serialize objects (and msgpack-numpy
to serialize numpy array) which is a serialization protocol independent of Python versions.
Related issue number
Checks
- [x] I've signed off every commit(by using the -s flag, i.e.,
git commit -s
) in this PR. - [x] I've run
scripts/format.sh
to lint the changes in this PR. - [x] I've included any doc changes needed for https://docs.ray.io/en/master/.
- [ ] I've added any new APIs to the API Reference. For example, if I added a
method in Tune, I've added it in
doc/source/tune/api/
under the corresponding.rst
file.
- [ ] I've added any new APIs to the API Reference. For example, if I added a
method in Tune, I've added it in
- [x] I've made sure the tests are passing. Note that there might be a few flaky tests, see the recent failures at https://flakey-tests.ray.io/
- Testing Strategy
- [x] Unit tests
- [x] Release tests
- [ ] This PR is not tested :(