ipex-llm
ipex-llm copied to clipboard
[Orca] Support saving/loading trained TF model with h5/saveModel format
Description
1. Why the change?
- Current
save
andload
do not supporth5
andsavemodel
format, since they mainly retrieve model states. - Users need the API to save model as
h5
orsavemodel
format.
2. User API changes
For new added API:
- save model
estimator.save(filepath, overwrite, include_optimizer, save_format, signatures, options)
- load model
estimator.load(filepath, custom_objects, compile)
For original save/load API:
- save -> save_checkpoint
estimator.save_checkpoint(checkpoint)
- load -> load_checkpoint
estimator.load_checkpoint(checkpoint)
3. Summary of the change
Provide new save
and load
API to support h5
and savemodel
format (both load and hdfs).
4. How to test?
- [ ] Unit test
- [x] Test saving/loading on HDFS
- [x] Test on Yarn