ipex-llm icon indicating copy to clipboard operation
ipex-llm copied to clipboard

[Orca] Support saving/loading trained TF model with h5/saveModel format

Open sgwhat opened this issue 2 years ago • 0 comments

Description

1. Why the change?

  • Current save and load do not support h5 and savemodel format, since they mainly retrieve model states.
  • Users need the API to save model as h5 or savemodel format.

2. User API changes

For new added API:

  • save model
    estimator.save(filepath, overwrite, include_optimizer, save_format, signatures, options)
    
  • load model
    estimator.load(filepath, custom_objects, compile)
    

For original save/load API:

  • save -> save_checkpoint
    estimator.save_checkpoint(checkpoint)
    
  • load -> load_checkpoint
    estimator.load_checkpoint(checkpoint)
    

3. Summary of the change

Provide new save and load API to support h5 and savemodel format (both load and hdfs).

4. How to test?

  • [ ] Unit test
  • [x] Test saving/loading on HDFS
  • [x] Test on Yarn

sgwhat avatar Aug 03 '22 02:08 sgwhat