Nano: add ipex and jit options to `bigdl.nano.pytorch.Trainer.trace`

[TheaperDeng]

Motivation

Since we find for some models(e.g. Autoformer in Chronos), JIT and ipex brings some improvement on inference latency while Onnxruntime and OpenVINO can not work properly (e.g. lack of operators).

API design

There is no special design in this issue, we are working on a really similar API as onnxruntime and openvino.

# ipex
model_ipex = Trainer.trace(model, accelerator=None, use_ipex=True...)
model_ipex(*args)

# jit
model_jit = Trainer.trace(model, accelerator="jit", input_sample=input_sample, use_ipex=False...)
model_jit(*args)

# jit + ipex
model_jit_ipex = Trainer.trace(model, accelerator="jit", input_sample=input_sample, use_ipex=True...)
model_jit_ipex(*args)

Working on a PR to implement this design.

[rnwang04]

# ipex
model_ipex = Trainer.trace(model, accelerator=None, use_ipex=True, ...)
model_ipex(*args)

# jit
model_jit = Trainer.trace(model, accelerator="jit", input_sample=input_sample, ...)
model_jit(*args)

# jit + ipex
model_jit_ipex = Trainer.trace(model, accelerator="jit", input_sample=input_sample, use_ipex=True, ...)
model_jit_ipex(*args)

Maybe this is more closer to the parameter of Trainer ？

[TheaperDeng]

# ipex
model_ipex = Trainer.trace(model, accelerator=None, use_ipex=True, ...)
model_ipex(*args)

# jit
model_jit = Trainer.trace(model, accelerator="jit", input_sample=input_sample, ...)
model_jit(*args)

# jit + ipex
model_jit_ipex = Trainer.trace(model, accelerator="jit", input_sample=input_sample, use_ipex=True, ...)
model_jit_ipex(*args)

Maybe this is more closer to the parameter of Trainer ？

yes, I have changed to this design, which is more consistent.