ipex-llm
ipex-llm copied to clipboard
Orca: Add 2 NCF PyTorch examples with data_loader or XShards as inputs.
Add NCF pytorch examples : train_data_loader.py
and train_xshards.py
to the NCF
directory, with a shared NCF-model model.py
.
1.The train_data_loader.py
example takes data_loader
as the input of the model, supporting fitting the estimator with ray
or spark
backend:
# create the estimator
est = Estimator.from_torch(model=model_creator, optimizer=optimizer_creator,loss=loss_function, metrics=[Accuracy()],backend=Config["backend"])# backend="ray" or "spark"
# fit the estimator
est.fit(data=train_loader_func, epochs=1)
2.The train_xshards.py
example takes XShards
as the input of the model, supporting fitting the estimator with ray
or spark
backend:
# create the estimator
est = Estimator.from_torch(model=model_creator, optimizer=optimizer_creator,loss=loss_function, metrics=[Accuracy()],backend=Config["backend"])# backend="ray" or "spark"
# fit the estimator
est.fit(data=train_shards, epochs=1,batch_size=Config["batch_size"],feature_cols=["x"],label_cols =["y"])
https://github.com/intel-analytics/BigDL/pull/5738 can remove model_dir after this PR merges.
Can we merge three train_*.py files?
Can we merge three train_*.py files?
To demonstrate different inputs, clearer to use separate scripts.