ipex-llm icon indicating copy to clipboard operation
ipex-llm copied to clipboard

Orca: Add 2 NCF PyTorch examples with data_loader or XShards as inputs.

Open zpeng1898 opened this issue 2 years ago • 3 comments

Add NCF pytorch examples : train_data_loader.py and train_xshards.py to the NCF directory, with a shared NCF-model model.py.

1.The train_data_loader.py example takes data_loader as the input of the model, supporting fitting the estimator with ray or spark backend:

# create the estimator est = Estimator.from_torch(model=model_creator, optimizer=optimizer_creator,loss=loss_function, metrics=[Accuracy()],backend=Config["backend"])# backend="ray" or "spark" # fit the estimator est.fit(data=train_loader_func, epochs=1)

2.The train_xshards.py example takes XShards as the input of the model, supporting fitting the estimator with ray or spark backend:

# create the estimator est = Estimator.from_torch(model=model_creator, optimizer=optimizer_creator,loss=loss_function, metrics=[Accuracy()],backend=Config["backend"])# backend="ray" or "spark" # fit the estimator est.fit(data=train_shards, epochs=1,batch_size=Config["batch_size"],feature_cols=["x"],label_cols =["y"])

zpeng1898 avatar Sep 08 '22 10:09 zpeng1898

https://github.com/intel-analytics/BigDL/pull/5738 can remove model_dir after this PR merges.

hkvision avatar Sep 13 '22 13:09 hkvision

Can we merge three train_*.py files?

lalalapotter avatar Sep 19 '22 03:09 lalalapotter

Can we merge three train_*.py files?

To demonstrate different inputs, clearer to use separate scripts.

hkvision avatar Sep 19 '22 03:09 hkvision