ipex-llm
ipex-llm copied to clipboard
fix DBFS file path problem on Dataricks
Description
when call
orca_estimator = Estimator.from_torch(model=model_creator,
optimizer=optimizer_creator,
loss=criterion,
metrics=[Accuracy()],
model_dir=model_dir',
use_tqdm=True,
backend=backend)
on Databricks cluster, model_dir should be a shared path to save files, which is a path starts with /dbfs or dbfs:/ on Databricks. but currently, the DBFS format path is not support yet, this PR fix this problem. This is verified on Databricks.