Could RL be applied in BigDL?

[fatenlouati]

[shane-huang]

Could you elaborate what you want to do? Are you trying to use BigDL to run RL applications, for example, use BigDL to (distributed) train RL applications, etc? What kind of RL applications are you running?

[fatenlouati]

@shane-huang yes exactly, How to use BigDL to train RL algorithms such us Deep Q learning, actor critic etc ?

[shane-huang]

Can I know what's your expectation of using BigDL for RL algorithms training? For example, are you trying to accelerate your RL training on single-node (e.g. laptop or workstation)? Or are you planning to train RL algorithms on a cluster? Or your simulation/env generates data on a Hadoop/Spark cluster in distributed data format, say, Spark DataFrame and you want to train RL algorithms on those data? @ToutaF

[shane-huang]

Can I know what's your expectation of using BigDL for RL algorithms training? For example, are you trying to accelerate your RL training on single-node (e.g. laptop or workstation)? Or are you planning to train RL algorithms on a cluster? Or your simulation/env generates data on a Hadoop/Spark cluster in distributed data format, say, Spark DataFrame and you want to train RL algorithms on those data? @ToutaF

For different scenarios we may recommend different toolsets for you. For example,

• if you want to use BigDL to simply accelerate your single-node training, you can use BigDL-Nano to do it. Refer to more details in BigDL-Nano documents.
• if you want to scale your local training to distributed cluster, you may use BigDL-Orca. Refer to more details in BigDL-Orca document.
• if you want to develop RL algorithms from scratch, you can also use BigDL-DLlib, which provides a Keras-like API. The developed RL model can be used to run on Spark cluster and process Spark dataframe input as well. Refer to more details in BigDL-DLlib documents.
• It is also possible to use built-in RL algorithms in Ray Rllib with BigDL. For more information, you can refer to ray related documents quickstart, and overview

[fatenlouati]

@shane-huang Thank you for your reply The 3rd scenario is mine. I want to write RL algorithm from scratch then train and test it on cluster spark.

[shane-huang]

@shane-huang Thank you for your reply The 3rd scenario is mine. I want to write RL algorithm from scratch then train and test it on cluster spark.

@ToutaF If you intend to train and test on Spark cluster, there're actually more than one ways to use BigDL.

1. You can start by writing standard pytorch or tensorflow programs. After you test it with small dataset on single node, you can use BigDL-Orca to run and test it on spark Cluster and train/inference on cluster data. You can get an idea of how to do it from a 2-mins reading at this page. For more information you may continue your reading about distributed data processing, distributed training/inference.

2. You can also use Bigdl-DLlib for development. Note that dllib provides a Keras-like API which is compatible to Keras 1.2.2. If you want to use other versions of Tensorflow/Keras or PyTorch, using BigDL-Orca may be a better idea. Anyway, to try dllib, you can follow the dllib python user guide to install dllib and get started. Also refer to API docs as needed.

[fatenlouati]

@shane-huang thank you so much for your reply