pipcook
pipcook copied to clipboard
meta: 2.0 roadmap
We decide to launch the 2.0 roadmap here. At 2.0, the main improvements are to:
- focus on TVM-based WebAssembly model generation which includes:
- compiling the Python-based model into WebAssembly formats.
- replacing the data processing with imgcook/datacook which is expected to be working on browser and Node.js.
- (we are trying to make datacook to be a Web standard for processing data).
- refactor the pipeline builder as:
- prebuild plugins on the new plugin registry, every plugin should be a single file.
- simplify the local build of a pipeline, now it's just downloading the plugin file.
- downgrade to the local build if the prebuilt plugins are unavailable.
- create some tutorials at Pipboard where the user could quickly learn data, model, and serving inside one page, it's inspired by https://teachablemachine.withgoogle.com/.
- upgrade Pipeline to 2.0 schema to be more extensible and powerful
- support for multiple nodes for the same type of plugin.
- better support for GAN & RL.
Besides we'd launch a new product named Pipcook Cloud at 2.0, which makes the developer could deploy Pipcook as clusters based on some cloud infra just like: Kubernetes, Flink/Blink, and MaxCompute. The Pipcook Cloud is to support the following features:
- big data(batch/stream) processing.
- distributed training.
- human-in-the-loop ML/DL pipeline.
- recommendation system in your hands.
From a technical perspective, we will try to migrate the API framework to loopback4, and unified the test framework to ava, both help us to reduce the unnecessary focus on the low-level dependencies.
In the end, we are willing to build a local tracker that collects some useful data and saves(not uploading), then we are able to request these collected data to be uploaded for OSS analytics to improve UX.