pipcook icon indicating copy to clipboard operation
pipcook copied to clipboard

meta: 2.0 roadmap

Open yorkie opened this issue 4 years ago • 0 comments

We decide to launch the 2.0 roadmap here. At 2.0, the main improvements are to:

  • focus on TVM-based WebAssembly model generation which includes:
    • compiling the Python-based model into WebAssembly formats.
    • replacing the data processing with imgcook/datacook which is expected to be working on browser and Node.js.
      • (we are trying to make datacook to be a Web standard for processing data).
  • refactor the pipeline builder as:
    • prebuild plugins on the new plugin registry, every plugin should be a single file.
    • simplify the local build of a pipeline, now it's just downloading the plugin file.
    • downgrade to the local build if the prebuilt plugins are unavailable.
  • create some tutorials at Pipboard where the user could quickly learn data, model, and serving inside one page, it's inspired by https://teachablemachine.withgoogle.com/.
  • upgrade Pipeline to 2.0 schema to be more extensible and powerful
    • support for multiple nodes for the same type of plugin.
    • better support for GAN & RL.

Besides we'd launch a new product named Pipcook Cloud at 2.0, which makes the developer could deploy Pipcook as clusters based on some cloud infra just like: Kubernetes, Flink/Blink, and MaxCompute. The Pipcook Cloud is to support the following features:

  • big data(batch/stream) processing.
  • distributed training.
  • human-in-the-loop ML/DL pipeline.
  • recommendation system in your hands.

From a technical perspective, we will try to migrate the API framework to loopback4, and unified the test framework to ava, both help us to reduce the unnecessary focus on the low-level dependencies.

In the end, we are willing to build a local tracker that collects some useful data and saves(not uploading), then we are able to request these collected data to be uploaded for OSS analytics to improve UX.

yorkie avatar Dec 10 '20 04:12 yorkie