dffml
dffml copied to clipboard
intel
webui: Data Flow creation
DFFML has started it's web UI.
Using something like https://github.com/alyssaxuu/flowy, we need to enable users to visually create dataflows.
This is similar to but instead of creating them via the command line and manually editing yaml/json files the user will use the web UI to specify the config properties for operations and link various operations inputs and outputs.
Notes
- Setting up the webui codebase for development can be found here
- Overview of the webui codebase can be found here
Project Description
AutoML or Automated Machine Learning as the name suggests automates the process of solving problems with Machine Learning. AutoML is generally helpful for people who aren't either familiar with Machine Learning or the involved programming. AutoML aims to improve the efficiency of any task involving Machine Learning.
The primary objective we are trying to achieve is to create a model that takes as a property of its config a set of models to used for hyperparameter tuning. Another property of its config is the set of models which we should attempt to tune (via the first set). Default values for these results in using all installed models to try to tune all installed model plugins.
-
To start, we should define a reduced set of models (not all the ones we have). We'll implement AutoML supporting only this reduced set. The first phase of this project will be to make sure that one model can be used to tune hyperparameters of another model.
-
The next phase will be to tune two models using the same tuning model. This followed by tuning two models, using two models which amounts to doing the previous task twice, with a different tuning model the second time.
-
The following phase will be to go through each model in each model plugin we have and see which ones have issues being tuned using the approach taken in the previous phase. This phase will help us determine which properties or methods we may need to add to models to help them self identify and thereby indicate their requirements for hyperparameter tuning, or maybe their inherent lack of support for it.
-
The final phase will be to implement hyperparameter tuning for N by N models, after implementing what we found to be gaps in the previous phase.
Due to the shortened GSoC cycle, we may end up not doing all of these phases. Which one we go to will be decided as we approach the selection process.
Skills
- JavaScript/TypeScript
- UI design
Difficulty
Intermediate/Hard
Related Readings
- https://github.com/intel/dffml/blob/master/docs/contributing/gsoc/2021.md
- https://scikit-learn.org/stable/model_selection.html#model-selection
- https://www.automl.org/automl/
- https://docs.lyrasearch.io/usage/create-a-new-lyra-instance
Getting Started
- Read the contributing guidelines
- https://intel.github.io/dffml/master/contributing/index.html
- Go through the quickstart
- https://intel.github.io/dffml/master/quickstart/model.html
- Go trough the model tutorials
- https://intel.github.io/dffml/master/tutorials/models/
- Go through the model plugins
- https://intel.github.io/dffml/master/plugins/dffml_model.html
- You don't need to go through all of them. Just get a feel for running a few
Potential Mentors
PerceptiLabs can be a very good inspiration as well.
Tagging: @pdxjohnny @mHash1m @sk-ip @sakshamarora1 @yashlamba
Orchest has very similar objectives like the webUI part of our project. We can take some inspiration from them as well.
Adding another drop to the ocean of your notifications: @pdxjohnny 😅
Another nice UI: https://noisecraft.app/
GitHub Repo: https://github.com/maximecb/noisecraft
Most flow based tools are developer facing. We're talking allowing for arbitrary software to allow users to define flows which solve domain specific problems
We want to do this project because we want to enable embeddability with ability to tailor for use case of the application which depends on the dataflow definition UI components.
We should look at the work we do here as if we are working towards introducing a new input type to HTML.
We're going for <input type="dataflow"> to be so ubiquitous because it useful for applications to allow users to define domain specific flows.
Part of this is the embeddability with ability to tailor for use case too though
- https://github.com/joeltg/react-dataflow-editor#demo