Justin Zhao

Black pre-commit and black command line disagree

3

## Version information pre-commit version: 2.19.0 ```bash > pre-commit --version pre-commit 2.19.0 ``` pre-commit `black` version: 22.3.0 (https://github.com/ludwig-ai/ludwig/blob/master/.pre-commit-config.yaml) ```yaml - repo: https://github.com/psf/black rev: 22.3.0 hooks: - id: black name: black...

Add in-training tooling to find a more optimal threshold for binary classification.

2

Ludwig uses a default threshold of 0.5 to calculate accuracy for binary classification problems. However, it's highly possible, especially for imbalanced datasets that a threshold of 0.5 is [not the...

Add a CLI for Ludwig AutoML

1

One idea, mirroring `ludwig.automl.auto_train()` ``` ludwig automl --dataset= --target= --enable_hyperopt= # whether to run hyperopt, default False. --time_limit_s= # How long to run hyperopt for. If specified, hyperopt is enabled....

The current "expanded" Ludwig config (saved as `model_hyperparameters.json`) contains user config-specified parameters merged with preprocesing and metadata parameters. However, there aren't any module-specific parameters (unless they were explicitly specified by...

Model-agnostic, predict-only parameters like `top_k` and `threshold` are "free" parameters since they have no bearing on how the model is architected or how the model is trained. However, they are...

While a precise time to completion would be difficult to estimate as convergence depends on a mix of non-deterministic factors like early stopping criteria, there’s likely a reasonably good "max...