jeandut

Is it possible to input ProxyCommands in settings.json ?

1

### Description My issue is more a feature request. I would like to be able to edit files with vscode on a distant machine that can only be accessed by...

One could add checks on the fact that the sent opener is indeed an instance of a `substratools opener` and optionally if fake data can be created, saved and loaded.

[Feature Request] Passing list of data hashes instead of sets to traintuples

3

One use-case that is not supported as of today in Substra would be to easily use a **static algo.py** to do either only one training epoch over a dataset or...

[Feature Request] Logging time per run for a solver

1

Currently the logging information does not display the time a run of the solver took (until max-runs or convergence). This is not entirely straightforward to log precisely because a solver...

[RFC] Passing hyperparameters to solvers (but also for i.e. dataset)

1

When running with the CLI a solver with a specific set of hyperparameters (say a learning rate or the l1_ratio.) Currently BenchOpt syntax for the CLI is a bit heavy...

[Feature Request] Adding subfolder keyword arg to `import_from` function to allow import from other subfolders than utils

4

[User story] One wants to factorize the body of its solvers (resp. datasets) using class inheritance as they all share the same structure and one wants to avoid code duplications....

It seems like as of now BenchOpt can display more than 53 solvers' curves on the same plot, but only 53 appear in the list of solvers used below the...

[Feature Request] Adding Confidence Intervals (Quantiles) for y (objective_value) (as is currently done for durations)

2

When running a DL solver, experiments are often heavily dependent on the seed (especially for RL), therefore it is common practice to average runs across repetitions as is done currently...

[RFC] How to deal with outliers when plotting convergence curves

4

Sometimes for some hyper-parameters solvers diverge (i.e. too large learning rates). Even when catching the divergence in custom StoppingCriterion logic and stopping the solver, the first value might still be...

For a new user it's not entirely clear what would be the recommended way to seed an experiment/benchmark to perform multiple repetitions if the solver is not deterministic (different initializations...