[FR] Allow passing in bool parameters in MLproject entrypoint

[tanmaymathur89]

Proposal

conda_env: my_env.yaml

entry_points:
  main:
    parameters:
      data_file: path
      regularization: {type: float, default: 0.1}
    command: "python script.py -r {regularization} {data_file}"
  validate:
    parameters:
      data_file: path
    command: "python validate.py {data_file}"

Only allows you to pass a string, float, path or uri. However, python allows you to pass bool command line arguments that we cannot fit into this format.

Motivation

Command I want to execute is python script.py --train -r {regularization} {data_file} where --train is an optional boolean argument.

Proposed Changes

conda_env: my_env.yaml

entry_points:
  main:
    parameters:
      data_file: path
      regularization: {type: float, default: 0.1}
      train : {type: bool, default: False}      
    command: "python script.py -r  {regularization} {train} {data_file} "
  validate:
    parameters:
      data_file: path
    command: "python validate.py {data_file}"

MLflow run execution: mlflow run . -P data_file=data.csv train=True translates to python script.py --train -r 0.1 data.csv

[lorenzwalthert]

I proposed this in #1572 and it was rejected for reasons I still don't understand.

[andrewbrooks-o]

Also looking for boolean parameter support.

[juntai-zheng]

Hi @tanmaymathur89, @lorenzwalthert, and @andrewbrooks-o! Thanks for your feedback - this seems like something that would be nice to support. Would any of you be willing to make a PR to add in this capability?

[tanmaymathur89]

@juntai-zheng I can create a PR for it. Thanks

[dbczumar]

@tanmaymathur89 @lorenzwalthert @andrewbrooks-o I think that adding boolean parameter support sounds great. #2453 currently introduces support for an action parameter, which appears to be relatively Python or bash-specific; @tanmaymathur89 can we refactor this to introduce support for True/False boolean parameters rather than actions?

[sn1c]

@dbczumar is this still open? If @tanmaymathur89 won't do it, I can prepare a PR for adding boolean parameter support.

[tanmaymathur89]

@dbczumar is this still open? If @tanmaymathur89 won't do it, I can prepare a PR for adding boolean parameter support.

Go ahead @sn1c. I thought I posted a comment earlier but can't see it anymore. I was wondering what the advantage is converting from action to boolean? We are essentially passing through the values to a downstream python code in this case.

[tanmaymathur89]

@dbczumar is this still open? If @tanmaymathur89 won't do it, I can prepare a PR for adding boolean parameter support.

Go ahead @sn1c. I thought I posted a comment earlier but can't see it anymore. I was wondering what the advantage is converting from action to boolean? We are essentially passing through the values to a downstream python code in this case.

Oh found it, https://github.com/mlflow/mlflow/pull/2453#issuecomment-602315646

[sn1c]

@tanmaymathur89 I feel like the reason for the stand for booleans and against actions is, that the developers seem to want MLflow to be as language-agnostic as possible. What is your take on that?

[tanmaymathur89]

@tanmaymathur89 I feel like the reason for the stand for booleans and against actions is, that the developers seem to want MLflow to be as language-agnostic as possible. What is your take on that?

IMO its just a pass through. Adding additional grammar is confusing but thats my opinion.

[EKami]

So, what's the workaround to work with that? I see no reference to boolean types in the MLFlow doc

[lorenzwalthert]

Well, a simple workaround is to use an integer parameter (and supply 0, 1 only) and convert to boolean in your script.

[EKami]

Ok, so here is my workaround:

# MLProject
name: my_project

entry_points:
  main:
    parameters:
      my_bool: {type: string, default: "no_my_bool"}
    command: python my_module.py --{my_bool}

In the calling code:

# main.py
mlflow.run(".", entrypoint, parameters={"my_bool": "with_my_bool" if a_boolean else "no_my_bool"})

And finally in the module

@click.command(help="Demo")
@click.option('--with_my_bool/--no_my_bool')
def main(my_bool):
    # Do something with my_bool

if __name__ == "__main__":
    main()

Notice you are forced to use a default in MLProject, use the False value by default which is equivalent to not specifying the flag.

[lorenzwalthert]

I am not sure I’d do it that way. Of course you can pass flags, but mostly you want to use key value pairs and not just flags (not sure what your use case is). Like early_stopping=True. This also makes more sense in the context of filtering runs I believe and is consistent with string and numeric parameters. So I'd just do early_stoping=1 as any other numeric parameter and then convert to bool in your script (not sure, maybe click supports that out of the box when parsing the parameter).

[EKami]

Both are valid I think, it's just a matter of taste. To me when the module is run separately as an independent script I find it cleaner to specify --early_stopping instead of --early_stoping=1

[florianblume]

Any update on the boolean parameter support? I'm in need of such a feature. If I can help somehow I'd be happy to.

[anton164]

How is this still not supported? Big gap in MLFlow.

[chenmoneygithub]

This issue has been put under a low priority, but I think we can do a quick fix for it. Keep it open and we will welcome OSS contributions.