Parameterized kernel specs proposal

[hbcarlos]

Parameterized kernel specs

In this JEP we propose to parameterize the kernel specs, simplifying the way some kernels are installed reducing the amount of kernel specs files, and at the same time improving the UI of some of the Jupyter front-ends.

Co-author: @SylvainCorlay

Contributors that may be interested in this topic from past conversations:

• @ellisonbg
• @jasongrout
• @rgbkrk

[dhirschfeld]

I imagine @kevin-bates would be interested in this too.

[blink1073]

@echarles has been exploring similar ideas in https://github.com/deshaw/ksmm

[echarles]

There also is a issue opened in jupyter-server team-compass https://github.com/jupyter-server/team-compass/issues/16 to give connect to people interested in this feature.

So basically, you are adding the following stanza to the kernelspec?

  "metadata": {
      "parameters": {
        "cpp_version": {
          "type": "string",
          "default": 'C++14',
          "enum": ['C++11', 'C++14', 'C++17'],
          "save": true
        }
      }
    },
}

[echarles]

... and that stanza is a json-schema.

For KSMM, we give the option to define env var (see https://raw.githubusercontent.com/deshaw/ksmm/main/screenshots/general_settings_ss.png). I read you say ...lists (respectively the command-line arguments and environment variables)... but I don't see example for such env var definitions.

[kevin-bates]

Thanks for the ping @dhirschfeld and reference to the team-compass issue @echarles.

@hbcarlos - this is great - thank you for opening this issue!

I'm really hoping this proposal can be expanded a bit to also include provisioner_parameters (and perhaps rename the parameters stanza to kernel_parameters) since this proposal appears to focus only on kernel-based parameters. For reference, here's the originally proposed JEP that I closed due to dormancy: https://github.com/kevin-bates/enhancement-proposals/blob/4c0727fbe03653aba789de67c388e2d28cda4cb9/parameterized-launch/parameterized-launch.md. It includes both schemas.

I would love to participate in any way possible. Thank you for opening this.

[SylvainCorlay]

For KSMM, we give the option to define env var (see https://raw.githubusercontent.com/deshaw/ksmm/main/screenshots/general_settings_ss.png). I read you say ...lists (respectively the command-line arguments and environment variables)... but I don't see example for such env var definitions.

Exactly, the env list of the kernelspec can also have reference to parameters as said in the proposal, although the example that we included does not make use of it.

E.g.

{
  "display_name": "C++",
  "argv": [
      "/home/user/micromamba/envs/kernel_spec/bin/xcpp",
      "-f",
      "{connection_file}",
      "-std={parameters.cpp_version}"
  ],
  env: [
    "XEUS_LOGLEVEL={parameters.xeus_log_level}"
  ],
  "language": "C++"
  "metadata": {
      "parameters": {
        "cpp_version": {
          "type": "string",
          "default": "C++14",
          "enum": ["C++11", "C++14", "C++17"],
          "save": true
        },
        "xeus_log_level": {
          "type": "string",
          "default": "ERROR",
          "enum": ["TRACE", "DEBUG", "INFO", "WARN", "ERROR", "FATAL"],
          "save": true
        }
      }
    },
}

[hbcarlos]

I would love to participate in any way possible. Thank you for opening this.

Thanks, @kevin-bates! Help is always welcome!

I'm really hoping this proposal can be expanded a bit to also include provisioner_parameters (and perhaps rename the parameters stanza to kernel_parameters) since this proposal appears to focus only on kernel-based parameters.

This proposal allows specifying parameters independently of how they may be used (as environment variables, command-line arguments). What we are trying to achieve on this JEP, is to include and formalize the concept of parameters in the kernel specs but not the semantics, so that developers can use these parameters for any purpose (provisioner parameters, environment variables, etc.). I'll update the proposal to make it more clear.