{code-cell} pygments_lexer purpose confusing and/or broken.

[mforbes]

Describe the bug

context

When writing a code cell, there is an optional value (don't know what to properly call it) ipython3 here

```{code-cell} ipython3
print("Hello!")
```

Examples in the documentation oscillate between having this and excluding it. This can become very prominent when syncing files with Jupytext because it it is converted to notebook.metadata.language_info.pygments_lexer, and often a source of VCS noise, with this being the only change.

expectation

After finding out about this jupytext conversion, I expected that I could alter the pygment lexing of code-cells as described for myst-parser: basic code blocks.

bug

Instead, the choice of pygments lexer seems to be fixed for all cells by the kernel as of myst-nb 18.3 (39c1bb9).

problem

This is a problem because this optional value (ipython3 above) can be quite visible, but is poorly documented and confusing. Some people may have a use-case where they want to customize the lexer and expect this behaviour based on the Jupytext conversion.

In my case, I just want to know what the purpose of this is, and if I should include or exclude it - especially for future proofing.

Here are the documents I have found discussing this value:

• jupyterbook: Provides an example, including the ipython3 and discusses the syntax highlighting, but makes no mention of the meaning of ipython3 or its relationship (or lack thereof) to syntax highlighting. I think this example should be updated to explain what the ipython3 means, as well as how the syntax highlighting is actually chosen (through the kernelspec I think).

• jupyterbook:

  You may also explicitly pass the kernel name after {code-cell} to make it clear which kernel you are running.... However, remember that there is only one kernel allowed per page.

  This is the clearest statement I could find. If one kernel per page is true, then there should be no issue with the pygments_lexer interpretation. However, there are issues considering multiple kernels, e.g. thebe #343), and this statement does imply that maybe this value will be used sometime in the future - and hence should be kept.

• myst-nb:

  The argument after {code-cell} (above, ipython3) is optional, and is used for readability purposes.

  This is ambiguous.

• jupytext:

  The ipython3 here is purely optional, as an aide for syntax highlighting. In the round-trip, it is copied from notebook.metadata.language_info.pygments_lexer.

  This is what Jupytext does. My impression is that jupytext is fairly critical to the usage of MyST-NB. If so, this behaviour should be supported, or at lease clearly documented.

Reproduce the bug

Typeset the following code or similar with Jupyter Book:

---
jupytext:
  formats: ipynb,md:myst
  text_representation:
    extension: .md
    format_name: myst
    format_version: 0.13
    jupytext_version: 1.14.4
kernelspec:
  display_name: Python 3 (python3)
  language: python
  name: python3
---

# Test

```{code-cell} ipython3
# This is formated as python code
x = 1
```

```{code-cell} latex
# This is also formatted as python code
x = 1
```

```latex
% This is formated as LaTeX code
# Not as python code (this line is not a latex comment)
x = 1
```

List your environment

$ jupyter-book --version
Jupyter Book      : 0.15.0
External ToC      : 0.3.1
MyST-Parser       : 0.18.1
MyST-NB           : 0.17.1
Sphinx Book Theme : 1.0.0
Jupyter-Cache     : 0.5.0
NbClient          : 0.5.13