Spyder outline

[Alecampoy]

Hello

Firstly, I find this plugin super usefull, I want to thank the developers.

My issue: I have several .py scripts written in Spyder. In Spyder I make lots of use of the outline, so every cell have a title next to #%%. This way I can sort my code in Spyder and access the different sections. An example:

What I get when using Jupytext are just code cells as shown in the following picture. Is there a way of making use of the titles of the Spyder cells following to #%% and have them as markdown titles in the notebook? the only way is having cells defined with #%% [markdown] in order to have them in ipynb cells?

Jupytext output:

[Alecampoy]

I've been playing some time with jupytext in order to try to convert spyder scripts, trying to convert the name of the cells in spyder (as in outline) to a markdown cell, but quite unsuccessfully. Any idea could be super helpfull

[mwouts]

Hello @Alecampoy , the title of the cell is still in the notebook, but it's located in the cell metadata. For instance:

$ echo "# %% Spyder title" | jupytext --to ipynb
{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "b1b343e8",
   "metadata": {
    "title": "Spyder title"
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "jupytext": {
   "cell_metadata_filter": "title,-all",
   "main_language": "python",
   "notebook_metadata_filter": "-all"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}

Of course it would be doable to move this metadata to a new markdown cell, but this would have to take place in a Python script on your end, not in Jupytext. The script would be based on nbformat, and I can guide you through this if you like the idea (please confirm).

[mwouts]

A sample script that can do the desired conversion is the one below. Please note that unlike Jupytext, this is a one way converter. In other words, if you apply this to a Spyder notebook, you will probably lose the outline in Spyder (but this time you'll get it in Jupyter)

import jupytext
from nbformat.v4.nbbase import new_markdown_cell, new_notebook
from jupytext.compare import compare


def export_spyder_titles_to_markdown_cells_in_notebook(nb):
    def insert_markdown_cells(nb):
        for cell in nb.cells:
            if cell.cell_type == "code" and "title" in cell.metadata:
                # Warning: this modifies the original notebook cell metadata!
                yield new_markdown_cell(cell.metadata.pop("title"))
            yield cell

    return new_notebook(metadata=nb.metadata, cells=list(insert_markdown_cells(nb)))


def export_spyder_titles_to_markdown_cells_in_text_notebook(text):
    """Moves the Spyder cell titles to Markdown cells in a text notebook.

    Hint: use read/write to handle files rather than text"""
    nb = jupytext.reads(text, fmt="py:percent")
    nb = export_spyder_titles_to_markdown_cells_in_notebook(nb)
    return jupytext.writes(nb, fmt="py:percent")


def test_export_spyder_titles_to_markdown_cells_in_text_notebook(
    input_py="""# %% Spyder title
1 + 1
""",
    expected_output_py="""# %% [markdown]
# Spyder title

# %%
1 + 1
""",
):
    actual_output_py = export_spyder_titles_to_markdown_cells_in_text_notebook(input_py)
    compare(actual_output_py, expected_output_py)

[Alecampoy]

Hi mwouts

Thank you very much¡ As soon as I have some time I'm going to try it (since I have no clue of nbformat and probably must check it). If I manage it is going to be really helpfull.

Best wishes

[Alecampoy]

Hi back there¡ I've been trying to make your script to work, but I couldn't make it. feeling desperate after struggling By just simply applying the function the function you created I got the following

export_spyder_titles_to_markdown_cells_in_text_notebook("test.ipynb") Out[107]: '# %%\ntest.ipynb\n'

export_spyder_titles_to_markdown_cells_in_text_notebook("test.py") Out[108]: '# %%\ntest.py\n'

test.py is my spyder notebook with the outline as explained, and test.ipynb is the bu default converted buu jupytext notebook. none works.

By trying to do some debbuging, I found that nb = jupytext.reads(text, fmt="py:percent") is not really reading anything, but if I change it to jupytext.read, then jupytext.write is not working. But really I do not know what I am doing here.

Note: I am running your functions in spyder under windows if that's critical.

[Alecampoy]

Made it work, thank you :)

[mwouts]

Yes, jupytext.reads(text, fmt="py:percent") assumes that text is the content of a text notebook, while jupytext.read(file, fmt="py:percent") treats file as either a file name or an open stream.

In the test above I didn't use files and passed directly the sample notebook, but yes certainly if you want to act on files you should use read and write directly - glad that you could make it in the end!