lazy cells (re-work of disabled cells)

[ggggggggg]

Description

I would like the "reactive execution"/"disabled" cell status be more clear and useful, by unifying it with "lazy execution". Currently there are multiple forms of lazyness that are presented with different names, different UIs, and different feature sets. In particular "reactive execution" lacks features that "lazy" has. Here I propose that these be combined into one feature, called "lazy".

Suggested solution

Unify on term, I suggest "lazy", with one UI, and one feature set. I suggest the UI and feature set that "lazy" currently has.

Alternative

1. Leave it unchanged, I believe the current state is a local minimum of usefulness and clarity, so this isn't a great solution.
2. Pick a different term to unify on, "reactive execution" is fine.

Additional context

In the settings menu you can set the notebook to "autorun" or "lazy". When in lazy mode, dependent cells of a changed cell are marked "stale" and require user interaction to run. For each cell we can choose the "reactive execution" state. In this case the changed cell is unable to be run.

The user actually can run it, by enabling "reactive execution", running it, then again disabling "reactive execution". It is more ceremony, but fundamentally the same functionality as lazy. I think "reactive execution" should be replaced by a cell by cell version of "lazy". This will provide the following benefits:

1. A single term the same functionality. I found the use of different terms misleading, as it suggested these features were fundamentally very different. So I made guesses about how they worked that were wrong. We can prevent this mistake for future users.
2. More functionality with based on cell by cell toggle. Allowing opt in cell by cell lazy makes it easy to break a notebook into chunks to separate out slow or annoying parts while working on an earlier part. There is a "run button" which is documented to provide a similar functionality, so it's clear this feature is useful.

[akshayka]

I always like having as few concepts as possible. In this case I view disabling cells as different than making them as lazy — disabled cells and their descendants never run, whereas a proposed "lazy" cell could still be executable.

The advantage of disabled functioning as it does is that you can still iterate on the rest of your notebook, even using the run all stale cells button, while having the confidence that your disabled cells won't run, not under any circumstance. For example, you may want to disable a cell from running if it writes to a production database or has some other side-effect.

Does this make sense? If so, do you have any suggestions on how we can make this behavior clearer?

[ggggggggg]

I see, so there is another use case. I was unaware of the run all stale cells button. Perhaps a 3 way toggle then.

1. Reactive
2. Lazy (or Manual)
3. Protected

For the third case if it's really meant to isolate critical parts like writing to a production database, I'd expect a stronger and more explicit UI element that expresses intention. Like you have to click on a button that says "warning: will write to production database on click". Maybe it's a "protected cell" and you have to type "yes I mean it" to get it to run. Or maybe it's built with the other UI components available, like a toggle switch on the cell to enable/disable one code path that writes to production.

Also, where is "run all stale cells?" I don't see it even when I switch to lazy execution, nor can I find it in the command pallet. Do you mean the "run all modified cells" button?

[akshayka]

Perhaps a 3 way toggle then ... For the third case if it's really meant to isolate critical parts like writing to a production database, I'd expect a stronger and more explicit UI element that expresses intention.

That's interesting. "Protected" would be a more relaxed form of disabled. I will consider, but I wonder if it's more confusing having even more options? ie a 3-way toggle vs two. That said I know what we have today confused you, so it seems there's ways for us to improve.

Also, where is "run all stale cells?" I don't see it even when I switch to lazy execution, nor can I find it in the command pallet. Do you mean the "run all modified cells" button?

Yes, that's what I meant. Sorry for misspeaking; perhaps we should update the tooltip to be "run all stale cells". It's the button in the bottom right of the notebook that appears when at least one cell is stale (due to lazy execution or due to modification).

[ggggggggg]

That's interesting. "Protected" would be a more relaxed form of disabled.

My intention of suggesting a name like "protected" isn't to be more relaxed, it is to more clearly express intention. Compare the suggested user interaction required to run a cell, and leave it in a non-reactive state. Protected: select protected box, type a short phrase and press a button Disabled: cell menu, enable reactive execution, ctrl-enter, cell menu, disable reactive execution

I don't see one of those as significantly more difficult or relaxed than the other. Instead I prefer "protected" because it would

1. naturally leave the cell in the same state after a single execution
2. Answer's a users potential question "why is this cell different from normal lazy cell?"

I will consider, but I wonder if it's more confusing having even more options? ie a 3-way toggle vs two. That said I know what we have today confused you, so it seems there's ways for us to improve.

There are already 3 options for how cells behave. Reactive, lazy, and disabled. I am suggesting unifying them into one UI to make their relationship more clear. I'm not tied to a 3-way toggle, but it seems pretty natural for a 3 option system.

Yes, that's what I meant. Sorry for misspeaking; perhaps we should update the tooltip to be "run all stale cells". It's the button in the bottom right of the notebook that appears when at least one cell is stale (due to lazy execution or due to modification).

I'm experimenting with running the whole notebook in lazy mode so I now see this button, soon I will have more insight into the advantages of the current design.

[akshayka]

Answer's a users potential question "why is this cell different from normal lazy cell?"

This is a good point, thanks for emphasizing it.

Protected: select protected box, type a short phrase and press a button Disabled: cell menu, enable reactive execution, ctrl-enter, cell menu, disable reactive execution

When a cell is disabled, the user can edit a notebook and add it to the graph without running it — the run button is replaced with an "add to graph" button when a cell is disabled. This allows the user to persist changes to the graph without running the cell, and is a functionality that some of our users requested when we designed this feature. When the cell is re-enabled, if it has been made stale by the execution of other cells, it will automatically run (if "on cell change" is set to autorun) or marked as stale (if "on cell change" is set to "lazy").

If we decided to replace "disabled" with "protected", we might want to to allow both running (with confirmation) and adding to the graph without running. This could perhaps be done by adding another button, or by a click on a cell's "run" button giving two options — actually run the cell, or add to the graph.

Thanks for the feedback, I will continue mulling it over and please feel free to share additional thoughts as they come.

[leventov]

Graying out the code of disabled cells seems unnecessarily penalising (why i should strain my eyes editing such uncontrastive text?) - the No symbol on the right is prominent enough and sufficient imo. Though, even it has some negative connotation - I'd replace it with pause sign ⏸️ perhaps.

[leventov]

Ok, some extra tinkering with the design in Claude led me to the following design:

These circled buttons are where there "action bar" currently on the right of the cell on mouse-over (please let me know how that UI area is called in marimo's dev parlance).

Currently, the run button (and stop execution) button appears only on mouseover, which makes sense to keep, to reduce unnecessary visual clutter. But I think it makes sense to always show other buttons and status signs (timer and lock), regardless of the mouse over.

[avrittrohwer]

+1 for adding 'manual mode' cells that only run when a user explicitly runs then (but are highlighted when they are out of date).

My specific use-case is using marimo for runnable playbooks. I use jupyter for this currently but am evaluating marimo because it is better for source control.

In the case of runnable playbooks, certain things should be reactive like cells that fetch and show/visualize information. Other things should definitely not be reactive like cells that update external systems.

So far the nicest solution I've identified is to use a mo.ui.switch to toggle cell execution. However this has drawbacks:

1. The switch value can not be accessed in the same cell that created it: RuntimeError Accessing the value of a UIElement in the cell that created it is not allowed. Fix: move the value access to another cell. So the switch is in a different cell than it protects.
2. You need to come up with a different name for every switch. In a notebook with lots of switches this is toilsome.

Currently marimo supports marking a cell disabled via the cell decorator:

@app.cell(disabled=True)

What if there was an additional run_mode parameter that accepted values:

• 'reactive' (default)
• 'manual'
• 'manual+confirm'

'manual' would execute the cell when the user clicked the run button or used the keyboard shortcut. 'manual+confirm' would show an additional confirmation popup that would need to be acked before the cell runs

[akshayka]

@avrittrohwer thanks for the write-up.

I think having lazy execution as an option at the level of individual cells makes some sense (though not sure I follow why manual+confirm is needed if manual requires the user to, well, manually run the cell). This would also require us to update the UI in the app preview / marimo run, to provide a way of running lazy cells (which is related to #5385).

However, I wonder if using mo.ui.run_button() along with mo.ui.stop() could get you far enough. How many switches / run buttons might you have in a single notebook? Here is the recipe we recommend: https://docs.marimo.io/guides/expensive_notebooks/#stop-execution-with-mostop

[avrittrohwer]

though not sure I follow why manual+confirm is needed if manual requires the user to, well, manually run the cell

Yeah after playing around in marimo some more this doesn't seem as big of a deal. My initial intuition came from in jupyterlab, when you ctrl+enter to execute a cell, it auto-advances focus to the next cell. So I often find myself ctrl+entering my way though a notebook and sometimes accidentally triggering cell execution for an expensive cell. In marimo you need to more manually target the cell you want to execute so this seems less likely.

Although, for some cell executions it might reduce human error to have that extra level of 'are you sure?' dialog before executing.

However, I wonder if using mo.ui.run_button() along with mo.ui.stop() could get you far enough

mo.io.run_button() or other manual UI element works but it is not super obvious in my opinion because the UI controlling execution of a cell is in a different cell:

@app.cell
def _(mo):
    b1 = mo.ui.run_button()
    b1
    return (b1,)


@app.cell
def _(b1, mo):
    mo.stop(not b1.value)
    # something expensive or potentially dangerous

Usability frictions with this approach:

1. There doesn't seem to be a way to select multiple cells when moving them around the notebook, so if you want to reorder the protected cell you need to remember to move the button too
2. I suspect mo.stop(not b1.value) has a not small likelyhood of being accidentally typed mo.stop(b1.value) especially for folks less familiar with marimo
3. Doesn't play well with SQL cells. I can add click the 'View as Python' button then add mo.stop(not b1.value) to the beginning of the cell, but then I can't click the 'View as SQL' anymore.
4. Requires extra work when running the notebook as a script. As best I can tell, I would have to add CLI args to the notebook to enable running the cell:

@app.cell
def _():
    import marimo as mo
    import argparse

    p = argparse.ArgumentParser()
    p.add_argument("--manual_cells_to_run", nargs="*", type=str, default=[])
    args = p.parse_args()
    return args, mo


@app.cell
def _(mo):
    b1 = mo.ui.run_button()
    b1
    return (b1,)


@app.cell
def shell_cell(args, b1, mo):
    mo.stop(False if "b1" in args.manual_cells_to_run else not b1.value)
    # something expensive or potentially dangerous

If the cell decorator had something like run_mode with 'reactive', 'manual', and 'manual+confirm', marimo could automatically have the a '--manual_cells_to_run` CLI flag and require that manual cells have a name if users want to run them when running the notebook as a script. For 'manual+confirm', marimo runtime could prompt the user 'Request to run 'cell_name', confirmation required [y,N]' and require 'y' to run otherwise exit.

This would also require us to update the UI in the app preview / marimo run, to provide a way of running lazy cells

I am happy to take a stab at implementation if we align on a design

[avrittrohwer]

https://github.com/marimo-team/marimo/issues/3956#issuecomment-3021942903 is a demo of the kind of user flow I'm thinking

[yan-hic]

+1 on If the cell decorator had something like run_mode with 'reactive',, regardless on the notebook run mode.

My use case is a bunch of cells with duckdb SQL that all query S3. Currently when I open marimo or restart the kernel (coz hit out of mem), I need first to run the "init" cell, that holds the s3 credentials or attach ducklake, before running the sql cell of my choice. Would be helpful if I could mark that init cell (+ some others) as always running/reactive.

[jonbeckman]

Requires extra work when running the notebook as a script. As best I can tell, I would have to add CLI args to the notebook to enable running the cell

@avrittrohwer this works well for me. It allows me to run expensive cells via button click in a notebook, while running that cell by default, without a new CLI arg:

@app.cell(hide_code=True)
def _(
    mo,
):
    def execute_workflow(_):
        #...
        return True

    if not mo.running_in_notebook():
        success = execute_workflow(None)
        if not success:
            raise SystemExit(1)

    run_workflow = mo.ui.button(
        label="Execute Data Fetching Workflow", on_click=execute_workflow
    )
    mo.vstack([run_workflow])

    return execute_workflow, run_workflow

[lukesdm]

I think about this issue quite a bit. I use the mo.ui.run_button approach in most of my notebooks at some point. It's ok, but a little awkward sometimes, especially with multiple stages of processing.

In my ideal world... A cell can be automatic, manual, or disabled. There is no notebook-wide setting; or it just sets the default for new cells. automatic and manual correspond to what are currently 'reactive' and 'lazy'. disabled is taken out of the DAG completely - basically treated as though it was commented out.

Of course it's not that simple, as combinations of dependencies of different modes have to be considered (plus staleness). But I'd be curious to hear others thoughts on this. My thinking is, I'd rather avoid having to talk about control flow or execution model with my sciencey colleagues, who aren't necessarily expert programmers, in an intro to marimo lecture.