dash icon indicating copy to clipboard operation
dash copied to clipboard

Support Dash prop callback connection closer to function signature

Open anders-kiaer opened this issue 4 years ago • 7 comments

Is your feature request related to a problem? Please describe. When creating complex Dash dashbords/callbacks, there can be a long list of input/state/output. We in some cases have seen things of the size of:

@app.callback(
    Output("someid", "someprop"),
    Output("someid", "someprop"),
    Output("someid", "someprop"),
    Input("someid", "someprop"),
    Input("someid", "someprop"),
    Input("someid", "someprop"),
    Input("someid", "someprop"),
    Input("someid", "someprop"),
    Input("someid", "someprop"),
    Input("someid", "someprop"),
    Input("someid", "someprop"),
    Input("someid", "someprop"),
    Input("someid", "someprop"),
    State("someid", "someprop"),
    State("someid", "someprop"),
    State("someid", "someprop"),
    State("someid", "someprop"),
    State("someid", "someprop"),
    State("someid", "someprop")
)
def _some_call_back(
    arg1,
    arg2,
    arg3,    
    arg4,    
    arg5,    
    arg6,    
    arg7,    
    arg8,    
    arg9,    
    arg10,    
    arg11,
    arg12,    
    arg13,    
    arg14,    
    arg15,    
    arg16,    
):
    ...

😱 Both during development, but also code review/maintenance, there is mental overhead with respect to seeing which (id, prop) pair belongs to which named argument in the function. This has gotten a bit better with https://dash.plotly.com/flexible-callback-signatures, but the underlying problem with (id, prop) and named argument in the function being defined in two different lines remains.

Describe the solution you'd like Utilize the Python 3+ typing syntax to annotate the (id, prop) closer to the argument. By using that we at the same time can avoid explicitly stating Output as that is already explicit by being defined as the returned type(s) of the function. The only thing to differentiate between is Input/State which could be done by some extra annotation tag. E.g. something like

from typing import Annotated as A

@app.callback
def _some_call_back(
    arg1: A[List[str], "someid", "someprop"],
    arg2: A[List[str], "someid", "someprop"],
    arg3: A[List[str], "someid", "someprop"],
    arg4: A[List[str], "someid", "someprop"],
    arg5: A[List[str], "someid", "someprop", "state"],
) -> Tuple[
        A[List[str], "someid", "someprop"],
        A[List[str], "someid", "someprop"],
        A[List[str], "someid", "someprop"],
    ]:
    ...

Another benefit is that we encourage Dash app developers to type hint their callback functions which again helps IDEs and mypy detect bugs (related to #1786).

Additional context

See https://docs.python.org/3/library/typing.html#typing.Annotated for documentation on typing.Annotated (backported to earlier Python 3 versions in typing_extensions).

Related to #1748 and #1786, but implementation is independent of those.

PoC code showing how type annotations potentially can be transformed into the Input/Output/State when callbacks are registered/decorated:

The below code will print out

Output('id1', 'propname')
Output('id2', 'propname')
Input('id3', 'propname')
Input('id4', 'propname')
State('id5', 'propname')
State('id6', 'propname')
State('id7', 'propname')
#############
# Dash core #
#############
import inspect

# In the std.lib typing library on Python 3.8+:
from typing_inspect import get_args, get_origin


def callback(func):
    for name, annotation in inspect.getfullargspec(func).annotations.items():
        if name == "return":
            if get_origin(annotation) == tuple:
                # Multi valued output
                for return_value in get_args(annotation):
                    print(f"Output{return_value.__metadata__}")
            else:
                print(f"Output{annotation.__metadata__}")
        else:
            if len(annotation.__metadata__) >= 3 and annotation.__metadata__[2] == "state":
                print(f"State{annotation.__metadata__[:2]}")
            else:
                print(f"Input{annotation.__metadata__}")

    return func


######################
# Dash app developer #
######################

from typing import List, Tuple
from typing_extensions import Annotated as A  # On Python 3.9+ this is in typing std.lib


@callback
def _update_plot(
    arg3: A[List[str], "id3", "propname"],
    arg4: A[List[str], "id4", "propname"],
    arg5: A[List[str], "id5", "propname"],
    arg6: A[List[str], "id6", "propname", "state"],
    arg7: A[List[str], "id7", "propname", "state"]
) -> Tuple[
       A[List[str], "id1", "propname"],
       A[List[str], "id2", "propname"] 
    ]:
    ...

anders-kiaer avatar Oct 20 '21 09:10 anders-kiaer

I really like this idea. I just put together a simple (but working) prototype implementation inspired by your code. So far, the syntax looks like this,

from dash_extensions.enrich import T, DashProxy, callback, html

@callback()
def hello(n_clicks: T[int, "btn", "n_clicks"]) -> T[str, "log", "children"]:
    return "Hello world!"

app = DashProxy(prevent_initial_callbacks=True)
app.layout = html.Div([html.Button("Click me", id="btn"), html.Div(id="log")])

if __name__ == '__main__':
    app.run_server()

where T is the import name for Annotated (where "T" is short for "Type", it could also just have been "A", or maybe "P" for "Prop"). My immediate idea was to create Ouput/Input/State classes by subclassing Annotated, but it seems this is not possible. What do you think, @anders-kiaer ? Do you have any ideas for improvements? :)

emilhe avatar Dec 13 '21 20:12 emilhe

Another option would be to use a syntax closer to the current one, e.g. something like

from dash_extensions.enrich import DashProxy, callback, html, Input, Output

@callback
def hello(n_clicks: Input("btn", "n_clicks")) -> Output("log", "children"):
    return "Hello world!"

app = DashProxy(prevent_initial_callbacks=True)
app.layout = html.Div([html.Button("Click me", id="btn"), html.Div(id="log")])

if __name__ == '__main__':
    app.run_server()

This approach doesn't allow the use of actual type annotations though, which might be a drawback. On the other hand, native Dash doesn't allow complex types as input/output for callbacks, so it might not be so bad after all.

emilhe avatar Dec 13 '21 22:12 emilhe

I really like this idea. I just put together a simple (but working) prototype implementation inspired by your code.

🎉🙂

Do you have any ideas for improvements? :)

Overall I like the syntax and think using type hint annotations improves readability. I'm not sure if the following are ideas, maybe more questions:

  • (Input/State) and Output are clearly distinguished by being input and output type hints of the callback function. How do we best distinguish between Input and State within the input arguments?
  • from dash_extensions.enrich import T - did you encounter issue with e.g. from typing import Annotated as A from std.lib?
  • Reading the Python standard library documentation of typing.Annotated, they have examples of using typing.Annoted[sometype, SomeClass(initval)], maybe because it is easier to distinguish then with isinstance (as they also use as example) should the end user have other use cases for annotating the arguments, in addition to dash (id, prop) specification (in pydantic they have some discussion on e.g. argument text descriptions as annotations). E.g.:
    from dash.dependencies import Prop
    
    @callback
    def hello(
        some_input: T[int, Prop("id", "prop")],
        some_state: T[int, Prop("id", "prop", trigger=False)],
     ) -> T[str, Prop("id", "prop")]:
        return "Hello world!"
    

On the other hand, native Dash doesn't allow complex types as input/output for callbacks, so it might not be so bad after all.

My hope is that it will change after #1786 - and then it would be nice to able to use the full power of type hint annotations. 🙂

One option (combining the class approach above and also reuse existing Input/State/Output) could be

@callback
def hello(n_clicks: A[sometype, Input("btn", "n_clicks")]) -> A[sometype, Output("log", "children")]:
    return "Hello world!" 

but it feels more verbose/explicit than necessary (Output is already clearly an output by being the returned type of the function).

anders-kiaer avatar Dec 14 '21 09:12 anders-kiaer

I have done another (small) iteration of the syntax, partly based on your comments, @anders-kiaer . My working example now looks like this,

from dash_extensions.enrich import DashProxy, callback, html, dcc, prop

@callback
def hello(n_clicks: prop("btn", "n_clicks", int), # the type definition "int" is optional
          name: prop("name", "value", trigger=False)) -> prop("log", "children"):
    return f"Hello {name}! (click count is {n_clicks})"

app = DashProxy(prevent_initial_callbacks=True)
app.layout = html.Div([
    dcc.Input(placeholder="Enter your name here", id="name"),
    html.Button("Click me!", id="btn"),
    html.Div(id="log")
])

if __name__ == '__main__':
    app.run_server()

A few notes on the syntax,

  • The Input/Output/State objects are replaced by a common prop function
  • The first two arguments of the prop function are component id and component property (both mandatory), the third argument is an optional type definition (which just defaults to any)
  • All argument props are per default interpreted as Input, but can be changed to State by passing trigger=False
  • Similarly, all return props are interpreted as Output, but can be changes to ServersideOutput by passing serverside=True

You should be able to try out the syntax with the following rc release,

https://pypi.org/project/dash-extensions/0.0.67rc2/

I haven't tested much more than the example above, so expect dragons :)

emilhe avatar Dec 16 '21 16:12 emilhe

Hey @anders-kiaer and @emilhe

I'd be interested to hear your feedback on #1952 Improved callback_context - especially the section "Bonus 1: Simplified callback function signatures" section

AnnMarieW avatar Mar 04 '22 18:03 AnnMarieW

@gvwilson @AnnMarieW Is there any progress? For my own purpose I have written the decorator https://gist.github.com/DerWeh/6a8105f8da35f56485880006a6964493.

Implementation

Supporting Python 3.8 makes things a bit ugly, and I found no way to currently check signatures with mypy.

The essence of the implementation is

def typed_callback(func: Union[Callback, None], **kwds) -> Callable:
    """Create `dash.callback` from type hints, forward `kwds`."""
    annotation = get_annotations(func, eval_str=True)
    return_type = annotation.pop("return")
    if get_origin(return_type) is tuple:
        output = tuple(_output(arg) for arg in get_args(return_type))
    else:
        output = _output(return_type)

    inputs = {key: _input(val) for (key, val) in annotation.items()}
    callback_decorator = callback(output=output, inputs=inputs, **kwds)
    if func is None:
        return callback_decorator
    return callback_decorator(func)

My decorator parses the signature and forwards it to the dash.callback. If people are interested in integrating this into Dash, I would be willing to draft a PR.

Examples

Common imports:

from typing_extensions import Annotated as A, Tuple
from dash import Dash, dcc, html, Input, Output  # type: ignore[import]

The example dash-app-with-multiple-inputs would turn into

@typed_callback
def update_graph(
    xaxis_column_name: A[str, Input("xaxis-column", "value")],
    yaxis_column_name: A[str, Input("yaxis-column", "value")],
    xaxis_type: A[str, Input("xaxis-type", "value")],
    yaxis_type: A[str, Input("yaxis-type", "value")],
    year_value: A[int, Input("year--slider", "value")],
) -> A[go.Figure, Output("indicator-graphic", "figure")]:
    dff = df[df['Year'] == year_value]

    fig = px.scatter(x=dff[dff['Indicator Name'] == xaxis_column_name]['Value'],
                     y=dff[dff['Indicator Name'] == yaxis_column_name]['Value'],
                     hover_name=dff[dff['Indicator Name'] == yaxis_column_name]['Country Name'])

    fig.update_layout(margin={'l': 40, 'b': 40, 't': 10, 'r': 0}, hovermode='closest')

    fig.update_xaxes(title=xaxis_column_name,
                     type='linear' if xaxis_type == 'Linear' else 'log')

    fig.update_yaxes(title=yaxis_column_name,
                     type='linear' if yaxis_type == 'Linear' else 'log')

    return fig

The example dash-app-with-multiple-outputs turns into

@typed_callback
def callback_a(
    x: A[int, Input("num-multi", "value")],
) -> Tuple[
    A[int, Output("square", "children")],
    A[int, Output("cube", "children")],
    A[int, Output("twos", "children")],
    A[int, Output("threes", "children")],
    A[int, Output("x^x", "children")],
]:
    return x**2, x**3, 2**x, 3**x, x**x

Rationale

I choose Annotated, this doesn't get in the way of regular type hints and aligns with the choice of the popular pydantic package: the-annotated-pattern. With newer Python versions, we could probably use a more concise syntax like Output[int](...) for Annotated[int, Output(...)] and Ouput(...) for Annoated[Any, Output(...)].

DerWeh avatar Sep 13 '25 11:09 DerWeh

@T4rk1n any thoughts on this one?

gvwilson avatar Sep 23 '25 15:09 gvwilson