`persona` and `unsupported_slash_commands` class attribute for custom provider does not affect anything

[skabbit]

trafficstars

Description

persona and unsupported_slash_commands class attribute for custom provider does not affect any behavior.

I see no name changed and unsupported_slash_commands in not filtered out.

Reproduce

class CustomProvider(BaseProvider):
    id = "custom"
    name = "Custom"
    models = ["custom"]
    model_id_key = "custom"

    # both of these params not working somehow, but expected to override the default ones
    unsupported_slash_commands = {'/ask', '/learn'}
    persona = Persona(name='Custom name',
                      avatar_route='api/ai/static/jupyternaut.svg')

And then go through all steps here: https://jupyter-ai.readthedocs.io/en/latest/developers/index.html#custom-model-providers

Expected behavior

I expect that persona name should be updated in chat tab.

Context

I'm using custom provider to enhance LLM with our internal knowledge.

[dlqqq]

@skabbit Thanks for reaching out! To use a custom provider, you have to:

• define this custom provider in a new custom package,
• serve this provider through the entry points API,
• install the custom package in your Python environment,
• and finally select that provider in the AI settings.

The docs should cover these steps. Assuming you've done all of these steps, note that the new persona will only show up after you select the custom provider in the AI settings & start a new chat. You can do so either by running /clear or by clicking the "+" (new chat) button on the top right, next to the settings icon.

You can use the --AiSettings.default_language_model flag when starting JupyterLab to set the default chat model to your custom provider. That way, you will always see your provider's persona and not see any disabled slash commands. Hope this helps.

We are redefining how this works in Jupyter AI v3, and I'm working on this now. See here: #1313.

[skabbit]

Hi @dlqqq

Yes, I have done all these steps and our custom provider works as expected, but only persona and slack commands are not changing. I have selected model in configuration, restarted JupyterLab, + new chat, /clear command, and tried multiple other things, but no luck yet. It still shows Jupyternaut persona with default list of slash commands. So I consider it is a bug.

Or did I get something wrong? Are these attributes must be within a class inherit from BaseProvider as I do?

class CustomProvider(BaseProvider):
    id = "custom"
    name = "Custom"
    models = ["custom"]
    model_id_key = "custom"

    # both of these params not working somehow, but expected to override the default ones
    unsupported_slash_commands = {'/ask', '/learn'}
    persona = Persona(name='Custom name',
                      avatar_route='api/ai/static/jupyternaut.svg')

Anyway, thank you for pointing on --AiSettings.default_language_model argument, which is useful for our solution to avoid manual selection.

[skabbit]

Also looks like a nice improvement you're working on here: https://github.com/jupyterlab/jupyter-ai/issues/1313 Thanks, good luck with that.

[dlqqq]

@skabbit Hey, sorry you're not able to get this working. It's a bit tricky to debug this without seeing some server logs. Can you share the terminal output of jupyter lab --AiSettings.default_language_model=<custom-model-uid>? If it's not sensitive, can you also share the full source code of your custom provider? We may be able to find you a fix with that info.

Also looks like a nice improvement you're working on here: https://github.com/jupyterlab/jupyter-ai/issues/1313

Yes, I'm opening a PR for that in the next couple of days. The new API will greatly simplify adding a persona and defining how that persona handles messages. You might find it easier to use the new API once it's shipped with a pre-release (e.g. v3.0.0a2), and stick with that until v3.0.0 official is released. I'll let you know once that's ready. 👍

[skabbit]

hello @dlqqq ! Now it is working fine to me, have no idea what changed, the only thing I done is upgrade from jupyter_ai==2.29.0 to jupyter_ai==2.31.4.

[dlqqq]

Thanks for letting us know! Glad the latest releases fixed your setup. Closing this issue.