Support disabling fallback models

[bracesproul]

We should expose a configuration that allows two things:

1. disable fallbacks
2. customize the fallback provider(s)

This should be set in configurable fields, but as a hidden field so that we can render a custom UI in the configuration object for it.

The custom UI should allow users to use a Switch to enable/disable fallbacks

if they have fallbacks disabled, it should make the agent simply throw an error if a model providers fails, instead of falling back

if they have it enabled, we should allow users to:

1. customize the order of fallback providers, and the fallback models
2. remove fallback providers

For example, users should be able to:

1. use the default fallback providers & models
2. remove a fallback provider (e.g. google genai) so that it will only fallback on openai/anthropic, but not attempt to fallback onto Google
3. customize the models of fallback providers (e.g. say for Google, use Gemini 2.5 flash for all fallbacks)

Make this set via configurable fields, and have a sleek and modern UI in the configurable tab in settings to customize this.

Also remember to update the getCustomConfigurableFields function to support passing this/these configurable fields through, even though they're marked as "hidden"

[open-swe[bot]]

🤖 Open SWE has been triggered for this issue. Processing...

View run in Open SWE here (this URL will only work for @bracesproul)

Dev Metadata

{ "runId": "1f073bc5-f36c-6580-8626-f958726b4dbf", "threadId": "072f6351-df6a-435d-94e3-dac257430cca" }

[crowecawcaw]

I think a better solution to the original issue would be if the agent did not fallback to providers that do not have an API key. If a provider does not have an API key configured, I'd guess it's more likely that the user does not intend to use it than the user wanted to fallback to it but forgot to add a key.

This configurable fallback behavior might be a nice add as a power feature, but it'd be nice to have a helpful error message for the common case where someone only has one API key configured.

[dlidstrom]

Not sure if this is relevant, but I've twice seen this error in the programmer flow:

An error occurred: All fallback models exhausted for task programmer. Last error: No API key found for provider: google-genai. Please add one in the settings page.

I only have an Anthropic key configured as per the note in the settings page.

[bracesproul]

@crowecawcaw good callout! I triggered a separate task for this here: #777