[cli] Fail gracefully on unsupported type annotations in config constructors

[jiito]

Description & Motivation

• When LightningCLI parses YAML configs it inspects constructor annotations via LightningArgumentParser/jsonargparse. Typing constructs such as typing.List[str] or Protocol instances (e.g., our ModelConverter) are not concrete runtime types, so LightningCLI raises validation/type errors before callbacks are instantiated (docs describe type-driven config parsing).
• In our repo we had to strip the annotations entirely just to unblock training:

trainer:
 
...

  callbacks:
    - class_path: crosslayer_transcoder.utils.callbacks.ModelConversionCallback
      init_args:
        converter:
          class_path: crosslayer_transcoder.utils.model_converters.circuit_tracer.CircuitTracerConverter  # implements ModelConverter Protocol 
          init_args:

          ...

        on_events: ["on_train_batch_end"]

class ModelConversionCallback(L.Callback):

    # Note: you can't type these directly with List or ModelConverter
    def __init__(
        self, 
        converter, # type: ModelConverter 
        on_events=["on_train_batch_end"], # type: List[str]
    ):
        super().__init__()

...

Pitch

• During schema generation, detect when an annotation is a typing object or protocol that jsonargparse can’t coerce.
• Instead of raising, fall back to treating the argument as Any (or defaulting to primitive validators) and log a warning pointing users to supported types.

Alternatives

No response

Additional context

No response

cc @lantiga @mauvilsa

[mauvilsa]

@jiito the behavior of jsonargparse when it encounters a type that it doesn't support is already to use Any. There could be bugs or edge cases. Better if you provide a minimal reproducible example for what is happening to you. The information above is not enough. Also I don't understand what you mean by

Typing constructs such as typing.List[str] or Protocol instances (e.g., our ModelConverter) are not concrete runtime types

If you have for on_events as type hint List[str] and you give to LightningCLI a list of strings, then there is no reason why it would fail. Is it that you did not know what the problem was, so you removed all type hints? Or does it actually fail if only List[str] is brought back?

Protocols are supported, but I would need more information to say anything about ModelConverter.

And please have a look in the docs How do I troubleshoot a CLI?.

[jiito]

@mauvilsa thank you for the response!

I've been trying to come up with a minimal reproduction and I think it has something to do with module import scope or "double importing".

I'm not familiar with this but trying to get a useful reproduction together and open it in the jsonargparse repo.

We can close this issue here.

[mauvilsa]

@jiito A minimal reproduction would be ideal, but I know that there are cases when it is difficult. If the code in question is public, then you could already share where ModelConverter is defined. Or share the logs with your original code with type hints as explained in How do I troubleshoot a CLI to see if the information there gives clues what is happening.