Allow overriding Spring Configuration Properties using PicoCLI-parsed arguments

[AlexandreCassagne]

Following the discussion with @remkop in https://github.com/kakawait/picocli-spring-boot-starter/issues/16#issuecomment-1053831961, I am moving the issue here.

Is it possible, or would it be feasible to implement, the possibility to override spring properties using arguments parsed at command-line?

For instance, suppose you are creating a Spring API client, and the remote server is configurable using property myapp.apiRoot=https://example.com, could we pass this as an argument mycommand --api-root https://example.com?

[AlexandreCassagne]

@remkop Has proposed a workaround and mentioned https://github.com/remkop/picocli/issues/1378 as having similarity.

I could possibly provide a pull request, but I am not 100% sure what the best approach is!

[remkop]

Looking at https://www.baeldung.com/spring-boot-command-line-arguments, it seems that Spring may already be doing this automatically.

The Spring example in the picocli user manual starts the application as follows:

    public static void main(String[] args) {
        // let Spring instantiate and inject dependencies
        // NOTE: Spring gets to see the command line args first!
        System.exit(SpringApplication.exit(SpringApplication.run(MySpringMailer.class, args)));
    }

    // Spring then invokes our CommandLineRunner implementation, which (finally) uses picocli to parse the args
    @Override
    public void run(String... args) {
        // let picocli parse command line args and run the business logic
        exitCode = new CommandLine(mailCommand, factory).execute(args);
    }

Assuming the following example Spring options:

--spring.main.banner-mode=off
--server.port=8085

So, it appears to me (I could be wrong) that Spring will already have processed these arguments, and there is not much that picocli can do to enhance that. However, the same arguments are passed to picocli, so we want to

• ensure that our application does not error on them as "unmatched options"
• maybe go one step further and show these options in the usage help message

That leads me to conclude that such applications should either allow and ignore unmatched options, or declare these options in the application, just to show them in the usage help message, without actually doing anything with these options in the application. For example:

// idea 1
@Unmatched List<String> unmatched; // ignore unmatched options

// idea 2
@Option(names = "--spring.main.banner-mode") String ignored1;
@Option(names = "--server.port") int ignored2;

Thoughts?

[remkop]

@AlexandreCassagne Can you let me know your thoughts on this?

[AlexandreCassagne]

@remkop It is a good point, and I hadn't noticed that Spring was already doing this. I'd need to play around with those unmatched options to make sure I understand well (little swamped by work at the moment—I'll get back to this eventually)

However, it implies that some of the facilities offered by PicoCLI's argument parser (e.g. use of properties files, use of interactive arguments, …) are not available.

I would like to investigate if it is possible to run Spring's argument parser by aggregating PicoCLI's arguments… I don't see any reason why not but I feel I've reached the limits of my understanding of the Spring framework's design - would need to research further

example data flow:

[ Spring Arg parser ] → [ PicoCLI arg parser ] → [inject PicoCLI commands' arguments] → [Spring Configuration]

[AlexandreCassagne]

Is that an unreasonable approach ?

[remkop]

However, it implies that some of the facilities offered by PicoCLI's argument parser (e.g. use of properties files, use of interactive arguments, …) are not available.

Good point. Yes, in principle, picocli has the ability to add values that were not provided on the command line, either from interactive options that prompt the user for input, or from the default values (which may come from properties file).

The question then becomes how to accomplish this, which leads to your next point:

I would like to investigate if it is possible to run Spring's argument parser by aggregating PicoCLI's arguments… I don't see any reason why not but I feel I've reached the limits of my understanding of the Spring framework's design - would need to research further

example data flow:

[ Spring Arg parser ] → [ PicoCLI arg parser ] → [inject PicoCLI commands' arguments] → [Spring Configuration]

I am also not familiar enough with Spring to know if it is possible to do what you outline above. I suspect that the first step (Spring Arg parser) would result in a full Spring Configuration, and it may not be possible to modify that afterwards. (Again, not sure about this.)

One alternative idea is to have a two phase approach. The user manual has an example of this approach. That example parses the input once (partially) just to configure the locale, then parses the same input again (fully) to start the application.

So, for Spring, you would need to

• define a Bean with Spring properties and default values (or interactive input)
• parse the input with that Bean first (ignoring all other command line args)
• ask that Bean to generate command line args to pass to Spring (which may come from user input or from default value or interactive input
• then start the usual chain Spring Arg parser -> picocli arg parser -> application business logic

The bean could look something like this:

// defines defaults in the annotations, could also come from a default provider or interactive input
class SpringOptions {
    @Option(names = "--spring.main.banner-mode", defaultValue = "OFF")
    String bannerMode;
    
    @Option(names = "--server.port", defaultValue = "12345") 
    int port;

    @Unmatched List<String> unmatched = new ArrayList<>(); // ignore remaining options

    public String[] asArgs() {
        List<String> result = new ArrayList<>();
        result.addAll(List.of("--spring.main.banner-mode", bannerMode, "--server.port", String.valueOf(port)));
        result.addAll(unmatched);
        return result.toArray(new String[0]);
    }
}

Then we could use that in the main method, like this:

@SpringBootApplication
public class SpringMain implements CommandLineRunner {

    private IFactory factory;
    private App app; 

    // constructor injection
    MySpringMailer(IFactory factory, App app) {
        this.factory = factory;
        this.app = app;
    }

    @Override
    public void run(String... args) {
        // let picocli parse command line args (fully) and run the business logic
        new CommandLine(app, factory).execute(args);
    }

    public static void main(String[] args) {
        // parse args once to provide default values
        SpringOptions options = new SpringOptions();
        new CommandLine(options).parseArgs(args);
        String[] updatedArgs = options.asArgs();

        // let Spring instantiate and inject dependencies
        System.exit(SpringApplication.exit(SpringApplication.run(MySpringMailer.class, updatedArgs)));
    }
}

But, to be honest, this is quite a bit of work, and I am not sure how much extra value it provides...

A lot depends on how much defaulting is available for the Spring properties; I believe they can already be configured from a combination of command line and config files, so that already provides a lot of the value we are looking for...

[remkop]

@AlexandreCassagne What shall we do with this ticket? The documentation currently has a TIP section at the end of the Spring Boot Example that mentions defining a --spring.config.location option to avoid getting "unmatched options" errors.

About defaulting, while it is possible to use picocli's defaulting mechanism to configure Spring (see example in my previous comment), it does not feel straightforward or easy. So I guess there are 2 questions:

• Is it possible to make this integration of defaulting mechanisms easier somehow? - The only other thing I can think of is to create a Default Provider that reads Spring configuration files. But I am not familiar enough with Spring to know what files to read.
• Any suggestions for improving the documentation? This topic feels quite specialized, like it deserves a blog post or article of its own. Do you feel like doing a writeup for this? I would be happy to add it to the picocli site with the other docs, and link to it from the manual, but I hesitate to include more details about this topic in the user manual directly. What do you think?

If you are too busy that is fine of course, just let me know.

[remkop]

I am cleaning up old tickets. I don't think there is any work remaining on this ticket, so I will close it. Thank you for raising this!