testsuite: add a queues personality / config option

[chu11]

As I was looking to add tests for #4604, realized it might be convenient to add either a personality or option to test_under_flux to auto config standard batch/debug queues for the tests.

Although it will probably become easier in time, I'm still in that "How do I configure queues?" phase and hunt for toml file examples.

[garlick]

I think the trickier bit in test is assigning properties to resources since a) the hostnames are all the same for test brokers so the TOML method that keys from hostname won't work, and b) resource and the scheduler must be reloaded whenever R changes. Maybe we could modify the job personality to assign batch and debug properties to its fake resources in some prescribed way? Then a test could more easily configure queues on the fly that reference those properties.

A related thought is that the broker has to have a toml directory in order to configure anything in test and the initial setup and then management of a toml directory is tedious. For situations where the configuration can be set on the fly (e.g. with flux config reload) it would be convenient if there were a way just skip the directory setup and have a flux config reload option to set the config object from stdin. Then you could do something like:

flux config reload --stdin <<-EOT
[queues.batch]
requires = ["batch"]
[queues.debug]
requires = ["debug"]
EOT

to reset the entire config and not worry about what may have been left over in the toml directory from a previous subtest.

[grondo]

I like that idea!

[chu11]

A related thought is that the broker has to have a toml directory in order to configure anything in test and the initial setup and then management of a toml directory is tedious.

Perhaps I'm missing something. Normally I just set FLUX_CONFIG_DIR=$(pwd) before starting flux and then something like:

	cat >kvs.toml <<-EOF &&
	[kvs]
	checkpoint-period = "1Z"
	EOF
	flux config reload

would just work.

But I like your idea, b/c I've forgotten to set FLUX_CONFIG_DIR several times, and fumbled trying to figure out why the config isn't being loaded. Your --stdin option idea makes things clearer and more mistake proof.

[grondo]

Yeah, we should get a separate issue open for the flux config reload --stdin (or should it just be flux config load?) option. This is the only way you can configure an instance that has already started without a configuration directory set. For instance, this would give users a way to configure their batch jobs from within the batch script.

[grondo]

I think the trickier bit in test is assigning properties to resources since a) the hostnames are all the same for test brokers so the TOML method that keys from hostname won't work, and b) resource and the scheduler must be reloaded whenever R changes.

I swear we had already an issue open on this, but it would help to be able to dynamically assign properties to ranks via a command line tool. The result of the tool could be set-property/remove-property events in the resource.eventlog and the equivalent in the resource acquisition protocol...

[grondo]

This should have been closed with #4678. Feel free to reopen if there is still a need.