Add --random option

[taiki-e]

Originally proposed by @orlp. https://discord.com/channels/273534239310479360/273541522815713281/1263250421653438625

        --random <NUM_SAMPLES>
            Performs with random feature combinations up to the number specified per crate.

            This flag can only be used together with --feature-powerset flag.

There are a few things missing as I have just roughly implemented the idea:

• [ ] Resolve todos in code comments: https://github.com/taiki-e/cargo-hack/blob/d6b31648b0ee18719ca7b15d0d998e1577a0d3fd/src/features.rs#L232-L236
• [ ] Add a seeding option (https://github.com/taiki-e/cargo-hack/pull/255#issuecomment-2940507876)

Example:

$ cargo hack check --feature-powerset --no-dev-deps --optional-deps --random 10
info: --no-dev-deps modifies real `Cargo.toml` while cargo-hack is running and restores it when finished
info: running `cargo check --no-default-features --features b,c,default,member1` on real (1/10)
    Finished `dev` profile [unoptimized + debuginfo] target(s) in 0.04s

info: running `cargo check --no-default-features --features c,default` on real (2/10)
    Finished `dev` profile [unoptimized + debuginfo] target(s) in 0.00s

info: running `cargo check --no-default-features --features a,b,default,member1` on real (3/10)
    Finished `dev` profile [unoptimized + debuginfo] target(s) in 0.03s

info: running `cargo check --no-default-features --features c,default` on real (4/10)
    Finished `dev` profile [unoptimized + debuginfo] target(s) in 0.00s

info: running `cargo check --no-default-features --features default,member1` on real (5/10)
    Finished `dev` profile [unoptimized + debuginfo] target(s) in 0.03s

info: running `cargo check --no-default-features --features a,b` on real (6/10)
    Finished `dev` profile [unoptimized + debuginfo] target(s) in 0.00s

info: running `cargo check --no-default-features --features c` on real (7/10)
    Finished `dev` profile [unoptimized + debuginfo] target(s) in 0.03s

info: running `cargo check --no-default-features --features a,b,c,member1` on real (8/10)
    Finished `dev` profile [unoptimized + debuginfo] target(s) in 0.03s

info: running `cargo check --no-default-features --features a,c,default` on real (9/10)
    Finished `dev` profile [unoptimized + debuginfo] target(s) in 0.00s

info: running `cargo check --no-default-features --features a,b,default` on real (10/10)
    Finished `dev` profile [unoptimized + debuginfo] target(s) in 0.00s

[elichai]

I might be misunderstanding this feature, or missing something about the code changes, but why not just call fastrand::shuffle on the Vec<Vec<Feature>> from fn powerset and then take the first num_samples?

[taiki-e]

The way to calculate the powerset first and then select specific combinations is fine if the number of feature combinations is small, but as the number of combinations increases, performance and memory usage issues arise, eventually leading to OOM.

[orlp]

Can there be a seeding option?

That is, --random N without specifying a seed picks a random seed and proceeds. Should an error occur it will report the error as well as the seed used (for later reproduction).

Then --random N --random-seed S can be used to reproduce that error and/or do deterministic runs if desired.

[taiki-e]

I hadn't thought about that, but indeed supporting seeding seems to be very useful.

Added it to the todo list in this PR's description.