[Feature Request][move-compiler-v2] Parallel .exp files for all tests to be able to regression-test Compilers V1 and V2 in parallel for a while

[brmataptos]

🚀 Feature Request

As Compiler V2 is able to work on most things, we increasingly need to have regression test output for V2 for all tests involving the compiler in the repo. We currently can force V2 compiler use with environment variable MOVE_COMPILER_V2=true, so we can run tests in parallel, but we need a way to keep track of whether the outputs are as expected

I see several possible approaches:

• have an aptos-core branch for V2 with the expected outputs, and run on that branch --- bad, because code may go out of sync
• a parallel set of expected outputs in each test directory (e.g., .v2.exp) to allow testing of V1. --- bad, because every test framework needs modification
• use a test script which uses the main branch for V2 testing, but overlays expected V2 outputs copied from somewhere else
  • e.g., a subdir
  • e.g., a different branch

The last approach seems easiest to maintain. Easiest may be maintaining a branch for the V2 test outputs, but using main branch code to run them.

[brmataptos]

@rahxephon89 @wrwg

[brmataptos]

I note that the usual expected output extension "exp" is defined in third_party/move/move-command-line-common/src/testing.rs:

/// Extension for expected output files
pub const EXP_EXT: &str = "exp";
/// Extension for expected output files compiled by v2
pub const EXP_EXT_V2: &str = "exp.v2";

Despite what is written there, the only tests currently generating V2-specific outputs are using "v2_exp" (the move-prover tests, due to code in third_party/move/move-prover/testsuite.rs).

So I suggest that current uses of EXP_EXT should be replaced by calls to a new function get_exp_ext(), defined as follows:

    /// Extension for expected output files
    pub const EXP_EXT: &str = "exp";
    /// Extension for expected output files compiled by v2
    pub const EXP_EXT_V2: &str = "v2_exp";
    /// Get the right extension for the current compilation environment
    fn get_exp_ext() -> &'static str {
        if read_bool_env_var(MOVE_COMPILER_V2) {
            EXP_EXT_V2
        } else {
            EXP_EXT
        }
    }

with the definition of EXP_EXT_V2 modified as shown to match the current practice in move-prover.

[brmataptos]

This approach is now illustrated in https://github.com/aptos-labs/aptos-core/pull/12944, but without a separate function yet since I want to reorganize it based on https://github.com/aptos-labs/aptos-core/pull/12943 once one of them gets committed.

[rahxephon89]

@brmataptos, maybe we can close this because of your PR https://github.com/aptos-labs/aptos-core/pull/13254?

[brmataptos]

We should track getting it in CI. But with the script we can call it done, maybe.

[brmataptos]

Done.