feat(Validium): Adding CI jobs for running integration tests in Validium mode

[jordibonet-lambdaclass]

What ❔

Using the matrix functionality of GitHub Actions (similar to the consensus implementation), add jobs to the CI to run the integration tests and the External Node in Validium Mode.

Changes made

Continuous Integration

Update ci-core-reusable.yml to run the integration tests and the External Node in Validium Mode using matrix functionality.

A new dev_validium_docker.toml is also needed to use the correct database configuration.

Fixing linter errors

Update system.test.ts and commit_batch_info.rs to make the linter pass.

Fixing unit test

Fix tests.rs to pass unit tests. The failing test was correct_order_for_confirmations, you can run it using this command:

zk test rust --package  zksync_core eth_sender::tests::correct_order_for_confirmations

This was due to the fact that contracts_are_pre_shared_bridge was set to false, which meant that we were using post shared bridge contracts and this is not the case.

Revert test and Upgrade test

Update revert-and-restart.test.ts and upgrade.test.ts so that the value that the sender has once it has been returned is greater than or equal to the value it had before. this is done in this way and we think it makes sense since in Validium it spends less gas.

Snapshot recovery test

Update snapshot-recovery.test.ts to use Validium external node configuration in order to pass the test. The test failed because of a timeout waiting for the connection. You can check the correction of this test by running the server in validium mode (zk && zk clean --all && zk env dev_validium && zk init --validium-mode && zk server) and running the following command in another terminal:

cd core/tests/snapshot-recovery-test/ && npm run snapshot-recovery-test

[jordibonet-lambdaclass]

While adapting the CI for Validium of the integration tests and the external node, we found a problem with the connection string for the databases: in Rollup it was postgres://postgres@postgres/prover_local and in Validium postgres://postgres@localhost/prover_local. We fixed it with the configuration file dev_validium_docker.toml, adding the same configuration as docker.toml.

Currently we have two doubts:

• We can't find where the connection string for databases has been configured to be postgres://postgres@localhost/prover_local.
• The contracts submodule seems not to be pointing to the right commit despite it is correctly set in .gitmodules, which causes the Verifier deployment to fail, because we need the deployment to take a parameter that has been added now, which is --validium-mode. You can check it in the last CI run. Is the submodule commit being updated somewhere else than in the Checkout system-contracts submodule step?

@popzxc

[ilitteri]

* The contracts submodule seems not to be pointing to the right commit despite it is correctly set in `.gitmodules`, which causes the `Verifier` deployment to fail, because we need the deployment to take a parameter that has been added now, which is `--validium-mode`. You can check it in the [last CI run](https://github.com/matter-labs/zksync-era/actions/runs/8160772169/job/22308165141?pr=1312). Is the submodule commit being updated somewhere else than in the _Checkout system-contracts submodule step_?

Probably there's something wrong with the commit that is pushed, I'll double check this

[popzxc]

@jordibonet-lambdaclass I think that by default the prover database URL is taken from etc/env/private.toml

[jordibonet-lambdaclass]

Thank you @popzxc. Apologies, I made a mistake, we were actually referring to the other one (postgres://postgres@postgres/prover_local). We can't find in which part of the CI it is assigned that one and not the default one in private.toml.

[popzxc]

@jordibonet-lambdaclass I'm not sure if I fully understand the issue, but as a rough hypothesis: recently we've changed the approach to the databases, instead of using "trust" auth method everywhere, we introduced a hardcoded password ("no secure password"). The connection string you mentioned has no password in it, so maybe you're trying to use an old connection string somewhere. If you're interested to see where the Postgres in configured in general, then it's this docker-compose file.

If that doesn't help, please elaborate what you're trying to do, what goes wrong, and what would you expect to work.

[ilitteri]

The integration tests that are failing are fixed in https://github.com/matter-labs/zksync-era/pull/1270 and https://github.com/matter-labs/zksync-era/pull/1296. We can wait for #1270 to be merged to pull from the base branch, but this PR can be reviewed in the mean time