nodejs-container-analysis: samples tests are disabled for this repository

[NimJay]

• This is regarding the samples inside /container-analysis/snippets.
• This issue was originally reported at https://github.com/googleapis/nodejs-containeranalysis/issues/441 by @alexander-fenster.
• The reported issue:

The current samples tests depend on the sequence of invocation: the resource is being created in the first test, used in the next test, then deleted in a separate test. This organization of tests leads to flaky tests and is error-prone. Samples tests should be redesigned so that the tests could be executed independently (so that skipping one test wouldn't cause failure of subsequent tests).

For now, I'm disabling all samples tests in https://github.com/googleapis/nodejs-containeranalysis/pull/440.

[ace-n]

This issue is specific to this test file.

[ace-n]

Splitting these tests apart would take ~1 engineer-day, which is quite a bit of time.

There are two easier approaches here:

1. add retries to each of the current tests
2. combine separate (create-get-delete) tests into a single test

@muncus and I discussed this, and we're in favor of Option 2.

@alexander-fenster @NimJay any objections?

[NimJay]

First off, Ace, thanks for looking into this issue (and even digging up the file that it's about, containerAnalysis.test.js).

Both approaches have their costs, so I'm good with either approach. If I understand correctly, approach 1 would add more noise to errors messages and use up more CPU (in some cases). Whereas, approach 2 would be less modular (e.g., the code might not be as clean).

[ace-n]

@NimJay one other point about Option 2:

If a particular step in the create-get-delete cycle fails (e.g. a get operation), it will cause the entire create-get-delete test to fail.

If that becomes an issue, I'm in favor of addressing it by adding retry-on-failure logic to these (broader) tests.

[pattishin]

Closing out as this is running now in https://github.com/GoogleCloudPlatform/nodejs-docs-samples/blob/main/.github/workflows/container-analysis-snippets.yaml.