[E2E][Snapshot] TestNameValidation is flaky

[barkbay]

TestNameValidation/longestPossibleName/ApmServer_should_accept_event_and_write_data_to_Elasticsearch failed with the following error:

=== RUN   TestNameValidation/longestPossibleName/ApmServer_should_accept_event_and_write_data_to_Elasticsearch
Retries (30m0s timeout): .......................................................
    step.go:43: 
        	Error Trace:	utils.go:88
        	Error:      	Received unexpected error:
        	            	elasticsearch client failed for https://es-naming-lhpx-xxxxxxxxxxxxxxxxxxxxx-es-internal-http.e2e-4bw2c-mercury.svc:9200/metrics-apm.app.1234_service_12a3-default/_count: 404 Not Found: {Status:404 Error:{CausedBy:{Reason: Type:} Reason:no such index [metrics-apm.app.1234_service_12a3-default] Type:index_not_found_exception StackTrace: RootCause:[{Reason:no such index [metrics-apm.app.1234_service_12a3-default] Type:index_not_found_exception}]}}
        	Test:       	TestNameValidation/longestPossibleName/ApmServer_should_accept_event_and_write_data_to_Elasticsearch
{"log.level":"error","@timestamp":"2022-08-01T01:28:51.677Z","message":"stopping early","service.version":"0.0.0-SNAPSHOT+00000000","service.type":"eck","ecs.version":"1.4.0","error":"test failure","error.stack_trace":"github.com/elastic/cloud-on-k8s/v2/test/e2e/test.StepList.RunSequential\n\t/go/src/github.com/elastic/cloud-on-k8s/test/e2e/test/step.go:44\ngithub.com/elastic/cloud-on-k8s/v2/test/e2e.testLongestPossibleName\n\t/go/src/github.com/elastic/cloud-on-k8s/test/e2e/naming_test.go:85\ntesting.tRunner\n\t/usr/local/go/src/testing/testing.go:1439"}
        --- FAIL: TestNameValidation/longestPossibleName/ApmServer_should_accept_event_and_write_data_to_Elasticsearch (1800.00s)

Integration seems outdated:

{
	"log.level": "error",
	"@timestamp": "2022-08-01T01:28:01.977Z",
	"log.logger": "modelindexer",
	"log.origin": {
		"file.name": "modelindexer/indexer.go",
		"file.line": 412
	},
	"message": "failed to index event (fail_processor_exception): Document produced by APM Server v8.4.0, which is newer than the installed APM integration (v8.3.3). The APM integration must be upgraded.",
	"service.name": "apm-server",
	"ecs.version": "1.6.0"
}

[barkbay]

APM package is already set to latest in Kibana:

spec:
  config:
    xpack.fleet.packages:
    - name: apm
      version: latest

I'm still trying to figure out what's going on, maybe it's an inconsistency in the snapshot images...

[barkbay]

Tests are back to green status: