Failing test: Jest Integration Tests.src/core/server/elasticsearch/integration_tests - fake elasticsearch should return unknown product when it cannot perform the Product check (503 response)

[kibanamachine]

A test failed on a tracked branch

Error: expect(received).toBe(expected) // Object.is equality

Expected: "critical"
Received: "unavailable"
    at Object.<anonymous> (/var/lib/buildkite-agent/builds/kb-n2-4-spot-bc5f5d6543db3d55/elastic/kibana-on-merge/kibana/src/core/server/elasticsearch/integration_tests/client.test.ts:94:44)
    at runMicrotasks (<anonymous>)
    at processTicksAndRejections (node:internal/process/task_queues:96:5)
    at _callCircusTest (/var/lib/buildkite-agent/builds/kb-n2-4-spot-bc5f5d6543db3d55/elastic/kibana-on-merge/kibana/node_modules/jest-circus/build/run.js:212:5)
    at _runTest (/var/lib/buildkite-agent/builds/kb-n2-4-spot-bc5f5d6543db3d55/elastic/kibana-on-merge/kibana/node_modules/jest-circus/build/run.js:149:3)
    at _runTestsForDescribeBlock (/var/lib/buildkite-agent/builds/kb-n2-4-spot-bc5f5d6543db3d55/elastic/kibana-on-merge/kibana/node_modules/jest-circus/build/run.js:63:9)
    at _runTestsForDescribeBlock (/var/lib/buildkite-agent/builds/kb-n2-4-spot-bc5f5d6543db3d55/elastic/kibana-on-merge/kibana/node_modules/jest-circus/build/run.js:57:9)
    at run (/var/lib/buildkite-agent/builds/kb-n2-4-spot-bc5f5d6543db3d55/elastic/kibana-on-merge/kibana/node_modules/jest-circus/build/run.js:25:3)
    at runAndTransformResultsToJestFormat (/var/lib/buildkite-agent/builds/kb-n2-4-spot-bc5f5d6543db3d55/elastic/kibana-on-merge/kibana/node_modules/jest-circus/build/legacy-code-todo-rewrite/jestAdapterInit.js:176:21)
    at jestAdapter (/var/lib/buildkite-agent/builds/kb-n2-4-spot-bc5f5d6543db3d55/elastic/kibana-on-merge/kibana/node_modules/jest-circus/build/legacy-code-todo-rewrite/jestAdapter.js:109:19)
    at runTestInternal (/var/lib/buildkite-agent/builds/kb-n2-4-spot-bc5f5d6543db3d55/elastic/kibana-on-merge/kibana/node_modules/jest-runner/build/runTest.js:380:16)
    at runTest (/var/lib/buildkite-agent/builds/kb-n2-4-spot-bc5f5d6543db3d55/elastic/kibana-on-merge/kibana/node_modules/jest-runner/build/runTest.js:472:34)

First failure: CI Build - main

[elasticmachine]

Pinging @elastic/kibana-core (Team:Core)

[kibanamachine]

New failure: CI Build - 8.2

[spalger]

Failed 5 PRs this week

8.3/main: https://github.com/elastic/kibana/commit/fc7d3850bd21ddb23cab50f41f16822147c61be7 8.2: https://github.com/elastic/kibana/commit/72501b9eec4f5011d9e9783c8bcec70b0bebb732

[pgayvallet]

This unavailable status is coming from here:

https://github.com/elastic/kibana/blob/5edf7e267aecb5b48c90ca3d616c272d194153b5/src/core/server/elasticsearch/status.ts#L20-L27

Which means that for some reason, esNodesCompatibility$ did not emit yet, or, more likely, the emission was not propagated to the observable used by the /api/status endpoint when we're reaching this line:

https://github.com/elastic/kibana/blob/fc7d3850bd21ddb23cab50f41f16822147c61be7/src/core/server/elasticsearch/integration_tests/client.test.ts#L94

Looking at the code, we have some debounces in the observable chain, e.g this one:

https://github.com/elastic/kibana/blob/9d5aca591b08f93729b702bcd47311cb33e227a8/src/core/server/status/status_service.ts#L72-L74

and we're using a combineLatest operator for the /status/api route:

https://github.com/elastic/kibana/blob/9d5aca591b08f93729b702bcd47311cb33e227a8/src/core/server/status/routes/status.ts#L64-L66

Given we're using a local 'fake' service and the overall quick time it takes for core to complete the setup phase, this looks like a simple race issue.

[kibanamachine]

New failure: kibana-7-dot-17-es-8-dot-14-forward-compatibility - 7.17

[afharo]

@gsoldevila, do you think that we can backport https://github.com/elastic/kibana/pull/148616 to 7.17? It looks like it's failing in that branch.

[gsoldevila]

In progress https://github.com/elastic/kibana/pull/190158

[afharo]

Backport merged (thanks @gsoldevila)! Closing this issue.