smoke: Get elastic agent and apm version scripts failing

[isaacaflores2]

The scripts to get elastic agent and apm server versions occasionally fail with the below error. This happens since the artifacts-api.elastic.co is not always available. I have observed this API not being available for a few minutes at a time.

-> Applying terraform configuration...

Error: External Program Execution Failed

  with module.standalone_apm_server.data.external.latest_elastic_agent,
  on ../../infra/terraform/modules/standalone_apm_server/main.tf line 285, in data "external" "latest_elastic_agent":
 285:   program = ["bash", "${path.module}/latest_elastic_agent.sh", "${var.stack_version}"]

The data source received an unexpected error while attempting to execute the
program.

Program: /usr/bin/bash
Error Message: jq: parse error: Invalid numeric literal at line 1, column 7

State: exit status 5

The observability robots team has mentioned we may be able to use another more reliable API: https://storage.googleapis.com/artifacts-api. We can explore if this API can replace artifacts-api.elastic.co

[simitt]

Switching to https://storage.googleapis.com/artifacts-api sounds reasonable. We'll need to look into how to easily get the version needed. By default the API returns several very old versions. The response of https://storage.googleapis.com/artifacts-api/releases.properties doesn't look quite right to me; I'd expect next_minor_9: 9.1.0at this point andnext_patch_9: 9.0.2`, but they are pointing to the same version (similar for other versions):

current_6=6.8.23
current_7=7.17.28
current_8=8.18.2
current_9=9.0.1
next_minor_7=7.17.29
next_patch_7=7.17.29
next_minor_8=8.18.3
next_patch_8=8.18.3
next_minor_9=9.0.2
next_patch_9=9.0.2
edge_8=8.19.0
edge_9=9.1.0
generated=https://github.com/elastic/observability-robots/actions/workflows/generate-elastic-stack-releases.yml

[isaacaflores2]

I found two other APIs we can use:

1. https://snapshots.elastic.co/latest/8.19.json
2. https://artifacts-snapshot.elastic.co/apm-server/latest/8.19.json

I am currently validating https://snapshots.elastic.co