Add tests to check data ingestion in tobs

[VineethReddy02]

As currently, tobs includes all the components i.e. Prometheus, Otel collector, and Promscale that are required to test the ingestion, and querying of data.

As part of this issue add nightly tests to make sure the recommended config with the latest Otel-collector, Prometheus and Promscale works as expected with data ingestion of metrics and traces.

[paulfantom]

Slack discussion was not about adding nightly tests, but about adding nightly release channel for promscale (which is a blocker for this task) and then e2e testing if ingestion through otel-collector works.

Prerequisites for starting this task:

• promscale nightly (or master) container image builds
• (most likely) prometheus-operator support for https://prometheus.io/docs/prometheus/latest/configuration/configuration/#tracing_config

DoD:

• ~prometheus is configured to send traces to promscale via otel-collector~
• e2e test suite checks if all components of the stack are up and running
• ~e2e test suite checks otel-collector is able to forward data~
• e2e tests are run on every merge to master

What is NOT required:

• setting up nightly testing channel.

[paulfantom]

Related to https://github.com/timescale/tobs/issues/288

[paulfantom]

We can use up application to test the data flow (cc @onprem)