[🚀 Feature]: Provide end to end Observability on Selenium via agent automatically

[alaahong]

Feature and motivation

As a project owner, I want to observe the whole end to end flow in the scenario level, So just raise the ticket to seek the possible on this.

Background: In a normal project, it's easy to get full trace flow, just need start via same agent and send to same jaeger. no need set additional span or metric, just use as automatically way.

https://github.com/open-telemetry/opentelemetry-java-instrumentation/releases/download/v1.15.0/opentelemetry-javaagent.jar
https://github.com/alaahong/sia4-appium-sample/releases/download/opentelemetry-1/opentelemetry-1.jar
https://github.com/alaahong/sia4-appium-sample/releases/download/opentelemetry-2/opentelemetry-2.jar

But we can't put Selenium into above flow with agent automatically too.

1. The Selenium trace record can't merge into Java Service automatically.
2. Even in Selenium, the trace will be splitting as request level, can't work as the scenario level.

So do we have the possible to combine Selenium OpenTelemetry evidence within transaction level (which means whole end to end execution, just like the java service client embed into java service host) or session level (which means Selenium execution level, not split into per request)? Might be customized span can do this, but no code change is better.

Usage example

Environment: opentelemetry-javaagent 1.15 Java 17 Windows 11 Selenium and Selenium Grid 4.3.0

Start jaeger

docker run -d --name jaeger   -e COLLECTOR_ZIPKIN_HOST_PORT=:9411   -e COLLECTOR_OTLP_ENABLED=true   -p 6831:6831/udp   -p 6832:6832/udp   -p 9778:5778   -p 16686:16686   -p 4317:4317   -p 4318:4318   -p 14250:14250   -p 14268:14268   -p 14269:14269   -p 9411:9411   jaegertracing/all-in-one:1.36

Start java service

java  -javaagent:"opentelemetry-javaagent.jar"  -D"otel.traces.exporter"="jaeger" -D"otel.exporter.jaeger.endpoint"="http://localhost:14250" -D"otel.resource.attributes"="service.name=java-service-client" -jar opentelemetry-1.jar --server.port=8081 --target-uri="http://localhost:8082/api/hello?name="

java  -javaagent:"opentelemetry-javaagent.jar"  -D"otel.traces.exporter"="jaeger" -D"otel.exporter.jaeger.endpoint"="http://localhost:14250" -D"otel.resource.attributes"="service.name=java-service-host" -jar opentelemetry-2.jar --server.port=8082 --"target-uri"="http://localhost:8081/api/hello?name="

Start Grid with same agent and jaeget

java   -javaagent:"opentelemetry-javaagent.jar"  -D"otel.traces.exporter"="jaeger" -D"otel.exporter.jaeger.endpoint"="http://localhost:14250" -D"otel.resource.attributes"="service.name=selenium-standalone"  -jar selenium-server-4.3.0.jar  --ext $(coursier fetch -p io.opentelemetry:opentelemetry-exporter-jaeger:1.15.0 io.grpc:grpc-netty:1.45.0) standalone --tracing false

Test Code

public class OpenTelemetryTest {

  protected WebDriver driver;

  @BeforeAll
  static void setupClass() {
    WebDriverManager.chromedriver()
            .setup();
  }

  @BeforeEach
  void setupTest() throws MalformedURLException {
    ChromeOptions options = new ChromeOptions();
    driver = new RemoteWebDriver(new URL("http://localhost:4444"), options);
  }

  @Test
  public void openTelemetryTest() {
    driver.get("http://localhost:8081/");
    driver.findElement(By.id("hello")).sendKeys("test");
    driver.findElement(By.xpath("//input[@type='submit']")).click();
    Assertions.assertThat(driver.findElement(By.tagName("h1")).getText())
            .isEqualTo("Hello TEST");
  }

  @AfterEach
  void teardown() {
    if (driver != null) {
      driver.quit();
    }
  }
}

[github-actions[bot]]

@alaahong, thank you for creating this issue. We will troubleshoot it as soon as we can.

---

Info for maintainers

Triage this issue by using labels.

If information is missing, add a helpful comment and then I-issue-template label.

If the issue is a question, add the I-question label.

If the issue is valid but there is no time to troubleshoot it, consider adding the help wanted label.

If the issue requires changes or fixes from an external project (e.g., ChromeDriver, GeckoDriver, MSEdgeDriver, W3C), add the applicable G-* label, and it will provide the correct link and auto-close the issue.

After troubleshooting the issue, please add the R-awaiting answer label.

Thank you!

[diemol]

I thought this was already possible. Do you know if this already works, @pujagani?

[pujagani]

We do have end-to-end observability. But as per the discussion with @alaahong in our Slack channel, there was a missing piece or an error when trying to use opentelemetry-javaagent.jar to achieve the same. Hence, I requested him to add an issue so we can track it and help fix it (Thank you @alaahong !). I suspect this is due to the OpenTelemetry's autoconfiguration mechanism that we are using. But will need to dig deeper.

[pujagani]

@alaahong I was revisiting this issue and based on the description it seems like https://github.com/SeleniumHQ/selenium/issues/9943 might help in what you want to achieve. I think the agent is not the key here, the key is injecting the context of the service running into calling the dependent service (Selenium). The same context will ideally stitch together all the traces. Traces by nature are at the request level, which is what Selenium provides by default. Am I missing something here?

[alaahong]

Hi @pujagani , thanks for your following. You're right, I might get the incorrect direction previously. The previous proposal is to automatedly compositing all actions via otel agent, this is caused by in development part, the e2e scenario is often from complex request and response, so we can meet a series span in one trace. After you left above comment and I took more tracing practice recently, seems as you mentioned, Selenium just trace as per request, it doesn't know the outside context if I didn't manually set traceId between different request.

Here is a sample for detail,

Start jaeger and selenium-server with opentelemetry way

java -javaagent:opentelemetry-javaagent.jar -Dotel.service.name=selenium-server -Dotel.metrics.exporter=none -Dotel.logs.exporter=none  -jar .\selenium-server-4.7.2.jar standalone --log-level FINE --tracing false

.\jaeger-all-in-one.exe --collector.otlp.enabled 

Visit a webpage then do something in one case(method)

public class ObservabilityTest {
  WebDriver webDriver = null;
  ChromeOptions chromeOptions = new ChromeOptions();

  @BeforeEach
  void setup() throws MalformedURLException {
    webDriver = new RemoteWebDriver(new URL("http://192.168.116.1:4444/"), chromeOptions,false);
  }

  @Test
  void tracingTest() {
    webDriver.get("http://localhost:8080/");
    webDriver.findElement(By.tagName("body"));
  }

  @AfterEach
  void teardown() {
    if (Objects.nonNull(webDriver)) {
      webDriver.quit();
    }
  }
}

Previsouly, I hope the tracingTest can be binding as one trace automatedly via agent, actually, I only can get below series of trace in jaeger part even they are from same method in Java side.

So, I have to find out a way to set the series actions under same trace and become children spans, otherwise, they only can be presenting by request level, right ?

[alaahong]

One more question here, does Selenium really support otlp at this moment? From other issues we can find out that almost tracing samples are based on jaeger even under opentelemetry configuration.

As my above code, I can work with opentelementry agent after disable selenium tracing, but I still can't find a sample to work within Selenium + otlp . Whatever to add below setProperty, I can't find out any trace in Jaeger UI for "selenium-client" if enabled selenium tracing and remove additional outside opentelementry-agent

  @Test
  void tracingTest() {
    System.setProperty("OTEL_TRACES_EXPORTER", "otlp");
    System.setProperty("OTEL_METRICS_EXPORTER", "none");
    System.setProperty("OTEL_TRACES_EXPORTER", "none");
    System.setProperty("OTEL_SERVICE_NAME","selenium-client");
    webDriver.get("http://localhost:8080/");
    webDriver.findElement(By.tagName("body"));
  }

Previously, I was working with Grafana stack (loki+tempo) , so I have the possible to combine traces, metrics and logs within one stack, seems I only can get the trace support in Jaeger? And do we have the possible to have the sample based on otlp?

[github-actions[bot]]

This issue is stale because it has been open 280 days with no activity. Remove stale label or comment or this will be closed in 14 days.

[pujagani]

We support tracing in Selenium to provide observability into the request going between various components using OpenTelemetry. The documentation uses jaeger as just an example to visualize traces. OpenTelemetry allows this to be pluggable via system properties, so feel free to use the trace exporter of your choice. We only provide the traces. This is useful when using the Grid in Hub and Node or Fully Distributed mode. I am afraid I don't fully understand how Selenium can support the OTLP protocol.

[pujagani]

Do you mean if we support "otlp" as an exporter? Ideally, we should support whatever OpenTelemetry allows users to configure because we don't do anything differently or specific to the exporter.

[pujagani]

"And do we have the possible to have the sample based on otlp"

I don't think this is a priority currently. But we are happy to receive contributions around it!

[pujagani]

Also, are you using RemoteWebDriver? Java client-side observability is only supported with RemoteWebDriver.

[alaahong]

Hi @pujagani , thanks for your following, I tried to find a simple way to clear my thoughts, here is the post questions.

Test Code

#Selenium Test

  @BeforeEach
  void setupTest() throws MalformedURLException {
    ChromeOptions options = new ChromeOptions();
    driver = new RemoteWebDriver(new URL("http://localhost:4444"), options, false);
  }
  @Test
  public void openTelemetryTest() {
    driver.get("http://localhost:8088/api/now");
    Assertions.assertThat(driver.getPageSource()).isNotEmpty();
  }
  @AfterEach
  void teardown() {
    if (driver != null) {
      driver.quit();
    }
  }

#Selenium Grid
java   -javaagent:tools\opentelemetry-javaagent.jar -Dotel.logs.exporter=none -Dotel.metrics.exporter=none -Dotel.traces.exporter=otlp -Dotel.resource.attributes=service.name=selenium-standalone  -jar selenium-server-4.15.0.jar standalone --tracing false

#SUT  

@SpringBootApplication
@RestController
@RequestMapping("/api")
public class HelloApplication {

  public static void main(String[] args) {
    SpringApplication app = new SpringApplication(HelloApplication.class);
    app.setBannerMode(Banner.Mode.OFF);
    app.run(args);
  }

  @RequestMapping(value = "/now", method = {RequestMethod.GET, RequestMethod.POST})
  public String now() {
    System.out.println(new RestTemplate()
            .getForObject("http://localhost:8088/api/hello?name=name", String.class));
    return LocalDateTime.now().toString();
  }

  @RequestMapping(value = "/hello", method = {RequestMethod.GET, RequestMethod.POST})
  public String hello(@RequestParam(value = "name") String name) {
    return "hello " + name;
  }
}

OpenTelemetry Agent Setting

1. As the given sample is based on Mac and Jaeger, I tried to seek the possible to execute Selenium with OpenTelemetry (OTLP) and Tempo (another tool from Grafana similar as Jaeger), but have no idea on this. The only way I tried excluding Jaeger is to disable Selenium Trace then start the test with Open Telemetry agent locally, but also failed in Docker-Selenium.

So, do we have the possible to do the tracing but excluding any jaeger dependencies or natively export to OTLP ?

2. There are three service, in the test flow [1. selenium-test : test code ; 2. selenium-standalone: selenium grid; 3. hello: sut] ,they are all have same otlp setting, As expected, they should pass the trace id end to end ,but actually, as below screenshot, we can't find out them together in one trace. I had checked the different from previous successful result as #9943 , seems they just manually set the span in code, but my sample is based on Automatic Instrumentation , do you have any idea to auto pass the trace, otherwise any gap from my code?

3. About the issue original question, it's asking for integrating with several Selenium trace within one. For example, Selenium is used to work as Integration Test, so it comes with some scenarios, not the one by one request. In selenium trace, we can find out per action has separate trace,

e.g. below code won't be tracing into one

  @Test
  public void openTelemetryTest() {
    driver.get("http://localhost:8088/api/now");
    Assertions.assertThat(driver.getPageSource()).isNotEmpty();
  }

But in Application part, we can find out the trace will handle all in the transaction level as below which means, if A call B then B call C, then we have the possible to find out them in one trace together. This might not the important part on Selenium usage, just wonder if any way to observe the Selenium on the end to end level.

Thanks.

[alaahong]

We support tracing in Selenium to provide observability into the request going between various components using OpenTelemetry. The documentation uses jaeger as just an example to visualize traces. OpenTelemetry allows this to be pluggable via system properties, so feel free to use the trace exporter of your choice. We only provide the traces. This is useful when using the Grid in Hub and Node or Fully Distributed mode. I am afraid I don't fully understand how Selenium can support the OTLP protocol.

If Trace in Selenium only means internal level, seems already there, Selenium can show all in self. Above comments are all trying to make Selenium be part of full end to end, treat as one service in whole flow.

[titusfortner]

@pujagani / @diemol is there something actionable for us with this?

[diemol]

I do not understand fully what is needed here. @pujagani , do you?

[pujagani]

Based on the questions, I dont think there is an action item on us. I am going to try and answer all concerns point by point here.

1. Jaeger is used as just an example to visualize traces. OpenTelemetry allows this to be pluggable via system properties, so feel free to use the trace exporter of your choice. If using a particular trace exporter does not work and if that is due to some missing configuration in Selenium, only then can we help.
2. As you mentioned, I know the manual way to pass in the trace ID and stitch that together. Please ask for help in OpenTelemetry-related forums regarding why Automatic instrumentation is not working. Based on the documentation, the jar should pick up the id and use it. Also, tracing is set to false while start the selenium grid. That might have an impact.
3. Ideally, tracing is done at a request level in complicated systems to keep track of the HTTP request lifecycle through various services (mainly microservices architecture makes this super useful). So Selenium emits trace at the request level since tracing is intended for it.

[github-actions[bot]]

This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.