Feature Request: Missing trace for JUnit5 BeforeAll/AfterAll methods in CIVisibility Instrumentation

[sergeidyga]

trafficstars

Hello team 👋

I'm using JUnit5 Instrumentation with following config:

dd.civisibility.enabled=true
dd.civisibility.agentless.enabled=true

Following class:

import datadog.trace.api.CorrelationIdentifier
import org.junit.jupiter.api.AfterAll
import org.junit.jupiter.api.AfterEach
import org.junit.jupiter.api.BeforeAll
import org.junit.jupiter.api.BeforeEach
import org.junit.jupiter.api.Test

class TestSuite {

    @BeforeAll
    fun beforeAll() {
        println("beforeAll " + CorrelationIdentifier.getTraceId())
    }

    @BeforeEach
    fun beforeEach() {
        println("beforeEach " + CorrelationIdentifier.getTraceId())
    }

    @Test
    fun test() {
        println("test " + CorrelationIdentifier.getTraceId())
    }

    @AfterEach
    fun afterEach() {
        println("afterEach " + CorrelationIdentifier.getTraceId())
    }

    @AfterAll
    fun afterAll() {
        println("afterAll " + CorrelationIdentifier.getTraceId())
    }

}

Gives following output:

beforeAll 0
beforeEach 8692252888825984513
test 8692252888825984513
afterEach 8692252888825984513
afterAll 0

I'm curious why active trace is missing in beforeAll and afterAll? I was expecting TestSession and TestModule spans to be already created at this point by CIVisibility auto-instrumentation.

Beside this main question I also noticed that beforeEach and afterEach spans are not visible in DD UI (only the test span).

Context: JUnit5 Lifecycle methods (methods annotated with @BeforeAll, @AfterAll, @BeforeEach, or @AfterEach that are used for setting up and tearing down tests) are significant subjects for tracing because, like test methods, they often involve network requests, database queries, and similar operations.

Thank you for looking into this!

[sergeidyga]

I've updated the description - turns out the trace is only missing for BeforeAll/AfterAll methods

[nikita-tkachenko-datadog]

Hi @sergeidyga, thank you for reporting this!

You're right that TestSession and TestModule spans already exist by the time a method marked with@BeforeAll is executed. Moreover, TestSuite exists as well, so it'd make sense to have spans for these lifecycle methods as children of their respective suite. The problem is that to have a suite child that is not a test case a few things have to be changed both in the tracer and in the Datadog backend. We've had this feature in our backlog for a while, unfortunately there are no plans to support it in the near future (although it might change given that there is user demand).

[sergeidyga]

Hi @nikita-tkachenko-datadog,

Thank you for confirming that the TestSuite span exists by the time JUnit lifecycle methods start. But although TestSuite span exist I'm not able to get its traceId with CorrelationIdentifier.getTraceId()? So I'm I doubt how to access a suite span in order to build a test span as its child?

Thanks!

[nikita-tkachenko-datadog]

Hi @sergeidyga, apologies for the delay.

I have managed to create an ad-hoc test span using hints from Datadog APM's Opentracing instrumentation guide and custom instrumentation guide.

It is somewhat hacky as you have to use Datadog's internal API alongside Opentracing API to create the span the way it should be (Test Visibility backend is rather opinionated and expects a very particular structure and set of tags).

Anyways, below are the steps:

1. Add Opentracing and Datadog APIs as dependencies, for example for Maven:

        <!-- OpenTracing API -->
        <dependency>
            <groupId>io.opentracing</groupId>
            <artifactId>opentracing-util</artifactId>
            <version>0.32.0</version>
        </dependency>
        <!-- Datadog API -->
        <dependency>
            <groupId>com.datadoghq</groupId>
            <artifactId>dd-trace-api</artifactId>
            <version>1.34.0</version>
        </dependency>

2. Create a utility method that constructs the ad-hoc test span as a child of current suite span:

    public static <T> T traceAsTest(String name, Supplier<T> action) {
        Span test = null;
        MutableSpan mutableTest = null;
        Tracer globalTracer = GlobalTracer.get();
        Span suite = globalTracer.activeSpan();
        if (suite instanceof MutableSpan) {
            MutableSpan mutableSuite = (MutableSpan) suite;

            test = globalTracer.buildSpan("junit.test").ignoreActiveSpan().start();
            if (test instanceof MutableSpan) {
                mutableTest = (MutableSpan) test;
                mutableTest.setResourceName(mutableSuite.getTag("test.suite") + "." + name);
                mutableTest.setSpanType("test");
                mutableTest.setTag(DDTags.ORIGIN_KEY, "ciapp-test");
                mutableTest.setTag("span.kind", "test");
                mutableTest.setTag("test.type", "test");
                mutableTest.setTag("test.name", name);
                mutableTest.setTag("component", "junit");
                mutableTest.setTag("test.suite", String.valueOf(mutableSuite.getTag("test.suite")));
                mutableTest.setTag("test.module", String.valueOf(mutableSuite.getTag("test.module")));
                mutableTest.setTag("test_suite_id", (Number) mutableSuite.getTag("test_suite_id"));
                mutableTest.setTag("test_module_id", (Number) mutableSuite.getTag("test_module_id"));
                mutableTest.setTag("test_session_id", (Number) mutableSuite.getTag("test_session_id"));
                mutableTest.setTag("test.status", "pass");
            }
        }

        try {
            return action.get();
        } catch (Exception e) {
            if (mutableTest != null) {
                mutableTest.setError(true);
                StringWriter errorString = new StringWriter();
                e.printStackTrace(new PrintWriter(errorString));
                mutableTest.setTag(DDTags.ERROR_STACK, errorString.toString());
                mutableTest.setTag("test.status", "fail");
            }
            throw e;
        } finally {
            if (test != null) {
                test.finish();
            }
        }
    }

3. Use the method to trace @BeforeAll, @AfterAll or any other piece of your code:

    @BeforeAll
    public static void setUpClass() {
        traceAsTest("beforeAll", () -> {
            // ...
        });
    }

If you follow the steps above you'll see your before all method as yet another test case in Datadog.

[sergeidyga]

@nikita-tkachenko-datadog I appreciate the detailed answer 🙇‍♂️. That is exactly how I tried to solve it before your reply. But session was still not there. And, blind me, the cause of missing session was the outdated version of dd-java-agent I was using - 1.5.0 !! 😅