opentelemetry-java-instrumentation Help wanted! Convert Groovy tests to Java

We've made some initial progress in #7148, #6613, #6537, #6497, #6484, #6486, #6485, #5774, #5688, #5679, #5668, #5660, #5596, #5581, #5521, #5504, #5399, #3768, #3525 (among others)

But there's so much more! 😅

Pick your favorite(s) and help us convert one or two (or ten!)

[x] apache-camel-2.20 #8813
[x] apache-dubbo-2.7 #12008
[x] apache-httpclient/apache-httpclient-2.0 #12102
[x] apache-httpclient/apache-httpclient-4.0 #7912
[x] async-http-client/async-http-client-1.9 #8954
[x] async-http-client/async-http-client-2.0 #8954
[ ] aws-sdk/aws-sdk-1.11 #10657 #10660
[ ] aws-sdk/aws-sdk-2.2 #11240
[x] cassandra/cassandra-3.0 #7390
[x] cassandra/cassandra-4.0 #7390
[x] couchbase/couchbase-2-common #9426
[x] couchbase/couchbase-2.0 #12193
[x] couchbase/couchbase-2.6 #12193
[x] couchbase/couchbase-3.1 #10880
[x] couchbase/couchbase-3.1.6 #12193
[x] couchbase/couchbase-3.2 #8837
[x] dropwizard/dropwizard-views-0.7 #8808
[x] elasticsearch/elasticsearch-rest-5.0 #8542
[x] elasticsearch/elasticsearch-rest-6.4 #8524
[x] elasticsearch/elasticsearch-rest-7.0 #8799
[ ] elasticsearch/elasticsearch-transport-5.0
[ ] elasticsearch/elasticsearch-transport-5.3
[ ] elasticsearch/elasticsearch-transport-6.0
[x] external-annotations #9557
[x] geode-1.4 #8141
[x] grails-3.0
[ ] grizzly-2.0 #12248
[x] gwt-2.0 #8201
[x] hibernate/hibernate-3.3 #9025
[x] hibernate/hibernate-4.0 #9211 #9249 #9282
[x] hibernate/hibernate-6.0 #8304
[x] hibernate/hibernate-procedure-call-4.3 #9290
[x] http-url-connection #7676
[x] hystrix-1.4 #9217
[x] internal/internal-class-loader #9560
[x] internal/internal-lambda #9611
[x] internal/internal-reflection #9612
[x] internal/internal-url-class-loader #9614
[x] java-util-logging #7631
[ ] jaxrs/jaxrs-1.0
[ ] jaxrs/jaxrs-2.0/jaxrs-2.0-annotations
[ ] jaxrs/jaxrs-2.0/jaxrs-2.0-cxf-3.2
[ ] jaxrs/jaxrs-2.0/jaxrs-2.0-jersey-2.0
[ ] jaxrs/jaxrs-2.0/jaxrs-2.0-resteasy-3.0
[ ] jaxrs/jaxrs-2.0/jaxrs-2.0-resteasy-3.1
[ ] jaxrs/jaxrs-3.0/jaxrs-3.0-annotations
[ ] jaxrs/jaxrs-3.0/jaxrs-3.0-jersey-3.0
[ ] jaxrs/jaxrs-3.0/jaxrs-3.0-resteasy-6.0
[ ] jaxrs-client/jaxrs-client-1.1
[ ] jaxws/jaxws-2.0
[ ] jaxws/jaxws-2.0-axis2-1.6
[ ] jaxws/jaxws-2.0-cxf-3.0
[ ] jaxws/jaxws-2.0-metro-2.2
[ ] jaxws/jaxws-jws-api-1.1
[x] jboss-logmanager/jboss-logmanager-appender-1.1 #7703
[x] jboss-logmanager/jboss-logmanager-mdc-1.1 #7703
[x] jdbc #11178
[x] jedis/jedis-1.4 #7731
[x] jedis/jedis-3.0 #7731
[x] jedis/jedis-4.0 #7731
[x] jetty/jetty-11.0 #7976
[x] jetty/jetty-8.0 #7975
[x] jetty-httpclient/jetty-httpclient-9.2 #9072
[x] jms-1.1 #9476
[x] jsf/jsf-mojarra-1.2 #11711
[x] jsf/jsf-mojarra-3.0 #11694
[x] jsf/jsf-myfaces-1.2 #11711
[x] jsf/jsf-myfaces-3.0 #11694
[x] jsp-2.3 #11827
[x] kafka/kafka-clients/kafka-clients-0.11 #7770
[ ] kafka/kafka-streams-0.11
[x] kubernetes-client-7.0 #9163
[x] lettuce/lettuce-4.0 #9419
[x] lettuce/lettuce-5.0 #9547
[x] lettuce/lettuce-5.1 #9876 #9994 #1004
[x] log4j/log4j-appender-1.2 #7631
[x] log4j/log4j-appender-2.17 #7631
[x] log4j/log4j-context-data/log4j-context-data-2.17 #9804
[x] log4j/log4j-context-data/log4j-context-data-2.7 #9804
[x] log4j/log4j-mdc-1.2 #8170
[x] logback/logback-mdc-1.0 #9490
[ ] mongo/mongo-3.1
[ ] mongo/mongo-3.7
[ ] mongo/mongo-4.0
[ ] mongo/mongo-async-3.3
[x] netty/netty-3.8 #7666
[x] netty/netty-4.0 #9976
[x] netty/netty-4.1 #12108
[x] opentelemetry-api/opentelemetry-api-1.0 #12133
[x] opentelemetry-extension-annotations-1.0
[x] opentelemetry-instrumentation-annotations-1.16 #7781
[ ] play/play-mvc/play-mvc-2.4
[ ] play/play-mvc/play-mvc-2.6
[ ] play/play-ws/play-ws-1.0
[ ] play/play-ws/play-ws-2.0
[ ] play/play-ws/play-ws-2.1
[x] rabbitmq-2.7 #9562
[ ] ratpack/ratpack-1.4
[x] reactor/reactor-netty/reactor-netty-0.9 #7635
[ ] rediscala-1.8
[x] redisson/redisson-3.0 #9060
[x] redisson/redisson-3.17 #9060
[ ] restlet/restlet-1.1
[ ] restlet/restlet-2.0
[x] rocketmq/rocketmq-client/rocketmq-client-4.8 #10520
[x] runtime-metrics #8928
[x] rxjava/rxjava-2.0 #8397
[x] rxjava/rxjava-3.0 #7924
[x] rxjava/rxjava-3.1.1 #7924
[ ] servlet/servlet-2.2 #12013
[ ] servlet/servlet-3.0 #12013
[ ] servlet/servlet-5.0
[x] spark-2.3 #8200
[x] spring/spring-batch-3.0 #12004
[x] spring/spring-core-2.0 #8166
[x] spring/spring-data/spring-data-1.8 #8124
[x] spring/spring-integration-4.1 #11999
[x] spring/spring-jms/spring-jms-2.0 #11195
[x] spring/spring-rabbit-1.0
[x] spring/spring-rmi-4.0
[x] spring/spring-scheduling-3.1 #9806
[x] spring/spring-webflux-5.0 #9677
[x] spring/spring-webmvc/spring-webmvc-3.1 #11114
[x] spring/spring-webmvc/spring-webmvc-6.0 #11114
[x] spring/spring-ws-2.0 #11196
[ ] spymemcached-2.12
[x] struts-2.3 #11468
[x] tapestry-5.4 #9684
[x] tomcat/tomcat-10.0 #11420
[x] tomcat/tomcat-7.0 #11402
[ ] twilio-6.6
[ ] undertow-1.4
[X] vaadin-14.2 #7444
[x] vertx/vertx-http-client/vertx-http-client-3.0 #10093
[ ] vertx/vertx-http-client/vertx-http-client-4.0
[ ] vertx/vertx-rx-java-3.5
[ ] vertx/vertx-web-3.0
[x] wicket-8.0 #9867

Nov 16 '22 01:11 trask

elasticsearch/elasticsearch-rest-5.0 and elasticsearch/elasticsearch-rest-6.4 already have Java tests.

Moved tests to Java for elasticsearch/elasticsearch-rest-7.0 as part of this open PR:

open-telemetry/opentelemetry-java-instrumentation#8799.

Jun 24 '23 11:06 AlexanderWert

I'm working on converting apache-camel-2.20

Jun 25 '23 15:06 jaydeluca

it was a rainy day here, so I wrote a little script to track the progress of this project from the time this issue was opened

example_output

Jul 16 '23 21:07 jaydeluca

I plan to work on the remaining hibernate tests (hibernate-3-3 and hibernate-4-0) if no one else is already working on them

Jul 21 '23 12:07 jaydeluca

Progress update, getting close to under 400 remaining files:

2023-09-18-otel-update

Also, just as a note, the following are also completed and can be updated on the original issue checklist:

hibernate/hibernate-3.3 - #9025
hystrix-1.4 (just needs to be checked off, PR already linked)
couchbase-2-common - #9426
kubernetes-client-7.0 - #9163

Sep 18 '23 11:09 jaydeluca

The issue has been open for about a year now, here's the latest progress report:

latest

Nov 16 '23 12:11 jaydeluca

Just tried both IDEA and copilot for converting, but neither was able to do it. Nor chatGPT. This is the test case: https://github.com/open-telemetry/opentelemetry-java-instrumentation/blob/main/instrumentation/spring/spring-batch-3.0/javaagent/src/test/groovy/SpringBatchTest.groovy

Dec 21 '23 14:12 zeitlinger

in my travels I've come across a few that can be checked off in the checklist:

elasticsearch/elasticsearch-rest-5.0 #8542
elasticsearch/elasticsearch-rest-6.4 #8524
elasticsearch/elasticsearch-rest-7.0 (i think this was always Java: #8911 )
opentelemetry-extension-annotations-1.0 (I couldn't find a conversion PR for this one)

Feb 24 '24 12:02 jaydeluca

Latest progress update:

May 16 '24 10:05 jaydeluca

I found a few more that we can check off the list:

[x] tomcat/tomcat-10.0 https://github.com/open-telemetry/opentelemetry-java-instrumentation/pull/11420
[x] tomcat/tomcat-7.0 https://github.com/open-telemetry/opentelemetry-java-instrumentation/pull/11402
[x] netty/netty-4.0 https://github.com/open-telemetry/opentelemetry-java-instrumentation/pull/9976
[x] spring/spring-jms/spring-jms-2.0 https://github.com/open-telemetry/opentelemetry-java-instrumentation/pull/11195
[x] rabbitmq-2.7 https://github.com/open-telemetry/opentelemetry-java-instrumentation/pull/9562
[x] spring/spring-rmi-4.0 (i'm not sure if there were ever groovy tests, wasn't able to find a PR)

Jun 29 '24 10:06 jaydeluca

Progress Update

We continue to burn down the groovy test files, and at the same time we can see further expansion of the java tests as more instrumentation is added:

We are approaching the milestone of less than 300 remaining files, with over 250 files converted thus far:

Here is a breakdown of the remaining instrumentation modules to be converted:

Instrumentation	Files to be converted
spring	52
jaxrs	37
servlet	23
restlet	22
couchbase	18
aws-sdk	17
ratpack	16
elasticsearch	15
play	15
jaxws	15
vertx	14
mongo	10
jdbc	8
apache-dubbo-2.7	7
jaxrs-client	5
netty	5
apache-httpclient	3
opentelemetry-api	3
grizzly-2.3	3
grails-3.0	3
undertow-1.4	3
kafka	3
internal	2
dropwizard	2
hibernate	1
rediscala-1.8	1
spymemcached-2.12	1
twilio-6.6	1
Total	305

Aug 10 '24 14:08 jaydeluca

I'll give (some) spring tests a shot

Aug 12 '24 11:08 zeitlinger

---------------------------------------------------------------------------------
Language                       files          blank        comment           code
---------------------------------------------------------------------------------
XSD                              406          27232           9758         293467
Java                            4065          46854          34176         248814
HTML                             553           1252           1580         152058
XML                             1181           1800           9778         126077
JSON                              62             50              0          92945
Kotlin                          1526          21083          59440          77275
Groovy                           361           6013           2643          33632

Aug 17 '24 12:08 zeitlinger

@zeitlinger I think you should exclude build directories

Aug 17 '24 13:08 laurit

good point

~/opt/cloc-2.02/cloc --match-d='/(src)/' .
    4728 text files.
    4612 unique files.                                          
     118 files ignored.

github.com/AlDanial/cloc v 2.02  T=4.51 s (1021.5 files/s, 84070.5 lines/s)
---------------------------------------------------------------------------------
Language                       files          blank        comment           code
---------------------------------------------------------------------------------
Java                            3962          45180          33339         237806
Groovy                           360           5992           2323          33562
YAML                              16             44              5           6394
Scala                             34            399            213           2865
Kotlin                            39            481            288           2520
XML                               84            399             80           2158
JSON                               6             49              0           2091
Gradle                            24            294            355           1664

Aug 17 '24 14:08 zeitlinger

we've got some serious momentum lately, and we're now below 200 files remaining. thanks @zeitlinger @laurit @shalk and @xiepuhuan!

Remaining modules by file count percentage:

instrumentation list sorted by sum of the file sizes:

Key	File Count	(Sum) File Size
aws-sdk	17	127827
jdbc	8	90890
jaxrs	37	72352
servlet	10	65662
vertx	14	56845
ratpack	16	51932
mongo	10	51661
play	15	48986
restlet	22	38226
jaxws	17	21595
kafka	3	21505
twilio-6.6	1	18936
jaxrs-client	5	16067
spymemcached-2.12	1	15630
undertow-1.4	3	12754
hibernate	1	12167
dropwizard	2	10789
rediscala-1.8	1	3898
grails-3.0	3	3201
internal	2	2603
Total	188	743526

Sep 25 '24 23:09 jaydeluca

using cloc

---------------------------------------------------------------------------------
Language                       files          blank        comment           code
---------------------------------------------------------------------------------
Java                            4222          48133          34902         254323
Groovy                           234           3698           1488          21169

Sep 26 '24 11:09 zeitlinger

opentelemetry-java-instrumentation opentelemetry-java-instrumentation copied to clipboard

Help wanted! Convert Groovy tests to Java

Progress Update

opentelemetry-java-instrumentation
opentelemetry-java-instrumentation copied to clipboard