rest
rest copied to clipboard
jakartaee
TCK migration into JAX-RS repository
At the moment the TCK is one big piece of code for all parts of Jakarta EE. Once the PMC allows us to do that, we should provide a concerted pair of PRs for the migration of the JAX-RS-piece of the TCK:
- [ ] PR to insert into JAX-RS repo
- [ ] PR to eliminate in Jakarte EE repo
I fully agree. But I think it will take some time before we can start working on this. The EE4J Projects - Technical Direction document states:
It will require some PMC guidance and community input about how new TCK tests are organized, what testing frameworks are used, etc.
IMO it would make sense that the PMC starts this discussion early and before the TCK is split up.
But of course we can also start some discussion about how we want to handle the TCK. Do we want to keep it in the same repo as the spec itself?
My experience of the past year is that the PMC sometimes needs to get pushed into discussions, so let's just start here. ;-)
I would say it is advisory to only have one repository per API, so in the case of JAX-RS I would say let's work towards a long-term structure like this one...
/api
/examples
/spec
/tck
...and publish not only the api on Maven Central but also the tck and spec. It might sound visionary, but I think it would be great if in some far future implementors of JAX-RS could simply have the tck as Maven dependency which runs on mvn integration-test, or that application programmers could tell their IDE to download and view the spec PDF for a particular API just as it already works with the source and Javadoc JARs. :-)
Ok, let's push the PMC a bit. :laughing:
I would say it is advisory to only have one repository per API
Agreed. It would be very beneficial to have everything in one place. Especially because changes to the API usually also affect the spec and the TCK. So pull requests should update all these components. If these would be spread across different repositories, updating all of them in a consistent way would be difficult.
It might sound visionary, but I think it would be great if in some far future implementors of JAX-RS could simply have the tck as Maven dependency which runs on
mvn integration-test
This may be a realistic option. The MVC 1.0 TCK for example is available as a Maven artifact. So if you want to run the TCK against some implementation, you just have to add the dependency on the TCK and tell Surefire to execute tests from the TCK artifact. Of course this is a bit simplified, because setting up Arquillian for the runtime you want to test is something you need to do anyway. But this step mostly depends on which server you are trying to run the tests against.
Once upon a time there was a product. The product was named Java EE CTS, aka Java EE TCK. That was a joint set of tests, some of which were part of TCKs, some were not. When another product, an application server passed the tests in Java EE CTS, the application server was Java EE compatible.
The main question is: Does the community want to have a process that marks an application server compatible with Jakarta EE and how it would look like?
Java EE CTS/Jakarta EE TCK uses a common set of libraries, dependencies, and scripts. Splitting the Jakarta EE TCK into many small TCKs possibly will:
- make every TCK use a different set of common libraries. Some TCKs will be mavenized, some will remain ant-based, some adopt Arquillian, some will continue to use JavaTest, but it would be hard to keep the versions of dependencies the same.
- will force every vendor to change their TCK/CTS pipeline to adopt the new way of configuring/running the tests
- will make it hard to run all Jakarta EE TCKs for Glassfish as well as for the vendors. It is important to decide whether the Jakarta EE TCK is needed as a product as it was back in Java EE times.
I am not saying it is a bad idea to bring the JAX-RS TCK into JAX-RS repo. I am saying it affects whole Jakarta EE comunity and it is on a broader discussion outside of JAX-RS.
I understand you sorrows, but I see some points largely different: The future is not in application servers (these are ancient relicts of the pre-cloud era), but in dynamically assembled sets of mostly independent APIs (like CDI + JAX-RS + NoSQL). Jakarta 8, possibly 9, typically will be a common API for existing application server products, but beyond that, the market will definitively strive for cloud-elastic solutions like Apache Meecrowave, Oracle Helidon, or even simpler, just standalone Jersey. Application servers sooner or later will definitively die. Sure, this might be in five or ten years, and we must support them until then. But it will be more and more essential to allow API implementors to most easily execute and contribute to API-specific test kits. This is performed best by allowing each project to develop their test kits in agile processes without being bound to other, higher level, projects.
Having said that, this discussion must start at the grass-root, as we are the future. Certainly once we find and agreement on the sub-project level, we have to share our findings and discuss in the larger community. But definitively the starting point must never be the past (app servers), it must be the future (APIs). As all EF projects are independent and self-governed by the committers, it makes no sense to decide at the big table first: If we don't want the tests, and the decide to give them to us, nobody would maintain them. If we want the tests, and they decide to not give them to us, we just fork them, and they cannot do anything against it. So it is essential to first make up our mind what we want before we got to them and tell them. They have to know what we want before they discuss.
@jansupol
I agree with you and share the same concerns.
A long time ago, before the Java EE CTS, we had tons of independent APIs that users could again supposedly dynamically assemble. "Dynamically" being a euphemism here for putting the burden of this assembling on the user.
This resulted in many Frankenstein stacks, that each were different from company to company. Skills and documentation were difficult to transfer since every stack was different, and the hard work of making sure some combination of components worked together being placed on the user caused those to invariably fail somewhere.
The power of Java EE comes from defining a coherent full stack API. Getting to a certain level of coherency has been a life long battle of Java EE. There are always people wanting to be an emperor on their own little island, and in the past this had led to different approaches to solving the same problem, and to incoherency between the different Java EE components. Just think about JAX-RS having its own component models, Servlet having its own, JSF having its own, etc etc.
This is obviously now what users want.
What users want is a full API that addresses most of their needs and where everything works seamlessly together.
During the last several years this is the direction Java EE has been moving in.
For this it's indeed necessary to have a TCK that tests the entire Java EE API. A collection of incoherent sub-project tests that all work differently and all have their own quirks is the very least we need.
I've argued this point before when people started to propose a (near) infinite number of profiles, and I've also emphasised this point by keeping as many tests in a consistent fashion at centrally at https://github.com/javaee-samples
While it would be good to be able to run tests individually, say only the JAX-RS TCK tests, we should be very careful that not all tests splinter off and essentially fade away in obscurity. Under the JCP we had the umbrella eg (the Java EE EG). Perhaps we'd need to have a good look at installing something like that again to make sure all constituent projects remain aligned and coherent.
@mkarg, @chkal
Agreed. It would be very beneficial to have everything in one place.
This seems tempting but it doesn't work well if tests should also cover integration of a spec with other specs, such as @mkarg example of JAX-RS + CDI + NoSQL. I believe that specs should work only as isolated APIs but define behavior when combined with other APIs. Maybe some basic tests that only cover the spec API without other APIs can be easily placed into the API repo, but more complete set of tests should reside in a separate repository.
It might sound visionary, but I think it would be great if in some far future implementors of JAX-RS could simply have the tck as Maven dependency which runs on mvn integration-test
I think, it would be equally visionary if it was possible to run integration tests for JAX-RS and CDI just with mvn -PCDI,JAXRS integration-test with a single Jakarta Test dependency. For plain JAX-RS, it would be equaly simple with mvn -PJAXRS integration-test. I think what you suggest with tests in the same repository are unit tests but not integration tests. It's perfectly fine if unit tests are part of the spec repository, but it doesn't make much sense for integration tests that cover integration of multiple APIs.
As an example, MicroProfile currently uses the approach of having the TCK in the API repository. And we're hitting the issue of how to write tests that ensure that multiple specs work together. E.g. MP Rest Client with MP Config, etc. The specs already describe what should happen if they are combined with other specs but it's hard to write and maintain tests for that in a single repo.
Completely understood, but easy to solve: JAX-RS stores its own test kit in the JAX-RS repo. The JAX-RS spec mentions specs which it should be compatible with. So it contains that particular compatibility tests. If the Jakarta EE spec mandates more things between two specs, that particular things have to be in the test kit stored in the Jakarta EE umbrella repository (just where the umbrella PDF sits). So still no need to have all source in one place or all tests in one kit, as Maven makes it easy to automate download of needed test dependencies.
BTW, we need to be fair to each other. Certainly what Payara customers want is (due to the fact that Payara is an application server) diametrically opposed to what Oracle Helidon customers want. There even might be more groups. So the target must be to find the best solution for all groups, not to convince us mutually that the group we speak for is "all users" or "the community". I respect Payara's situation as a maintainer of a GlassFish-derived application server, but I also respect the outcome of several community surveys which point in the post-application-server direction.
Thanks everyone for sharing your thoughts. I absolutely agree that there are many challenges and that how to handle the TCK and where to manage it is a tradeoff. Every way to do it has pros and cons. Maybe we can find some kind of compromise with a single Jakarta EE TCK repository containing shared tests and git submodules including some tests managed by the individual API project?
Just a small side note. The EE4J Projects - Technical Direction published by the PMC contains this statement about the TCK:
Split stand-alone Jakarta EE TCK into individual projects Each API project should have a corresponding TCK test repository project. This technical direction is near the top of the list because this effort is not trivial! All projects have to use the same standard mechanism for TCK. We don’t want to face a situation where different projects use different frameworks and it is a challenge to run TCK tests for each of them. It will require some PMC guidance and community input about how new TCK tests are organized, what testing frameworks are used, etc.
So my impression was that Jakarta EE definitely wants to split the TCK repository.
@chkal Your are absolutely right. I do not understand the fuzz about letting this be decided at a higher level as it was already discussed at this level and the outcome was the document you cite from. So while it is OK for me to discuss that again and again, my guideline is that document.
@arjantijms
For this it's indeed necessary to have a TCK that tests the entire Java EE API. A collection of incoherent sub-project tests that all work differently and all have their own quirks is the very least we need.
Agreed, although the standalone tests themselves might be developed independently and referenced via Git submodules from a central repository. That way each project is in charge of its TCK, but we maintain an overall integrated experience.
The central repository would also contain integrations between specs defined on the umbrella project.
While it would be good to be able to run tests individually, say only the JAX-RS TCK tests, we should be very careful that not all tests splinter off and essentially fade away in obscurity. Under the JCP we had the umbrella eg (the Java EE EG). Perhaps we'd need to have a good look at installing something like that again to make sure all constituent projects remain aligned and coherent.
Isn't the Jakarta EE Platform project the equivalent of the umbrella spec? https://projects.eclipse.org/projects/ee4j.jakartaee-platform
@ggam
Agreed, although the standalone tests themselves might be developed independently and referenced via Git submodules from a central repository. That way each project is in charge of its TCK, but we maintain an overall integrated experience.
Something like that might be an option indeed. The PMC already started clearly in the beginning that a non-coherent and totally independent series of tests is certainly not what we should strive for.
Probably a first action point would be agree upon the tooling being used. Looking at the MP and CDI TCKs I think Arquillian and testing in the unit test phase instead of the integration phase, is almost a given. Both these TCKs also use TestNG instead of Junit. Although I'm not specifically a TestNG fan, we'd probably best use that if only for consistency sake.
The parent framework is probably best developed by the Jakarta EE or EE4J projects, e.g. all the well known server definitions for WildFly, Payara, Liberty etc, should not have to be redefined by each project.
The central repository would also contain integrations between specs defined on the umbrella project.
Probably, it's always a bit of a struggle where to put these things exactly and how to organise these. For instance, in the JASPIC tests there's scenarios (integrations) to test for EJB and JACC etc, but the tests are primarily JASPIC still. Other tests are too much a combination of different technologies to really single any tech out as the primary one.
Isn't the Jakarta EE Platform project the equivalent of the umbrella spec?
I don't know tbh. It doesn't seem to have any (political) power yet.
For now, is the jax-rs TCK available to download at all for individuals? I'm implementing the spec but struggling to find how to get the TCK.
The JAX-RS TCK is part of the Jakarta EE TCK which you can find here:
https://github.com/eclipse-ee4j/jakartaee-tck
@danielflower What is the target of your work, another open source JAX-RS framework? 8-)
Thanks for the link!
This is for a little web server library I'm writing with a partial jax-rs implementation built in: https://github.com/3redronin/mu-server
It is different from the existing implementations I've seen in that the user to constructs their own resources and passes them to the library, e.g. if Fruit is a jax-rs resource class: MuServer server = MuServerBuilder.httpsServer().addHandler(RestHandlerBuilder.restHandler(new Fruit())).start();
Fyi I have put together few notes about one possibility how to setup the TCK build and run from that repo.
Now that Jakarta EE 8 is through the door, let's resume this thread. :-)
Proposal for location: Let's put the JAX-RS's "own" part of the TCK into our existing repository in the top-level folder tck. :-)
Currently, I heard 4 different ideas for the TCK location:
- A folder in this repository
- A new repository the EF will create for us
- A folder in the spec repository https://github.com/jakarta-ee/rest-spec
- Keep the TCK intact until Jakarta EE 9.
-- The following are the folders that are used for the Jakarta REST TCK (AFAIK):
- bin
- install/jaxrs
- install/common
- internal/jaxrs
- release
- src/com/sun/ts/lib
- src/com/sun/ts/tests/common/vehicle
- src/com/sun/ts/tests/common/webclient
- src/com/sun/ts/tests/jaxrs
- src/com/sun/ts/tests/servlet/common/util
- src/com/sun/ts/tests/signaturetest
- tools - the folder with ant, not part of the Jakarta EE TCK repo, but required for the build
- user_guides
-- There are common parts that are used by all the TCKs:
- bin
- install/common
- release
- src/com/sun/ts/lib
- src/com/sun/ts/tests/common/vehicle
- src/com/sun/ts/tests/common/webclient
- src/com/sun/ts/tests/signaturetest
Possibly, we may duplicate what is shared with servlet (src/com/sun/ts/tests/servlet/common/util) but the discussion should be what to do with the parts we share. If 2 TCKs each create a change in what is shared that is in a contradiction to each other, never it may be possible to put the Jakarta EE TCK together again.
-- We have the tests that are not part of the standalone TCK (some of them require a working specification in the environment [CDI, BV, EJB, JSON-B, JSON-P], some Servlet 3.0, or Servlet 3.1 for Async). We need to know what to do with these tests:
- src/com/sun/ts/tests/jaxrs/platform
- src/com/sun/ts/tests/jaxrs/servlet3
- src/com/sun/ts/tests/jaxrs/jaxrs21/platform
In fact, I have not much experience with the TCK but I'm willing to learn. ;-)
At first glance it looks as if the TCK was made to test Application Servers against compliance with a Java EE Profile, which explains the tight coupling of all those stuff. But application servers are not the sole target for Jakarte EE in future anymore, while smaller building blocks are (like smaller profiles or even single spec tests). So if the current structure of the TCK does not for a split-up, then that structure has to change. My two cents is that someone from the community possibly could rewrite our part of that structure, in particular the one that deals with the interfaces to other specifications.
The technical direction document of the Eclipse EE4J PMC explicitly proposes that split to be done by the individual projects: Quote: "Split stand-alone Jakarta EE TCK into individual projects - Each API project should have a corresponding TCK test repository project. This technical direction is near the top of the list because this effort is not trivial! All projects have to use the same standard mechanism for TCK. We don’t want to face a situation where different projects use different frameworks and it is a challenge to run TCK tests for each of them. It will require some PMC guidance and community input about how new TCK tests are organized, what testing frameworks are used, etc."(https://www.eclipse.org/ee4j/direction.php#split-stand-alone-jakarta-ee-tck-into-individual-projects)
My interpretation of that is that all decisions are completely up to each project. Instead of waiting for a PMC adivice that possibly never will come, we should decide ourselfs. At the moment, there are +2 for the tck folder but not other opinions (so far) ;-). I think we should not duplicate anything but just move "our" share here (so we have sole control) and have the common share kept as a Maven dependency in the Jakarta EE TCK project (so they keep sole control).
At the moment, there are +2 for the
tckfolder but not other opinions (so far) ;-).
Actually I've a slight preference for putting our part of the TCK into a new repository jaxrs-tck or rest-tck. IMO the specification (API + spec document) and the TCK are two different things that should be kept separate. Especially because I think that they will have to be versioned independently.
I think we should not duplicate anything but just move "our" share here (so we have sole control) and have the common share kept as a Maven dependency in the Jakarta EE TCK project (so they keep sole control).
Of course, it would be great if the common shared code for all TCKs would be "just" a compile/test-scoped dependency for us. But I guess that's something that need to discussed in the current Jakarta TCK project!?!
At the moment, there are +2 for the
tckfolder but not other opinions (so far) ;-).Actually I've a slight preference for putting our part of the TCK into a new repository
jaxrs-tckorrest-tck. IMO the specification (API + spec document) and the TCK are two different things that should be kept separate. Especially because I think that they will have to be versioned independently.
I know that the recommendation is to have the TCK in a separate repo, but I do not see the benefit of that (just many drawbacks). Can you please elaborate on the organizational or technical benefit we would have from separating repositories and versions? My opinion is it would be best to have all in one repo so an agreed change is to be done in one single commit, making it pretty clear to follow the history for readers.
I think we should not duplicate anything but just move "our" share here (so we have sole control) and have the common share kept as a Maven dependency in the Jakarta EE TCK project (so they keep sole control).
Of course, it would be great if the common shared code for all TCKs would be "just" a compile/test-scoped dependency for us. But I guess that's something that need to discussed in the current Jakarta TCK project!?!
There is no such active team in reality. The TCK is currently mostly a bucket holding all non-claimed subproject TCKs plus the glue code. In the end of the splitting process there will not be much left over. It is up to the subproject teams to contribute the needed glue code between the new TCKs then.
I know that the recommendation is to have the TCK in a separate repo, but I do not see the benefit of that (just many drawbacks). Can you please elaborate on the organizational or technical benefit we would have from separating repositories and versions? My opinion is it would be best to have all in one repo so an agreed change is to be done in one single commit, making it pretty clear to follow the history for readers.
This is just my opinion, but I really think that separating what is specified (API + spec document) should be managed separately from how it is tested. I don't think that it will be possible in practice to modify all three artifacts (API + spec document + TCK) always in a single PR. This makes it also harder to review. So we will most likely get even fewer reviewers than we currently have. Also, if we version artifacts independently, tagging is much simpler if we have separate repositories.
However, I also see a few benefits in managing everything in one place, but also a few downsides (as mentioned above). That's why I wrote that I have a slight preference for the separate repository.
There is no such active team in reality. The TCK is currently mostly a bucket holding all non-claimed subproject TCKs plus the glue code.
Yes, that may be the case. However, creating the common shared code and version it correctly is something that needs to be coordinated. I don't say that it will be easy. My hope is that the people having more experience with the TCK can help with this.
In the end of the splitting process there will not be much left over. It is up to the subproject teams to contribute the needed glue code between the new TCKs then.
Sure, also possible. However, this should be a fallback option.