Define core deployment

[sbordet]

Jetty version(s) 12.0.x

Jetty Environment core

Description Currently, the deployment of a Jetty core web application not documented and cumbersome (but works).

To deploy a core webapp, one needs the following structure:

$JETTY_BASE/webapps
├── foo-webapp.xml
└── foo-webapp.d
       ├── classes
       │   └── org
       │       └── foo
       │           └── ServerApp.class
       └── lib
           ├── jetty-util-12.0.9.jar
           ├── ...
           └── slf4j-api-2.0.12.jar

where foo-webapp.xml is a Jetty context XML file, foo-webapp.d/classes contains the webapp classes, and foo-webapp.d/lib contains the webapp dependencies.

Creating this structure is currently not automated, and users must manually create it. Furthermore, it is not a single file like a *.war, so it's not particularly easy to copy around, atomically deploy it, etc. Users would also have troubles generating the lib directory with all the dependencies.

We should discuss if there are simpler options to allow users to deploy a Jetty core web application, possibly using a single file.

[sbordet]

One idea is to re-use the *.war format, with no WEB-INF/web.xml but a WEB-INF/jetty-web.xml.

We already support WEB-INF/jetty-web.xml for eeN deployments, so we should be able to reuse all the code.

This solution has the benefit that people can develop webapps using the IDE, the m-war-p, without having to resort to special build configurations (for example, using the m-dependency-p to figure out all the transitive dependencies that must go into lib).

It would also benefit of the fact that WEB-INF is protected from remote access, while it is not clear (or currently strongly defined) for foo-webapp.d.

[gregw]

The minimal requirements for deploying a core application is just a context.xml file. It is entirely plausible for such an app to take all it's classes from the containers classpath, maybe even only using utility handlers, or perhaps some application code dropped into $jetty.home/lib.
If fact your example is kind of wrong as you should not need to provide either jetty-util nor jslf4j-api jars to a core application. We do not hide any "server" jars from a core application.

I think we should look at an example of creating and deploying a core application with our module system, which already has the ability to move jars into lib, config into etc (or a .d) and to deploy an xml in webapps. I think one of the main advantages of a core application is that it blurs the line between container and application, so it could be thought of as a server component.

However, I agree that is not the only way that users might look at a core application and having a single file deployable is also a good idea. But I don't think we should try to invent our own single file deployment format... but I'm not opposed to reusing the war format (even if it is not very good for purpose). I'm not sure how much code reuse we can achieve, as a core app will be based on a ContextHandler not a WebAppContext, so it will not have the Configuration mechanism used to configure an EE webapp. That is not necessarily a bad thing, as the EE webapp deployment stuff is already insanely complex with programmatic, annotated, descriptor, convention based configuration from many generations of specification with a smear of CDI influence on top. Adding more conditions to that code is probably not a great idea - it needs to stay focused on meeting the spec. To rewrite some code that will use the contents of a core-war as the resourceBase and protect the WEB-INF subdirectory is not that difficult. We probably would want to extract some common code for unpacking (but again that is already so much more complex than you'd think).

[sbordet]

The minimal requirements for deploying a core application is just a context.xml file. It is entirely plausible for such an app to take all it's classes from the containers classpath, maybe even only using utility handlers, or perhaps some application code dropped into $jetty.home/lib.

Yes, but my point is that developers would rather have a clear path of what to do. The suggestion to use JETTY_HOME/lib kinda goes against our JETTY_BASE concept, so it works, but not ideal. Sure we can document to use the ext Jetty module, but that would be accessible to all webapps, which perhaps it's not ideal.

If fact your example is kind of wrong as you should not need to provide either jetty-util nor jslf4j-api jars to a core application. We do not hide any "server" jars from a core application.

Fair enough, mine was an example.

think one of the main advantages of a core application is that it blurs the line between container and application, so it could be thought of as a server component.

But there is also the case where I just want to deploy a better integrated Jetty webapp as if it was a normal webapp.

For example, CometD 8 has now a CometDHandler, so the deployment of such CometD webapp would be:

$JETTY_BASE/webapps
├── cometd.xml
└── cometd.d
       ├── classes
       │   └── org
       │       └── acme
       │           └── cometd
       │               └── MyCometDService.class
       └── lib
           ├── cometd-java-server-http-jetty-8.0.2.jar
           ├── ...
           └── jetty-util-ajax-12.0.10.jar

In particular:

• Need to have all the CometD jar dependencies, but only for this webapp, in cometd.d/lib.
• I can reuse the Jetty server dependencies exposed in the server class-path, provided that they are exposed to webapps, but for example I doubt jetty-util-ajax.jar is exposed by the server, so likely needs to be in cometd.d/lib.
• cometd.xml contains the declaration for CometDHandler (from CometD), and possibly other Jetty utility handlers (from the server class-path).

I would say that for a non-trivial Jetty core webapp, we need <app>.d/classes and <app>.d/lib in almost all cases, plus app.xml.

With a *.war format it would be:

$JETTY_BASE/webapps
└── cometd.war
       ├── index.html
       ├── cometd.js
       └── WEB-INF
           ├── jetty-web.xml
           ├── classes
           │   └── org
           │       └── acme
           │           └── cometd
           │               └── MyCometDService.class
           └── lib
               ├── cometd-java-server-http-jetty-8.0.2.jar
               ├── ...
               └── jetty-util-ajax-12.0.10.jar

The root of the *.war would contain the CometD JavaScript files to serve to clients, as well as HTML, CSS, etc.

With this, we would not need the <app>.d (we may have it, but it would not be mandatory like it is now, as the *.war is self-contained), and reuse a well known deployment format that would be familiar to developers.

[janbartel]

One idea is to re-use the *.war format, with no WEB-INF/web.xml but a WEB-INF/jetty-web.xml.

We already support WEB-INF/jetty-web.xml for eeN deployments, so we should be able to reuse all the code. It would be a matter of copying the code up into jetty-core somewhere as reading of WEB-INF/jetty-web.xml is read by the JettyWebXmlConfiguration class, and Configuration classes are not part of jetty-core.

This solution has the benefit that people can develop webapps using the IDE, the m-war-p, without having to resort to special build configurations (for example, using the m-dependency-p to figure out all the transitive dependencies that must go into lib).

It would also benefit of the fact that WEB-INF is protected from remote access, while it is not clear (or currently strongly defined) for foo-webapp.d.

Again, IIRC WEB-INF is not protected in jetty-core, but by the eeX specific webapp modules.

I'm not sure I like reusing WEB-INF directory for this purpose. We could use META-INF instead as maven already supports putting stuff in there? In fact, the jetty-osgi code already has a series of extra headers defined for MANIFEST.MF that allows it to deploy a plain .jar file that contains classes, lib ,context.xml and static content. Maybe we could look at that type of solution?

[sbordet]

@janbartel it would be complicated to set up for users. Right now with the m-war-p everything is automatic with almost zero POM configuration.

Rebuiliding exactly the same structure under META-INF would be a lot more POM configuration with possible errors, etc. for exactly the same structure, but now you have to use also the m-dependency-p, the m-jar-p, etc.

[olamy]

if needed we can create our own maven packaging type (and plugin) to help users. so the user setup would be as simple as adding a line such

<packaging>jetty-core-or-whatever-name-we-prefer</packaging>

[sbordet]

if needed we can create our own maven packaging type (and plugin) to help users.

Sure, but my point is that we don't need to, as there already exist one that is widespread and supported by IDEs (ours won't be) and tools.

[gregw]

I'm not sure I like reusing WEB-INF directory for this purpose. We could use META-INF instead as maven already supports putting stuff in there? In fact, the jetty-osgi code already has a series of extra headers defined for MANIFEST.MF that allows it to deploy a plain .jar file that contains classes, lib ,context.xml and static content. Maybe we could look at that type of solution?

I think we should protect both WEB-INF and META-INF in a core single file deployment, not least because I can imagine a real war being accidentally deployed as core and we should not expose its web.xml and other WEB-INF content. I don't mind this as it is convention over configuration.

I also think that if OSGi has a single file format that we should look at supporting that as well... maybe even both at the same time.

[joakime]

A file structure that I like.

I chose the extension *.core to differentiate the deployment, and not have accidental war deployment in core. In reality it is still a JAR file (like the WAR file).

<name>.core
/jetty.xml  - defines the XML that becomes the ContextHandler.setHandler(<here>) on deployment
/static/    - the only place static resources are served from
/classes/   - classes for this core webapp
/lib/       - libs for this core webapp

The choice of /static/ is so that there is no need for protecting special directories and whatnot. It becomes either the ContextHandler.baseResource or the ResourceHandler.baseResource If the archive doesn't have a /static/ directory then there is no ResourceHandler automatically added. The user can add their own ResourceHandler via the /jetty.xml if they so desire.

The /jetty.xml can use <Property> elements. Those properties can be configured via the <name>.properties file that we already have. I don't think we should have any extra lifecycle (eg: servlet init phase) or automatic discovery (eg: bytecode scanning) in this feature.

[sbordet]

@joakime while I don't disagree with your approach, there is the issue of tools.

IDEs know the *.war format, and so do the Maven WAR plugin, etc. so the idea is to try to reuse as much tooling as possible, rather than inventing a new format that would never be a first class citizen in tools.

I agree that the *.war extension is a problem, but that should be only true if there is not a <name>.properties that specifies the core environment.

[joakime]

@joakime while I don't disagree with your approach, there is the issue of tools.

IDEs know the *.war format, and so do the Maven WAR plugin, etc. so the idea is to try to reuse as much tooling as possible, rather than inventing a new format that would never be a first class citizen in tools.

IDEs barely know what a war is as well. They can create it, with the help of build tooling, but they cannot use it. That's something else, like a plugin to deploy to a web container. The IDEs don't load a WAR, they don't open a WAR, they don't list a WARs contents, etc. This is in the same scope as any of the countless of other types of archives out there (like <type>bundle</type>). Having it be *.jetty or *.core or *.war or *.ear is exactly the same in the eyes of all IDEs (and build tools) out there. They are an abstract concept that is not loadable by a classloader.

The proposed features of this Issue also has none of the baggage of WAR, with its rigid rules and validation that are present in most of the build tools around war files. (eg: overlays, resource validation, deployment descriptor validation, web.xml validation, resource encodings, dependent web resource structure validation, etc)

We also don't what adjacent build tooling looking for war being built or existing in the dependencies and doing extra work when it is not a war.

This proposal is actually closer to just a normal zip file than anything else.

This issue is FAR easier to implement as a JAR or ZIP in maven or gradle than a war.

I agree that the *.war extension is a problem, but that should be only true if there is not a <name>.properties that specifies the core environment.

You need the core-deploy module enabled, and consequently you need that <name>.properties to deploy it to core environment. If we have included class files or libs then we need that environment to exist (we cannot just a deploy without a environment in these cases) so that we can isolate that apps classloader.

[sbordet]

I like the /static/ concept, but @gregw pointed out that if deployed to non-Jetty, the non-war format would expose (as static files) jetty.xml, /classes and /lib.

So perhaps a middle ground could be:

<name>.war
├── WEB-INF/
│   ├── classes/
│   ├── lib/
│   ├── jetty.xml
│   └── <web.xml> // optional, to disable deployment in other containers by 403-ing every request
└── static/

Classloading would be parent-first, so webapps would not be able to modify server classes (as they will be loaded first). This means that you cannot have different versions of jetty-util, or jetty-client in the webapp, they will be the server version. I think this is acceptable.

[janbartel]

I still think we shouldn't reuse a .war to refer to a non-servlet webapp.

I don't like the presence WEB-INF and web.xml which are servlet concepts. For starters, it will make the OSGi deployer difficult to implement. Currently it looks for the presence of these kind of files as signifiers that a bundle (aka jar) file that has just been deployed into the container is a webapp that should be deploy into jetty. How can it determine that it isn't, in fact, a webapp?

I prefer the idea of using just a plain .jar with anything deployment related inside META-INF. The layout could be something like:

.jar |- class files |- META-INF/ |- jetty-context.xml |- resources

It's extremely easy to make a jar. It's extremely easy to configure maven to include META-INF/jetty-context.xml and META-INF/resources. If you're happy to have all the dependencies unpacked into the jar then you can use the shade plugin. Otherwise, use the dependency plugin to copy them in to META-INF/lib. Or alternatively ask @olamy to write a spiffy new plugin to do it for you.

[olamy]

yup, we can definitely create our own Maven packaging, and this will make life easier for our users. Not sure about the naming, though:

• jetty-core-handler
• jetty-handler
• something else more fancy

[gregw]

@sbordet @joakime Can you schedule a meeting to discuss this. I did not realize that you were proposing to deploy "/foo.d/static". I'm pretty much against this and I think that "/foo.d/" should be left as an ignored directory and all deployment should be done from "/foo/" even if that has "/foo/static".

The issue of telling what kind of application is within "/foo/" is not different from "EEx" vs "EEy" that from "core" to "EEx". Either we only have a single default environment; or we have a properties file that tells us what it is; or perhaps we have some magic files to auto-detect?

[joakime]

This is now merged as part of #12998