jetty.project
jetty.project copied to clipboard
jetty
"Ambiguous URI path encoding" when upgrading to Jetty 12 EE8
Jetty Version
Jetty 12.0.12
Jetty Environment EE8
Java Version 17
Question
On XWiki, I'm trying to upgrade from Jetty 10 to Jetty 12.0.12 and one of our tests is failing when http://localhost:8080/xwiki/rest/wikis/xwiki/spaces/AttachmentsResourceIT/pages/testPUTGETAttachments/attachments/%25percent.txt is called. It leads to a 400 error with the message "Ambiguous URI path encoding".
The problem is that we've set jetty.httpConfig.uriCompliance=RFC3986 (we had that in Jetty 10 already) but it's not helping.
I've read:
- https://github.com/jetty/jetty.project/issues/11890
- https://github.com/jetty/jetty.project/issues/11448#issuecomment-1969206031
I'm still puzzled and don't understand what we can do to allow % to be passed in URLs in Jetty 12. In the threads above it's mentioned that the issue is that Servlet 6 spec doesn't allow it. That's fine but I don't fully understand why the RFC3986 compliance doesn't allow it (I've also tried other values like jetty.httpConfig.uriCompliance=RFC3986,AMBIGUOUS_PATH_ENCODING,AMBIGUOUS_PATH_SEPARATOR), but more importantly since we've configured Jetty to use EE8 (ie Servlet 4), I don't understand why we would be subject to Servlet 6 rules :)
Any idea?
Thanks a lot
The reason Servlet 6 rejects these sequences is that is causes bugs in Servlet land that cannot be resolved or fixed without big changes to the Servlet API.
When things are ambiguous in the path section, then things like security constraints, url-patterns, redirect location logic, and various values on HttpServletRequest are bad and you cannot tell the difference between someone using a raw % or encoded %25 or double-encoded %2525 sequence. Allowing %25 can, for example, allowed a bypass into the WEB-INF directory, via requests like /%2557EB-INF/one.js%23/ simply because of how requests are simultaneously encoded and decoded in the HttpServletRequest object depending on the API in use (fixed in Jetty by finding all WEB-INF directories in the base resource tree and rejecting access to those specific paths via the ResourceService after the requested resource has been resolved). Similar bypasses exist for security constraints and url-patterns that cannot be resolved in Servlet land. Hence the new rules in Servlet 6 to just reject those ambiguous character sequences.
These bugs also exist in all prior versions of Servlet too, and it is not possible to resolve them.
You have bugs if you accept /%25percent.txt on the path, ones that cannot be resolved while in Servlet land.
Regarding your specific case, that seems to be triggering the UriCompliance.Violation.AMBIGUOUS_PATH_ENCODING specific violation.
Using jetty.httpConfig.uriCompliance=RFC3986,AMBIGUOUS_PATH_ENCODING should be enough.
Confirmed on fresh jetty-base running Jetty 12.0.12 with http and ee8-demos installed.
Default behavior
$ java -jar ~/dists/jetty-home-12.0.12/start.jar
...(snip startup)..
# In different terminal
$ echo -e "GET /test/%25foo.txt HTTP/1.0\r\n\r\n" | nc -vvv localhost 8080
Connection to localhost (::1) 8080 port [tcp/http-alt] succeeded!
HTTP/1.1 400 Bad Request
Server: Jetty(12.0.12)
Cache-Control: must-revalidate,no-cache,no-store
Content-Type: text/html;charset=iso-8859-1
Content-Length: 452
<html>
<head>
<meta http-equiv="Content-Type" content="text/html;charset=ISO-8859-1"/>
<title>Error 400 Ambiguous URI path encoding</title>
</head>
<body>
<h2>HTTP ERROR 400 Ambiguous URI path encoding</h2>
<table>
<tr><th>URI:</th><td>/badURI</td></tr>
<tr><th>STATUS:</th><td>400</td></tr>
<tr><th>MESSAGE:</th><td>Ambiguous URI path encoding</td></tr>
</table>
<hr/><a href="https://jetty.org/">Powered by Jetty:// 12.0.12</a><hr/>
</body>
</html>
Using configuration to allow it.
$ java -jar ~/dists/jetty-home-12.0.12/start.jar jetty.httpConfig.uriCompliance=RFC3986,AMBIGUOUS_PATH_ENCODING
... (snip lots of output) ...
# In different terminal
$ echo -e "GET /test/%25foo.txt HTTP/1.0\r\n\r\n" | nc -vvv localhost 8080
Connection to localhost (::1) 8080 port [tcp/http-alt] succeeded!
HTTP/1.1 404 Not Found
Server: Jetty(12.0.12)
Set-Cookie: visited=yes
Cache-Control: must-revalidate,no-cache,no-store
Content-Type: text/html;charset=iso-8859-1
Content-Length: 449
<html>
<head>
<meta http-equiv="Content-Type" content="text/html;charset=ISO-8859-1"/>
<title>Error 404 Not Found</title>
</head>
<body><h2>HTTP ERROR 404 Not Found</h2>
<table>
<tr><th>URI:</th><td>/test/%25foo.txt</td></tr>
<tr><th>STATUS:</th><td>404</td></tr>
<tr><th>MESSAGE:</th><td>Not Found</td></tr>
<tr><th>SERVLET:</th><td>default</td></tr>
</table>
<hr/><a href="https://jetty.org/">Powered by Jetty:// 12.0.12</a><hr/>
</body>
</html>
The behavior you are experiencing is coming from org.eclipse.jetty.http.HttpURI btw.
package uri;
import java.net.URI;
import org.eclipse.jetty.http.HttpURI;
import org.eclipse.jetty.http.UriCompliance;
public class JettyUriTest
{
public static void main(String[] args)
{
String name = "5boards.txt";
dump("http://localhost/foo/%" + name);
dump("http://localhost/foo/%25" + name);
dump("http://localhost/foo/%2525" + name);
}
public static void dump(String rawinput)
{
System.out.println("\n--- raw input \"" + rawinput + "\" ---");
dump(HttpURI.build(rawinput));
dump(URI.create(rawinput));
}
private static void dump(HttpURI uri)
{
System.out.println("Jetty HttpURI = " + uri);
System.out.println(" .toString = " + uri.toString());
// System.out.println(" .isAbsolute = " + uri.isAbsolute());
// System.out.println(" .scheme = " + uri.getScheme());
// System.out.println(" .authority = " + uri.getAuthority());
// System.out.println(" .user = " + uri.getUser());
// System.out.println(" .host = " + uri.getHost());
// System.out.println(" .port = " + uri.getPort());
System.out.println(" .path: " + uri.getPath());
System.out.println(" .decodedPath: " + uri.getDecodedPath());
if (uri.hasViolations())
{
System.out.println(" -VIOLATIONS-");
for (UriCompliance.Violation violation : uri.getViolations())
{
System.out.println(" * violation = " + violation);
}
}
}
private static void dump(URI uri)
{
System.out.println("java.net.URI = " + uri);
System.out.println(" .toASCIIString = " + uri.toASCIIString());
// System.out.println(" .isAbsolute = " + uri.isAbsolute());
// System.out.println(" .scheme = " + uri.getScheme());
// System.out.println(" .authority = " + uri.getAuthority());
// System.out.println(" .userInfo = " + uri.getUserInfo());
// System.out.println(" .host = " + uri.getHost());
// System.out.println(" .port = " + uri.getPort());
System.out.println(" .rawPath: " + uri.getRawPath());
System.out.println(" .path = " + uri.getPath());
System.out.println(" .schemeSpecificPart = " + uri.getSchemeSpecificPart());
System.out.println(" .normalize = " + uri.normalize());
}
}
With the output ...
--- raw input "http://localhost/foo/%5boards.txt" ---
Jetty HttpURI = http://localhost/foo/%5boards.txt
.toString = http://localhost/foo/%5boards.txt
.path: /foo/%5boards.txt
.decodedPath: /foo/[oards.txt
java.net.URI = http://localhost/foo/%5boards.txt
.toASCIIString = http://localhost/foo/%5boards.txt
.rawPath: /foo/%5boards.txt
.path = /foo/[oards.txt
.schemeSpecificPart = //localhost/foo/[oards.txt
.normalize = http://localhost/foo/%5boards.txt
--- raw input "http://localhost/foo/%255boards.txt" ---
Jetty HttpURI = http://localhost/foo/%255boards.txt
.toString = http://localhost/foo/%255boards.txt
.path: /foo/%255boards.txt
.decodedPath: /foo/%5boards.txt
-VIOLATIONS-
* violation = AMBIGUOUS_PATH_ENCODING
java.net.URI = http://localhost/foo/%255boards.txt
.toASCIIString = http://localhost/foo/%255boards.txt
.rawPath: /foo/%255boards.txt
.path = /foo/%5boards.txt
.schemeSpecificPart = //localhost/foo/%5boards.txt
.normalize = http://localhost/foo/%255boards.txt
--- raw input "http://localhost/foo/%25255boards.txt" ---
Jetty HttpURI = http://localhost/foo/%25255boards.txt
.toString = http://localhost/foo/%25255boards.txt
.path: /foo/%25255boards.txt
.decodedPath: /foo/%255boards.txt
-VIOLATIONS-
* violation = AMBIGUOUS_PATH_ENCODING
java.net.URI = http://localhost/foo/%25255boards.txt
.toASCIIString = http://localhost/foo/%25255boards.txt
.rawPath: /foo/%25255boards.txt
.path = /foo/%255boards.txt
.schemeSpecificPart = //localhost/foo/%255boards.txt
.normalize = http://localhost/foo/%25255boards.txt
Demo of this setup in Jetty 12 embedded.
package uri;
import java.io.IOException;
import java.io.InputStream;
import java.io.OutputStream;
import java.net.Socket;
import java.net.URI;
import java.nio.charset.StandardCharsets;
import org.eclipse.jetty.http.HttpHeader;
import org.eclipse.jetty.http.UriCompliance;
import org.eclipse.jetty.io.Content;
import org.eclipse.jetty.server.Handler;
import org.eclipse.jetty.server.HttpConfiguration;
import org.eclipse.jetty.server.HttpConnectionFactory;
import org.eclipse.jetty.server.Request;
import org.eclipse.jetty.server.Response;
import org.eclipse.jetty.server.Server;
import org.eclipse.jetty.server.ServerConnector;
import org.eclipse.jetty.util.Callback;
import org.eclipse.jetty.util.IO;
import org.eclipse.jetty.util.component.LifeCycle;
public class JettyEncodedPercentDemo
{
public static void main(String[] args) throws Exception
{
Server server = new Server();
HttpConfiguration httpConfiguration = new HttpConfiguration();
UriCompliance specViolatingCompliance = UriCompliance.from("RFC3986,AMBIGUOUS_PATH_ENCODING");
httpConfiguration.setUriCompliance(specViolatingCompliance);
ServerConnector connector = new ServerConnector(server, new HttpConnectionFactory(httpConfiguration));
connector.setPort(8080);
server.addConnector(connector);
Handler.Abstract handler = new Handler.Abstract()
{
@Override
public boolean handle(Request request, Response response, Callback callback)
{
response.setStatus(200);
response.getHeaders().add(HttpHeader.CONTENT_TYPE, "text/plain;charset=utf-8");
StringBuilder str = new StringBuilder();
str.append("Got Raw URI: ").append(request.getHttpURI().toString());
str.append("\nDecoded Path: ").append(request.getHttpURI().getDecodedPath());
Content.Sink.write(response, true, str.toString(), callback);
return true;
}
};
server.setHandler(handler);
try
{
server.start();
URI serverURI = server.getURI();
String name = "5boards.txt";
sendRequest(serverURI.getHost(), serverURI.getPort(), "/foo/%" + name);
sendRequest(serverURI.getHost(), serverURI.getPort(), "/foo/%25" + name);
sendRequest(serverURI.getHost(), serverURI.getPort(), "/foo/%2525" + name);
}
finally
{
LifeCycle.stop(server);
}
}
private static void sendRequest(String host, int port, String path) throws IOException
{
try (Socket client = new Socket(host, port);
OutputStream out = client.getOutputStream();
InputStream in = client.getInputStream())
{
String rawRequest = """
GET %s HTTP/1.1
Host: %s:%d
Connection: close
""".formatted(path, host, port);
System.out.println("\n----- REQUEST -----\n" + rawRequest);
out.write(rawRequest.getBytes(StandardCharsets.UTF_8));
out.flush();
String rawResponse = IO.toString(in, StandardCharsets.UTF_8);
System.out.println("----- RESPONSE -----\n" + rawResponse);
}
}
}
With output ...
2024-08-13 12:23:18.933:INFO :oejs.Server:main: jetty-12.0.12; built: 2024-07-25T21:58:37.668Z; git: cc6f1b74db755fed228b50701ad967aeaa68e83f; jvm 21.0.3+9-LTS
2024-08-13 12:23:18.978:INFO :oejs.AbstractConnector:main: Started ServerConnector@1f554b06{HTTP/1.1, (http/1.1)}{0.0.0.0:8080}
2024-08-13 12:23:18.996:INFO :oejs.Server:main: Started oejs.Server@27efef64{STARTING}[12.0.12,sto=0] @416ms
----- REQUEST -----
GET /foo/%5boards.txt HTTP/1.1
Host: 192.168.1.215:8080
Connection: close
----- RESPONSE -----
HTTP/1.1 200 OK
Server: Jetty(12.0.12)
Date: Tue, 13 Aug 2024 17:23:19 GMT
Content-Type: text/plain;charset=utf-8
Content-Length: 85
Connection: close
Got Raw URI: http://192.168.1.215:8080/foo/%5boards.txt
Decoded Path: /foo/[oards.txt
----- REQUEST -----
GET /foo/%255boards.txt HTTP/1.1
Host: 192.168.1.215:8080
Connection: close
----- RESPONSE -----
HTTP/1.1 200 OK
Server: Jetty(12.0.12)
Date: Tue, 13 Aug 2024 17:23:19 GMT
Content-Type: text/plain;charset=utf-8
Content-Length: 89
Connection: close
Got Raw URI: http://192.168.1.215:8080/foo/%255boards.txt
Decoded Path: /foo/%5boards.txt
----- REQUEST -----
GET /foo/%25255boards.txt HTTP/1.1
Host: 192.168.1.215:8080
Connection: close
----- RESPONSE -----
HTTP/1.1 200 OK
Server: Jetty(12.0.12)
Date: Tue, 13 Aug 2024 17:23:19 GMT
Content-Type: text/plain;charset=utf-8
Content-Length: 93
Connection: close
Got Raw URI: http://192.168.1.215:8080/foo/%25255boards.txt
Decoded Path: /foo/%255boards.txt
2024-08-13 12:23:19.107:INFO :oejs.Server:main: Stopped oejs.Server@27efef64{STOPPING}[12.0.12,sto=0]
2024-08-13 12:23:19.111:INFO :oejs.AbstractConnector:main: Stopped ServerConnector@1f554b06{HTTP/1.1, (http/1.1)}{0.0.0.0:8080}
Thanks a lot @joakime for your answer (and a fast one), much appreciated :)
The reason Servlet 6 rejects these sequences is that is causes bugs in Servlet land that cannot be resolved or fixed without big changes to the Servlet API.
Ok, but here since I'm using EE8, I'm on Servlet 4 so the rules for Servlet 6 shouldn't apply, right? Aren't they the same rules that were used in Jetty 10?
Regarding your specific case, that seems to be triggering the UriCompliance.Violation.AMBIGUOUS_PATH_ENCODING specific violation.
Using jetty.httpConfig.uriCompliance=RFC3986,AMBIGUOUS_PATH_ENCODING should be enough.
Unfortunately it didn't help. I had tried a lot of values, including using UNSAFE. It always fails as if the config property is not used. I checked and re-checked how the property is used but it looks good; in jetty.xml there's <Set name="uriCompliance"><Call class="org.eclipse.jetty.http.UriCompliance" name="from"><Arg><Property name="jetty.httpConfig.uriCompliance" default="DEFAULT"/></Arg></Call></Set> and then in a xwiki.ini file, I have jetty.httpConfig.uriCompliance=RFC3986,AMBIGUOUS_PATH_ENCODING (I've updated it locally to add AMBIGUOUS_PATH_ENCODING).
I'm still getting:
HTTP ERROR 400 Ambiguous URI path encoding
URI: /badURI STATUS: 400 MESSAGE: Ambiguous URI path encoding
Any idea of what I could try?
Thanks a lot @joakime for your answer (and a fast one), much appreciated :)
twas an easy one to answer too, and I happened to have these demo projects already written. :)
The reason Servlet 6 rejects these sequences is that is causes bugs in Servlet land that cannot be resolved or fixed without big changes to the Servlet API.
Ok, but here since I'm using EE8, I'm on Servlet 4 so the rules for Servlet 6 shouldn't apply, right? Aren't they the same rules that were used in Jetty 10?
The bugs exists in the Servlet API (no specific version). The bug is that any ambiguous encoding in the path portion of the URL/URI cannot be properly decode / resolved / made available in the APIs / re-encoded due to the API calls / etc ...
The "any ambiguous encoding in the path portion" is any encoded character that impacts URI reserved characters or the % symbol itself. (There is also an ambiguous path segment one where the path segment is empty and has nothing to do with encoding)
https://datatracker.ietf.org/doc/html/rfc3986#section-2.2
The bug is not unique to Servlet 6, it exists all the way back to Servlet 1.0 too.
Regarding your specific case, that seems to be triggering the UriCompliance.Violation.AMBIGUOUS_PATH_ENCODING specific violation. Using jetty.httpConfig.uriCompliance=RFC3986,AMBIGUOUS_PATH_ENCODING should be enough.
Unfortunately it didn't help. I had tried a lot of values, including using
UNSAFE. It always fails as if the config property is not used. I checked and re-checked how the property is used but it looks good; injetty.xmlthere's<Set name="uriCompliance"><Call class="org.eclipse.jetty.http.UriCompliance" name="from"><Arg><Property name="jetty.httpConfig.uriCompliance" default="DEFAULT"/></Arg></Call></Set>and then in axwiki.inifile, I havejetty.httpConfig.uriCompliance=RFC3986,AMBIGUOUS_PATH_ENCODING(I've updated it locally to addAMBIGUOUS_PATH_ENCODING).I'm still getting:
HTTP ERROR 400 Ambiguous URI path encoding URI: /badURI STATUS: 400 MESSAGE: Ambiguous URI path encoding
Any idea of what I could try?
The "Ambiguous URI path encoding" message applies to all ambiguous path issues. Which includes ...
- AMBIGUOUS_PATH_SEPARATOR
- AMBIGUOUS_PATH_SEGMENT
- AMBIGUOUS_PATH_PARAMETER
- AMBIGUOUS_PATH_ENCODING
- AMBIGUOUS_EMPTY_SEGMENT
The fact that you see /badURI here is because the failure is detected early, well before it is even dispatched to your webapp.
I suspect you are not setting that configuration either in the right place (so it isn't applied to the same connector you are testing), or the right time (so it isn't applied), or something else.
I was going to have you look at the HttpConfiguration at runtime, but discovered those configurations are not present in either the dump or JMX (grumble grumble), so I opened #12163 Any chance you can look at the HttpConfiguration in use via a debugger?
Any chance you can look at the HttpConfiguration in use via a debugger?
Seems you're right and it's not set (DEFAULT is used somehow):
Now need to find why...
Works if I modify jetty.xml from <Set name="uriCompliance"><Call class="org.eclipse.jetty.http.UriCompliance" name="from"><Arg><Property name="jetty.httpConfig.uriCompliance" default="DEFAULT"/></Arg></Call></Set> to <Set name="uriCompliance"><Call class="org.eclipse.jetty.http.UriCompliance" name="from"><Arg>UNSAFE</Arg></Call></Set>, so the issue is the passing of the property from our xwiki.ini file somehow.
How can I know if the xwiki.ini file located in start.d is used?
I guess it's used since otherwise the --module=xwiki command wouldn't be used and the xwiki.mod wouldn't be used and Jetty wouldn't deploy XWiki.
So maybe the issue is just the syntax for defining a property in xwiki.ini?
https://github.com/xwiki/xwiki-platform/blob/477dbd0edd0572cd3102c7355308a1c7c9f58596/xwiki-platform-tools/xwiki-platform-tool-jetty/xwiki-platform-tool-jetty-resources/src/main/resources/start.d/xwiki.ini
Knowing that this used to work in Jetty 10, maybe there's been a change in Jetty 12?
If I remove our xwiki.ini and try to regenerate it with java --add-opens java.base/java.lang=ALL-UNNAMED --add-opens java.base/java.io=ALL-UNNAMED --add-opens java.base/java.util=ALL-UNNAMED --add-opens java.base/java.util.concurrent=ALL-UNNAMED -Djava.io.tmpdir=./tmp -Djetty.home=jetty -Djetty.base=. -Dfile.encoding=UTF8 -Djetty.http.port=8080 -jar jetty/start.jar --add-modules=xwiki jetty.http.port=8080 STOP.KEY=xwiki STOP.PORT=8079, I get some warnings/errors (no idea if that matters):
INFO : server transitively enabled, ini template available with --add-modules=server
INFO : ee8-plus transitively enabled
INFO : ee-webapp transitively enabled, ini template available with --add-modules=ee-webapp
INFO : ee8-websocket-jetty transitively enabled
INFO : threadpool transitively enabled, ini template available with --add-modules=threadpool
INFO : ee8-annotations transitively enabled
INFO : deploy transitively enabled
INFO : security transitively enabled
INFO : ee8-servlet transitively enabled
INFO : console-capture transitively enabled, ini template available with --add-modules=console-capture
INFO : client transitively enabled
INFO : bytebufferpool transitively enabled, ini template available with --add-modules=bytebufferpool
INFO : ext transitively enabled
INFO : ee8-deploy transitively enabled, ini template available with --add-modules=ee8-deploy
INFO : sessions transitively enabled, ini template available with --add-modules=sessions
INFO : ee8-webapp transitively enabled, ini template available with --add-modules=ee8-webapp
INFO : requestlog transitively enabled, ini template available with --add-modules=requestlog
INFO : ee8-apache-jsp transitively enabled
INFO : http-forwarded transitively enabled, ini template available with --add-modules=http-forwarded
INFO : resources transitively enabled
INFO : plus transitively enabled
INFO : ee8-security transitively enabled
INFO : xwiki-logging transitively enabled
INFO : ee8-websocket-javax transitively enabled
INFO : jndi transitively enabled
INFO : http transitively enabled, ini template available with --add-modules=http
INFO : xwiki initialized in /Users/vmassol/dev/xwiki/xwiki-platform/xwiki-platform-core/xwiki-platform-rest/xwiki-platform-rest-test/xwiki-platform-rest-test-tests/target/xwiki/start.d/xwiki.ini
INFO : logging/slf4j dynamic dependency of xwiki-logging
java.io.IOException: For security reasons, Jetty start is unable to process file resource not in ${jetty.base} - webapps/
at org.eclipse.jetty.start.FileInitializer.getDestination(FileInitializer.java:114)
at org.eclipse.jetty.start.fileinits.LocalFileInitializer.create(LocalFileInitializer.java:36)
at org.eclipse.jetty.start.BaseBuilder.processFileResource(BaseBuilder.java:370)
at org.eclipse.jetty.start.BaseBuilder.processFileResources(BaseBuilder.java:407)
at org.eclipse.jetty.start.BaseBuilder.build(BaseBuilder.java:338)
at org.eclipse.jetty.start.Main.start(Main.java:551)
at org.eclipse.jetty.start.Main.main(Main.java:82)
java.io.IOException: For security reasons, Jetty start is unable to process file resource not in ${jetty.base} - logs/
at org.eclipse.jetty.start.FileInitializer.getDestination(FileInitializer.java:114)
at org.eclipse.jetty.start.fileinits.LocalFileInitializer.create(LocalFileInitializer.java:36)
at org.eclipse.jetty.start.BaseBuilder.processFileResource(BaseBuilder.java:370)
at org.eclipse.jetty.start.BaseBuilder.processFileResources(BaseBuilder.java:407)
at org.eclipse.jetty.start.BaseBuilder.build(BaseBuilder.java:338)
at org.eclipse.jetty.start.Main.start(Main.java:551)
at org.eclipse.jetty.start.Main.main(Main.java:82)
java.io.IOException: For security reasons, Jetty start is unable to process file resource not in ${jetty.base} - lib/
at org.eclipse.jetty.start.FileInitializer.getDestination(FileInitializer.java:114)
at org.eclipse.jetty.start.fileinits.LocalFileInitializer.create(LocalFileInitializer.java:36)
at org.eclipse.jetty.start.BaseBuilder.processFileResource(BaseBuilder.java:370)
at org.eclipse.jetty.start.BaseBuilder.processFileResources(BaseBuilder.java:407)
at org.eclipse.jetty.start.BaseBuilder.build(BaseBuilder.java:338)
at org.eclipse.jetty.start.Main.start(Main.java:551)
at org.eclipse.jetty.start.Main.main(Main.java:82)
java.io.IOException: For security reasons, Jetty start is unable to process file resource not in ${jetty.base} - lib/ext/
at org.eclipse.jetty.start.FileInitializer.getDestination(FileInitializer.java:114)
at org.eclipse.jetty.start.fileinits.LocalFileInitializer.create(LocalFileInitializer.java:36)
at org.eclipse.jetty.start.BaseBuilder.processFileResource(BaseBuilder.java:370)
at org.eclipse.jetty.start.BaseBuilder.processFileResources(BaseBuilder.java:407)
at org.eclipse.jetty.start.BaseBuilder.build(BaseBuilder.java:338)
at org.eclipse.jetty.start.Main.start(Main.java:551)
at org.eclipse.jetty.start.Main.main(Main.java:82)
java.io.IOException: For security reasons, Jetty start is unable to process file resource not in ${jetty.base} - logs/
at org.eclipse.jetty.start.FileInitializer.getDestination(FileInitializer.java:114)
at org.eclipse.jetty.start.fileinits.LocalFileInitializer.create(LocalFileInitializer.java:36)
at org.eclipse.jetty.start.BaseBuilder.processFileResource(BaseBuilder.java:370)
at org.eclipse.jetty.start.BaseBuilder.processFileResources(BaseBuilder.java:407)
at org.eclipse.jetty.start.BaseBuilder.build(BaseBuilder.java:338)
at org.eclipse.jetty.start.Main.start(Main.java:551)
at org.eclipse.jetty.start.Main.main(Main.java:82)
java.io.IOException: For security reasons, Jetty start is unable to process file resource not in ${jetty.base} - resources/
at org.eclipse.jetty.start.FileInitializer.getDestination(FileInitializer.java:114)
at org.eclipse.jetty.start.fileinits.LocalFileInitializer.create(LocalFileInitializer.java:36)
at org.eclipse.jetty.start.BaseBuilder.processFileResource(BaseBuilder.java:370)
at org.eclipse.jetty.start.BaseBuilder.processFileResources(BaseBuilder.java:407)
at org.eclipse.jetty.start.BaseBuilder.build(BaseBuilder.java:338)
at org.eclipse.jetty.start.Main.start(Main.java:551)
at org.eclipse.jetty.start.Main.main(Main.java:82)
WARN : Failed to process all file resources.
- [IOException] For security reasons, Jetty start is unable to process file resource not in ${jetty.base} - webapps/ - webapps/
- [IOException] For security reasons, Jetty start is unable to process file resource not in ${jetty.base} - logs/ - logs/
- [IOException] For security reasons, Jetty start is unable to process file resource not in ${jetty.base} - lib/ - lib/
- [IOException] For security reasons, Jetty start is unable to process file resource not in ${jetty.base} - lib/ext/ - lib/ext/
- [IOException] For security reasons, Jetty start is unable to process file resource not in ${jetty.base} - logs/ - logs/
- [IOException] For security reasons, Jetty start is unable to process file resource not in ${jetty.base} - resources/ - resources/
java.lang.RuntimeException: Failed to process all file resources.
- [IOException] For security reasons, Jetty start is unable to process file resource not in ${jetty.base} - webapps/ - webapps/
- [IOException] For security reasons, Jetty start is unable to process file resource not in ${jetty.base} - logs/ - logs/
- [IOException] For security reasons, Jetty start is unable to process file resource not in ${jetty.base} - lib/ - lib/
- [IOException] For security reasons, Jetty start is unable to process file resource not in ${jetty.base} - lib/ext/ - lib/ext/
- [IOException] For security reasons, Jetty start is unable to process file resource not in ${jetty.base} - logs/ - logs/
- [IOException] For security reasons, Jetty start is unable to process file resource not in ${jetty.base} - resources/ - resources/
at org.eclipse.jetty.start.BaseBuilder.processFileResources(BaseBuilder.java:427)
at org.eclipse.jetty.start.BaseBuilder.build(BaseBuilder.java:338)
at org.eclipse.jetty.start.Main.start(Main.java:551)
at org.eclipse.jetty.start.Main.main(Main.java:82)
Usage: java -jar $JETTY_HOME/start.jar [options] [properties] [configs]
java -jar $JETTY_HOME/start.jar --help # for more information
We have:
$ tree -L 1
.
├── data
├── jetty
├── logs
├── resources
├── start.d
├── start_xwiki.bat
├── start_xwiki.sh
├── start_xwiki_debug.bat
├── start_xwiki_debug.sh
├── stop_xwiki.bat
├── stop_xwiki.sh
├── tmp
└── webapps
The --add-modules=xwiki command simply adds the module to the configuration file (and ensures that it's files section is sane), it is not enabled with that command.
The --module=xwiki is the command that enables it.
You can verify the configuration with start.jar --list-config command.
That will show the list of modules enabled, and the properties present from the configuration.
-Djava.io.tmpdir=./tmp -Djetty.home=jetty -Djetty.base=.
All three of those must be absolute paths, not relative.
Also, you should never have the jetty-home or the jetty-base nested inside each other in either direction.
This is bad.
/path/to/dist/jetty-home
/path/to/dist/jetty-home/jetty-base
This is also bad.
/path/to/dist/jetty-base
/path/to/dist/jetty-base/jetty-home
It should be ...
/path/to/dist/jetty-home
/path/to/dist/jetty-base
Thanks @joakime for the recommendations.
However, this shouldn't affect why the jetty.httpConfig.uriCompliance=RFC3986,AMBIGUOUS_PATH_ENCODING property in startd.d/xwiki/ini isn't used in etc/jetty.xml to replace <Property name="jetty.httpConfig.uriCompliance" default="DEFAULT"/> in <Set name="uriCompliance"><Call class="org.eclipse.jetty.http.UriCompliance" name="from"><Arg><Property name="jetty.httpConfig.uriCompliance" default="DEFAULT"/></Arg></Call></Set>, right?
This is what I got with --list-config, it seems the properties are read correctly. I don't know why they're listed as "ee8" properties and if that has some importance or not:
Enabled Modules:
----------------
0) resources transitive provider of resources for xwiki
1) logging/slf4j dynamic dependency of xwiki-logging
2) xwiki-logging transitive provider of logging for threadpool
3) bytebufferpool transitive provider of bytebufferpool for server
ini template available with --add-modules=bytebufferpool
4) client transitive provider of client for ee8-websocket-javax
5) console-capture transitive provider of console-capture for xwiki
ini template available with --add-modules=console-capture
6) ext transitive provider of ext for xwiki
7) threadpool transitive provider of threadpool for server
ini template available with --add-modules=threadpool
8) server transitive provider of server for xwiki
ini template available with --add-modules=server
9) deploy transitive provider of deploy for ee8-deploy
10) sessions transitive provider of sessions for ee8-servlet
ini template available with --add-modules=sessions
11) ee8-servlet transitive provider of ee8-servlet for ee8-security
12) security transitive provider of security for ee8-security
13) ee8-security transitive provider of ee8-security for ee8-plus
14) ee-webapp transitive provider of ee-webapp for ee8-webapp
ini template available with --add-modules=ee-webapp
15) ee8-webapp transitive provider of ee8-webapp for ee8-plus
ini template available with --add-modules=ee8-webapp
16) plus transitive provider of plus for ee8-annotations
17) jndi transitive provider of jndi for ee8-plus
18) ee8-plus transitive provider of ee8-plus for ee8-annotations
19) ee8-annotations transitive provider of ee8-annotations for xwiki
20) ee8-apache-jsp transitive provider of ee8-apache-jsp for xwiki
21) ee8-deploy transitive provider of ee8-deploy for xwiki
ini template available with --add-modules=ee8-deploy
22) ee8-websocket-javax transitive provider of ee8-websocket-javax for xwiki
23) ee8-websocket-jetty transitive provider of ee8-websocket-jetty for xwiki
24) http transitive provider of http for xwiki
ini template available with --add-modules=http
25) http-forwarded transitive provider of http-forwarded for xwiki
ini template available with --add-modules=http-forwarded
26) requestlog transitive provider of requestlog for xwiki
ini template available with --add-modules=requestlog
27) xwiki ${jetty.base}/start.d/xwiki.ini
JVM Version & Properties:
-------------------------
java.home = /Library/Java/JavaVirtualMachines/jdk-17.jdk/Contents/Home
java.vm.vendor = Oracle Corporation
java.vm.version = 17+35-LTS-2724
java.vm.name = Java HotSpot(TM) 64-Bit Server VM
java.vm.info = mixed mode, sharing
java.runtime.name = Java(TM) SE Runtime Environment
java.runtime.version = 17+35-LTS-2724
java.io.tmpdir = ./tmp
user.dir = /Users/vmassol/dev/xwiki/xwiki-platform/xwiki-platform-core/xwiki-platform-rest/xwiki-platform-rest-test/xwiki-platform-rest-test-tests/target/xwiki
user.language = en
user.country = FR
Jetty Version & Properties:
---------------------------
jetty.version = 12.0.12
jetty.tag.version = jetty-12.0.12
jetty.build = cc6f1b74db755fed228b50701ad967aeaa68e83f
jetty.home = jetty
jetty.base = .
Config Search Order:
--------------------
<command-line>
${jetty.base} -> /Users/vmassol/dev/xwiki/xwiki-platform/xwiki-platform-core/xwiki-platform-rest/xwiki-platform-rest-test/xwiki-platform-rest-test-tests/target/xwiki/.
${jetty.home} -> /Users/vmassol/dev/xwiki/xwiki-platform/xwiki-platform-core/xwiki-platform-rest/xwiki-platform-rest-test/xwiki-platform-rest-test-tests/target/xwiki/jetty
System Properties:
------------------
(no system properties specified)
Properties: Jetty
-----------------
STOP.KEY = xwiki
STOP.PORT = 8079
java.version = 17
java.version.major = 17
java.version.micro = 0
java.version.minor = 0
java.version.platform = 17
jetty.http.port = 8080
jetty.requestlog.dir = logs
jetty.webapp.addHiddenClasses = org.eclipse.jetty.logging.,${jetty.home.uri}/lib/logging/,org.slf4j.
runtime.feature.alpn = true
slf4j.version = 2.0.13
Classpath: Jetty
----------------
Version Information on 17 entries in the classpath.
Note: order presented here is how they would appear on the classpath.
changes to the --module=name command line options will be reflected here.
0: (dir) | ${jetty.home}/resources
1: 2.0.13 | ${jetty.home}/lib/logging/slf4j-api-2.0.13.jar | http://www.opensource.org/licenses/mit-license.php
2: 12.0.12 | ${jetty.home}/lib/logging/jetty-slf4j-impl-12.0.12.jar | EPL-2.0 OR Apache-2.0
3: 12.0.12 | ${jetty.home}/lib/jetty-client-12.0.12.jar | EPL-2.0 OR Apache-2.0
4: 2.16.1 | ${jetty.home}/lib/ext/commons-io-2.16.1.jar | https://www.apache.org/licenses/LICENSE-2.0.txt
5: 16.7.0-SNAPSHOT | ${jetty.home}/lib/ext/xwiki-platform-tool-jetty-listener-16.7.0-20240814.065843-28.jar | GNU LESSER GENERAL PUBLIC LICENSE Version 2.1, February 1999
6: 12.0.12 | ${jetty.home}/lib/jetty-http-12.0.12.jar | EPL-2.0 OR Apache-2.0
7: 12.0.12 | ${jetty.home}/lib/jetty-server-12.0.12.jar | EPL-2.0 OR Apache-2.0
8: 12.0.12 | ${jetty.home}/lib/jetty-xml-12.0.12.jar | EPL-2.0 OR Apache-2.0
9: 12.0.12 | ${jetty.home}/lib/jetty-util-12.0.12.jar | EPL-2.0 OR Apache-2.0
10: 12.0.12 | ${jetty.home}/lib/jetty-io-12.0.12.jar | EPL-2.0 OR Apache-2.0
11: 12.0.12 | ${jetty.home}/lib/jetty-deploy-12.0.12.jar | EPL-2.0 OR Apache-2.0
12: 12.0.12 | ${jetty.home}/lib/jetty-session-12.0.12.jar | EPL-2.0 OR Apache-2.0
13: 12.0.12 | ${jetty.home}/lib/jetty-security-12.0.12.jar | EPL-2.0 OR Apache-2.0
14: 12.0.12 | ${jetty.home}/lib/jetty-ee-12.0.12.jar | EPL-2.0 OR Apache-2.0
15: 12.0.12 | ${jetty.home}/lib/jetty-plus-12.0.12.jar | EPL-2.0 OR Apache-2.0
16: 12.0.12 | ${jetty.home}/lib/jetty-jndi-12.0.12.jar | EPL-2.0 OR Apache-2.0
Active XMLs: Jetty
------------------
${jetty.home}/etc/jetty-bytebufferpool.xml
${jetty.home}/etc/console-capture.xml
${jetty.home}/etc/jetty-threadpool.xml
${jetty.home}/etc/jetty.xml
${jetty.home}/etc/jetty-deploy.xml
${jetty.home}/etc/sessions/id-manager.xml
${jetty.home}/etc/jetty-ee-webapp.xml
${jetty.home}/etc/jetty-http.xml
${jetty.home}/etc/jetty-http-forwarded.xml
${jetty.home}/etc/jetty-requestlog.xml
Properties: ee8
---------------
contextHandlerClass = org.eclipse.jetty.ee8.webapp.WebAppContext
ee8.asm.version = 9.7
ee8.jakarta.annotation.api.version = 1.3.5
ee8.jsp.impl.version = 9.0.90
jetty.deploy.extractWars = false
jetty.deploy.scanInterval = 0
jetty.httpConfig.uriCompliance = RFC3986,AMBIGUOUS_PATH_ENCODING
Classpath: ee8
--------------
Version Information on 21 entries in the classpath.
Note: order presented here is how they would appear on the classpath.
changes to the --module=name command line options will be reflected here.
0: 4.0.6 | ${jetty.home}/lib/jetty-servlet-api-4.0.6.jar | http://www.apache.org/licenses/LICENSE-2.0, http://www.eclipse.org/org/documents/epl-v10.php
1: 12.0.12 | ${jetty.home}/lib/jetty-ee8-nested-12.0.12.jar | EPL-2.0 OR Apache-2.0
2: 12.0.12 | ${jetty.home}/lib/jetty-ee8-servlet-12.0.12.jar | EPL-2.0 OR Apache-2.0
3: 12.0.12 | ${jetty.home}/lib/jetty-ee8-security-12.0.12.jar | EPL-2.0 OR Apache-2.0
4: 12.0.12 | ${jetty.home}/lib/jetty-ee8-webapp-12.0.12.jar | EPL-2.0 OR Apache-2.0
5: 12.0.12 | ${jetty.home}/lib/jetty-ee8-plus-12.0.12.jar | EPL-2.0 OR Apache-2.0
6: 1.3.3 | ${jetty.home}/lib/jakarta.transaction-api-1.3.3.jar | http://www.eclipse.org/legal/epl-2.0, https://www.gnu.org/software/classpath/license.html
7: 12.0.12 | ${jetty.home}/lib/jetty-ee8-annotations-12.0.12.jar | EPL-2.0 OR Apache-2.0
8: 9.7 | ${jetty.home}/lib/ee8-annotations/asm-9.7.jar | BSD-3-Clause;link=https://asm.ow2.io/LICENSE.txt
9: 9.7 | ${jetty.home}/lib/ee8-annotations/asm-commons-9.7.jar | BSD-3-Clause;link=https://asm.ow2.io/LICENSE.txt
10: 9.7 | ${jetty.home}/lib/ee8-annotations/asm-tree-9.7.jar | BSD-3-Clause;link=https://asm.ow2.io/LICENSE.txt
11: 1.3.5 | ${jetty.home}/lib/ee8-annotations/jakarta.annotation-api-1.3.5.jar | http://www.eclipse.org/legal/epl-2.0, https://www.gnu.org/software/classpath/license.html
12: 9.0.90 | ${jetty.home}/lib/ee8-apache-jsp/apache-el-9.0.90.jar | http://www.apache.org/licenses/LICENSE-2.0
13: 12.0.12 | ${jetty.home}/lib/jetty-websocket-core-common-12.0.12.jar | EPL-2.0 OR Apache-2.0
14: 12.0.12 | ${jetty.home}/lib/jetty-websocket-core-client-12.0.12.jar | EPL-2.0 OR Apache-2.0
15: 12.0.12 | ${jetty.home}/lib/jetty-websocket-core-server-12.0.12.jar | EPL-2.0 OR Apache-2.0
16: 12.0.12 | ${jetty.home}/lib/ee8-websocket/jetty-ee8-websocket-servlet-12.0.12.jar | EPL-2.0 OR Apache-2.0
17: 1.1.2 | ${jetty.home}/lib/ee8-websocket/jetty-javax-websocket-api-1.1.2.jar | http://www.apache.org/licenses/LICENSE-2.0, http://www.eclipse.org/org/documents/epl-v10.php
18: 12.0.12 | ${jetty.home}/lib/ee8-websocket/jetty-ee8-websocket-javax-client-12.0.12.jar | EPL-2.0 OR Apache-2.0
19: 12.0.12 | ${jetty.home}/lib/ee8-websocket/jetty-ee8-websocket-javax-common-12.0.12.jar | EPL-2.0 OR Apache-2.0
20: 12.0.12 | ${jetty.home}/lib/ee8-websocket/jetty-ee8-websocket-javax-server-12.0.12.jar | EPL-2.0 OR Apache-2.0
Active XMLs: ee8
----------------
${jetty.home}/etc/jetty-ee8-webapp.xml
${jetty.home}/etc/jetty-ee8-deploy.xml
${jetty.home}/etc/jetty-xwiki.xml
Any idea what could be wrong?
Thx
I don't know why they're listed as "ee8" properties and if that has some importance or not:
Probably because xwiki.ini is the match for xwiki.mod which has:
[environment]
ee8
As you can see in your output ...
Properties: ee8
---------------
contextHandlerClass = org.eclipse.jetty.ee8.webapp.WebAppContext
ee8.asm.version = 9.7
ee8.jakarta.annotation.api.version = 1.3.5
ee8.jsp.impl.version = 9.0.90
jetty.deploy.extractWars = false
jetty.deploy.scanInterval = 0
jetty.httpConfig.uriCompliance = RFC3986,AMBIGUOUS_PATH_ENCODING
That property is being placed into the ee8 environment, very much due to your module definition.
So it is not available to the core level configuration for the Connectors.
Another way to see this ...
start.jar --list-config=args- shows what arguments (and properties) are used at core.start.jar --list-config=envs- shows what each environment is setup as (look for--env <name>as separator between each environment)
Thanks @joakime
I've now removed the following from xwiki.mod and the properties from xwiki.ini are now defined as core and thus the jetty.httpConfig.uriCompliance property is seen from jetty.xml:
[environment]
ee8
All working!
Only remaining question I guess is whether there's a syntax to force a property to be defined as core?
Only remaining question I guess is whether there's a syntax to force a property to be defined as core?
That would mean 2 separate modules (one for server, one for ee8) You could have ...
xwiki-server.mod- where you put the server level configuration that xwiki wants (no[environment]section)xwiki-ee8.mod- where you put the ee8 level configuration that xwiki webapp wants (with[environment]section atee8)xwiki.mod- meta module that really only depends onxwiki-ee8, andxwiki-server
@vmassol the problem, as @joakime pointed out, is that you have defined a core property as being in the ee8 environment, which means it is not available for substitution when the jetty.xml file (a core file) is read before any of the environments are activated. The appropriate place to define these core property values is in the matching core .ini file, which in this case is server.ini. Since we moved to environments, a property can't be defined just anywhere on the command line (or in any module), it must be defined appropriately:
a) if specifying things on the command line, the property must be defined before the module or xml file that uses it
b) if using only modules, the property must be defined before or in the module that actually uses the property
I think we should update our doco to be clearer about this change.
@janbartel Thanks a lot for the additional explanations! So far I've removed the ee8 environment from my xwiki.mod. I could add it back and instead add a servet.ini file in start.d.
TBH I'm not sure what adding the ee8 environment does. I have the feeling it's useful if you want to run the same jetty engine several times with different environments, as a way to differentiate what gets defined. In our case, we run a single environment so I guess it doesn't matter.
Thanks again