unicodetools jsp: properties.jsp fails, missing

@iwsfutcmd found this when using the docker image of a recent build:

properties.jsp failed as below.

Indeed, /var/lib/jetty/data/unicodetools/Generated/BIN/UCD_Data16.0.0.bin doesn't exist.

Did something change in the build process? assign this back to me if you have comments

Stacktrace:
	at org.apache.jasper.servlet.JspServletWrapper.handleJspException(JspServletWrapper.java:605)
	at org.apache.jasper.servlet.JspServletWrapper.service(JspServletWrapper.java:498)
	at org.apache.jasper.servlet.JspServlet.serviceJspFile(JspServlet.java:383)
	at org.apache.jasper.servlet.JspServlet.service(JspServlet.java:331)
	at org.eclipse.jetty.jsp.JettyJspServlet.service(JettyJspServlet.java:106)
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:790)
	at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1450)
	at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799)
	at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1656)
	at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:292)
	at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193)
	at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1626)
	at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:552)
	at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
	at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600)
	at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)
	at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235)
	at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624)
	at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233)
	at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440)
	at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188)
	at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:505)
	at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594)
	at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186)
	at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355)
	at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
	at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:234)
	at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146)
	... 16 more
Caused by: org.unicode.text.utility.ChainException: Cant build data file for 16.0.0 ({0})
	at org.unicode.text.UCD.UCD.fillFromFile(UCD.java:1972)
	at org.unicode.text.UCD.UCD.make(UCD.java:61)
	at org.unicode.text.UCD.ToolUnicodePropertySource.<init>(ToolUnicodePropertySource.java:86)
	at org.unicode.text.UCD.ToolUnicodePropertySource.make(ToolUnicodePropertySource.java:79)
	at org.unicode.text.UCD.VersionedProperty.set(VersionedProperty.java:133)
	at org.unicode.jsp.XPropertyFactory.getProperty(XPropertyFactory.java:67)
	at org.unicode.jsp.UnicodeUtilities.showProperties(UnicodeUtilities.java:1464)
	at org.unicode.jsp.UnicodeJsp.showProperties(UnicodeJsp.java:135)
	at org.apache.jsp.character_jsp._jspService(character_jsp.java:306)
	at org.apache.jasper.runtime.HttpJspBase.service(HttpJspBase.java:70)
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:790)
	at org.apache.jasper.servlet.JspServletWrapper.service(JspServletWrapper.java:465)
	... 42 more
Caused by: java.io.FileNotFoundException: /var/lib/jetty/data/unicodetools/Generated/BIN/UCD_Data16.0.0.bin (Permission denied)
	at java.base/java.io.FileOutputStream.open0(Native Method)
	at java.base/java.io.FileOutputStream.open(Unknown Source)
	at java.base/java.io.FileOutputStream.<init>(Unknown Source)
	at java.base/java.io.FileOutputStream.<init>(Unknown Source)
	at org.unicode.text.UCD.ConvertUCD.writeJavaData(ConvertUCD.java:762)
	at org.unicode.text.UCD.ConvertUCD.toJava(ConvertUCD.java:371)
	at org.unicode.text.UCD.ConvertUCD.main(ConvertUCD.java:302)
	at org.unicode.text.UCD.UCD.fillFromFile(UCD.java:1970)
	... 53 more

Jan 28 '25 20:01 srl295

it's trying to write and has permission denied. I can fix permissions on that directory
however, shouldn't we have such data precalculated?

Jan 28 '25 20:01 srl295

however, shouldn't we have such data precalculated?

We have the IndexUnicodeProperties data precalculated, but probably not the ToolUnicodeProperties data which lurks in the darker and more ancient corners of this code base.

Jan 28 '25 20:01 eggrobin

@eggrobin i'm not sure what either of those are…

Jan 28 '25 20:01 srl295

There are two main implementations of UCD parsing and data structures for accessing the UCD in here:

org.unicode.text.UCD, vintage 1996, which has ToolUnicodeProperties built on top of it;
the shiny new (2011) https://github.com/unicode-org/unicodetools/blob/main/unicodetools/src/main/java/org/unicode/props/IndexUnicodeProperties.java, which more systematically parses everything and which we rely on in the invariant tests etc.

Someday, and that day may never come, we will retire org.unicode.text.UCD… See #639.

But to the issue at hand, we should have write permissions to Generated. It is weird that you don’t, because when I do a docker build myself I do not run into that issue… I did change the Dockerfile a while back to create the Generated directory: https://github.com/unicode-org/unicodetools/blame/main/UnicodeJsps/Dockerfile#L28.

Jan 28 '25 20:01 eggrobin

maybe when it does the copy it has more restrictive permissions.

But again, shouldn't we pre-create the content in Generated rather than have it created on the fly in each Cloud Run instantiation?

Jan 28 '25 22:01 srl295

But again, shouldn't we pre-create the content in Generated rather than have it created on the fly in each Cloud Run instantiation?

In principle, sure, in practice, it’s not that costly (compared to, e.g., just loading the much larger precomputed IUP data), and making sure that we have generated everything we need is tricky (especially since we shouldn’t need too much, as this is the legacy part: like the new IUP, the old TUP is also capable of handling old versions of the UCD, albeit with questionable correctness, but I don’t know how much of that can actually be hit from the JSPs). And investing too much energy into the twenty-nine year old part of the infrastructure rather than the fourteen year old part seems unwise.

Jan 31 '25 00:01 eggrobin