apache
Classloader issues when running Lucene under a java SecurityManager [LUCENE-5471]
I see the following error when I run Lucene 4.5.0 under a java SecurityManager. I will attach a test program which shows this problem. The program works fine when a SecurityManager is not installed. But the program fails when I install a SecurityManager. Even more puzzling, the program works if I first run it without a SecurityManager, then install a SecurityManager, then re-run the program, all within the lifetime of a single JVM. I would appreciate advice about how to work around this problem:
Exception in thread "main" java.lang.ExceptionInInitializerError at org.apache.lucene.index.LiveIndexWriterConfig.<init>(LiveIndexWriterConfig.java:122) at org.apache.lucene.index.IndexWriterConfig.<init>(IndexWriterConfig.java:165) at SecureLucene$1.run(SecureLucene.java:129) at SecureLucene$1.run(SecureLucene.java:122) at java.security.AccessController.doPrivileged(Native Method) at SecureLucene.getIndexWriter(SecureLucene.java:120) at SecureLucene.runTest(SecureLucene.java:72) at SecureLucene.main(SecureLucene.java:52) Caused by: java.lang.IllegalArgumentException: A SPI class of type org.apache.lucene.codecs.Codec with name 'Lucene45' does not exist. You need to add the corresponding JAR file supporting this SPI to your classpath.The current classpath supports the following names: [] at org.apache.lucene.util.NamedSPILoader.lookup(NamedSPILoader.java:109) at org.apache.lucene.codecs.Codec.forName(Codec.java:95) at org.apache.lucene.codecs.Codec.<clinit>(Codec.java:122) ... 8 more
Migrated from LUCENE-5471 by Richard N. Hillegas, updated Mar 03 2014 Attachments: SecureLucene.java (versions: 2) Linked issues:
Richard N. Hillegas (migrated from JIRA)
Attaching SecureLucene.java, a program which shows this problem. To run the program, I put the program and the following jar files on the classpath:
lucene-analyzers-common-4.5.0.jar lucene-core-4.5.0.jar lucene-queryparser-4.5.0.jar
I see these successful results...
> java SecureLucene Running unsecured... Success! Running with security manager... Success!
...and this unsuccessful result...
> java SecureLucene true Running with security manager... Exception in thread "main" java.lang.ExceptionInInitializerError at org.apache.lucene.index.LiveIndexWriterConfig.<init>(LiveIndexWriterConfig.java:122) at org.apache.lucene.index.IndexWriterConfig.<init>(IndexWriterConfig.java:165) at SecureLucene$1.run(SecureLucene.java:129) at SecureLucene$1.run(SecureLucene.java:122) at java.security.AccessController.doPrivileged(Native Method) at SecureLucene.getIndexWriter(SecureLucene.java:120) at SecureLucene.runTest(SecureLucene.java:72) at SecureLucene.main(SecureLucene.java:52) Caused by: java.lang.IllegalArgumentException: A SPI class of type org.apache.lucene.codecs.Codec with name 'Lucene45' does not exist. You need to add the corresponding JAR file supporting this SPI to your classpath.The current classpath supports the following names: [] at org.apache.lucene.util.NamedSPILoader.lookup(NamedSPILoader.java:109) at org.apache.lucene.codecs.Codec.forName(Codec.java:95) at org.apache.lucene.codecs.Codec.<clinit>(Codec.java:122) ... 8 more
Robert Muir (@rmuir) (migrated from JIRA)
I think its a bug in your securitymanager config (e.g. you are missing reflectpermission or something really necessary).
We use lucene in a securitymanager when running tests, see config here: http://svn.apache.org/repos/asf/lucene/dev/trunk/lucene/tools/junit4/tests.policy
Richard N. Hillegas (migrated from JIRA)
Thanks, Robert. After adding all of those permissions to my policy file, the test runs cleanly. Thanks.
Chris M. Hostetter (@hossman) (migrated from JIRA)
Even more puzzling, the program works if I first run it without a SecurityManager, then install a SecurityManager, then re-run the program, all within the lifetime of a single JVM.
I suspect this is because of when/how SPI is used - if the codec is resolved before you load the security manager, then that info is still available in NamedSPILoader.
if you did a NamedSPILoader.reload() on your classloader after activating the SecurityManager, you'd probably see the same problem no matter what.
I think its a bug in your securitymanager config (e.g. you are missing reflectpermission or something really necessary).
yeah ... w/o showing us what security manager you are using, it's hard to guess what the specific problem might be.
that said: it's strange that you got an IllegalArgumentException from NamedSPILoader instead of a SecurityException ... i can't think of any (legitimate) reason why that should have happened, unless we're swallowing RuntimeExceptions in the SPI Loader code?
Uwe Schindler (@uschindler) (migrated from JIRA)
Hoss: We don't swallow the SecurityException. I think, with a SecurityManager enabled and some missing persmissions, stuff like ClassLoader#getResources() simply don't work and return an empty enumeration. So it does not find any resources from classloader, because code has no permissions to "see" the code.
This is defined in the spec - see http://docs.oracle.com/javase/7/docs/api/java/lang/ClassLoader.html#getResources(java.lang.String): "An enumeration of URL objects for the resource. If no resources could be found, the enumeration will be empty. Resources that the class loader doesn't have access to will not be in the enumeration." This is a bit unclear, but getResource() is very good documented: "A URL object for reading the resource, or null if the resource could not be found or the invoker doesn't have adequate privileges to get the resource."
So there is nothing we can do against that.
Chris M. Hostetter (@hossman) (migrated from JIRA)
null if the resource could not be found or the invoker doesn't have adequate privileges to get the resource.
Grr... yeah, i forgot about that ... lack of permissions for accessing "thing" frequently looks the same as if "thing" doesn't exist.
We could, conceivably, ask the SecurityManager if we have the permissions we know we are going to need to do the things we want to do though, correct? and then if we get a SecurityException, wrap it in another SecurityException with details about why lucnee can't work w/o that permision?
(I'm not saying i'm convinced we should – just wondering if other people think it would be helpful)
Uwe Schindler (@uschindler) (migrated from JIRA)
I thought about that, too. Question is: What privilege is needed to do this? The problem is: depending on the ClassLoader we may see some stuff, others not. Has to do with the crazy codebase attribute in the policy file. You may only see some specific JAR files or only some java packages. So mostly impossible to get the real reason, without the code down in the ClassLoader hierarchy throwing a SecurityEx. I think we should keep it as it is. Maybe document it, that you need permissions to "see" the JAR files and resources in it.
Richard N. Hillegas (migrated from JIRA)
Thanks for the help and the discussion so far, Hoss and Uwe.
Attaching a second rev of the SecureLucene test program. This version pares back the permissions in order to expose the minimal attack surface which I can configure by myself. Here are the minimal permissions which the test program grants in order to run successfully under a Java Security Manager:
// permissions granted to Lucene
grant codeBase "file:/Users/rh161140/derby/derby-590/trunk/tools/java/lucene-core-4.5.0.jar"
{
// permissions for file access, write access only to sandbox:
permission java.io.FilePermission "<<ALL FILES>>", "read";
permission java.io.FilePermission "/Users/rh161140/derby/derby-590/luceneTest", "read,write,delete";
permission java.io.FilePermission "/Users/rh161140/derby/derby-590/luceneTest/-", "read,write,delete";
// Basic permissions needed for Lucene to work:
permission java.util.PropertyPermission "user.dir", "read";
permission java.util.PropertyPermission "sun.arch.data.model", "read";
permission java.lang.reflect.ReflectPermission "*";
permission java.lang.RuntimePermission "*";
};
// permissions granted to the application
grant codeBase "file:/Users/rh161140/src/"
{
// permissions for file access, write access only to sandbox:
permission java.io.FilePermission "<<ALL FILES>>", "read";
permission java.io.FilePermission "/Users/rh161140/derby/derby-590/luceneTest", "read,write";
permission java.io.FilePermission "/Users/rh161140/derby/derby-590/luceneTest/-", "read,write,delete";
// Basic permissions needed for Lucene to work:
permission java.util.PropertyPermission "user.dir", "read";
permission java.util.PropertyPermission "sun.arch.data.model", "read";
};
I have some follow on comments and questions:
-
Is it really necessary to grant Lucene every RuntimePermission and the privilege to read every file in the file system? Maybe these grants can be tightened.
-
I don't understand why the calling, application code needs to be granted any permissions. Maybe some more privilege blocks could be added to the Lucene code? In particular, it seems a shame that the application has to be granted the privilege to read every file in the file system.
-
Most of the application permissions are self-revealing. That is, if I omit one of them, then I get an exception telling me that the permission needs to be granted. However, that is not the case for the first permission granted to the application...
permission java.io.FilePermission "<<ALL FILES>>", "read";
...Without that permission, I get the original puzzling exception: "Caused by: java.lang.IllegalArgumentException: A SPI class of type org.apache.lucene.codecs.Codec...", which doesn't really tell me what the problem is. Maybe the wording of that exception could be improved so that the user can be told that one of its root causes is a failure to grant the application and Lucene read access to every file in the file system.
Thanks, -Rick
Richard N. Hillegas (migrated from JIRA)
Attaching a third version of SecureLucene.java. I have wrapped a privilege block around the application's Lucene calls and pruned some more permissions. This significantly reduces the attack surface.
At this point, I do not need to grant Lucene any runtime or reflection permissions and I do not need to grant Lucene read access to all of the files in the file system. I do not need to grant the application read access to all files in the file system, either. It turns out that Codec loading simply depends on the ability to read the Lucene core jar file.
Here are the reduced permissions which I need to grant:
grant codeBase "file:/Users/rh161140/derby/derby-590/trunk/tools/java/lucene-core-4.5.0.jar"
{
// permissions for file access, write access only to sandbox:
permission java.io.FilePermission "/Users/rh161140/derby/derby-590/luceneTest", "read,write,delete";
permission java.io.FilePermission "/Users/rh161140/derby/derby-590/luceneTest/-", "read,write,delete";
// Basic permissions needed for Lucene to work:
permission java.util.PropertyPermission "user.dir", "read";
permission java.util.PropertyPermission "sun.arch.data.model", "read";
};
grant codeBase "file:/Users/rh161140/src/"
{
// permissions for file access, write access only to sandbox:
permission java.io.FilePermission "/Users/rh161140/derby/derby-590/trunk/tools/java/lucene-core-4.5.0.jar", "read";
permission java.io.FilePermission "/Users/rh161140/derby/derby-590/luceneTest", "read,write";
permission java.io.FilePermission "/Users/rh161140/derby/derby-590/luceneTest/-", "read,write,delete";
// Basic permissions needed for Lucene to work:
permission java.util.PropertyPermission "user.dir", "read";
permission java.util.PropertyPermission "sun.arch.data.model", "read";
};
SecurityManager is now deprecated for removal, so this issue might no longer be relevant going forward.
