Could XML Input Factory leak file handles ultimately giving OutOfFileHandles

[sjeevan09]

Hi,

Describe the bug

I am using jackson-databind in my AWS Lambda developed in Java 8. Along with related jackson libraries ... a screen shot of my pom.xml is attached.

My Lambda works beautifully and parses the XML into java object as expected ... but under heavy load I start getting an exception that points in the direction - is the Jackson Databind library leaking file handles/ descriptors.

I get too many open files.

Version information 2.13.0 Snap of pom.xml below

To Reproduce My code is a AWS Lambda which works in a constrained Linux environment with only 1024 File handles available. The error comes when I request parsing of an XML object into Java object very frequently (load test)

Code snippet attached as a screen shot.

Java stack trace also attached

Could XML Input Factory leak file handles ultimately giving OutOfFileHandles

Provider for class javax.xml.stream.XMLInputFactory cannot be created: javax.xml.stream.FactoryConfigurationError
javax.xml.stream.FactoryConfigurationError: Provider for class javax.xml.stream.XMLInputFactory cannot be created
	at javax.xml.stream.FactoryFinder.findServiceProvider(FactoryFinder.java:370)
	at javax.xml.stream.FactoryFinder.find(FactoryFinder.java:313)
	at javax.xml.stream.FactoryFinder.find(FactoryFinder.java:227)
	at javax.xml.stream.XMLInputFactory.newInstance(XMLInputFactory.java:154)
	at com.fasterxml.jackson.dataformat.xml.XmlFactory.<init>(XmlFactory.java:113)
	at com.fasterxml.jackson.dataformat.xml.XmlFactory.<init>(XmlFactory.java:100)
	at com.fasterxml.jackson.dataformat.xml.XmlFactory.<init>(XmlFactory.java:84)
	at com.fasterxml.jackson.dataformat.xml.XmlMapper.<init>(XmlMapper.java:122)
	at com.ppg.silverprinting.lambda.utilities.PicturesXML.<init>(PicturesXML.java:52)
	at com.ppg.silverprinting.lambda.AbstractProcessor.<init>(AbstractProcessor.java:99)
	at com.ppg.silverprinting.lambda.AbstractInfotechProcessor.<init>(AbstractInfotechProcessor.java:24)
	at com.ppg.silverprinting.lambda.InfotechProcessor.<init>(InfotechProcessor.java:40)
	at com.ppg.silverprinting.lambda.InfotechGeneratorHandler.invokeProcessor(InfotechGeneratorHandler.java:82)
	at com.ppg.silverprinting.lambda.entry.SilverPrintProcessor.processBase64EncodedRecord(SilverPrintProcessor.java:69)
	at com.ppg.silverprinting.lambda.entry.SilverprintRequestHandler.handleRequest(SilverprintRequestHandler.java:68)
	at com.ppg.silverprinting.lambda.entry.SilverprintRequestHandler.handleRequest(SilverprintRequestHandler.java:17)
	Caused by: java.lang.RuntimeException: Provider for class javax.xml.stream.XMLInputFactory cannot be created
	at javax.xml.stream.FactoryFinder.findServiceProvider(FactoryFinder.java:367)
... 15 more
	Caused by: java.util.ServiceConfigurationError: javax.xml.stream.XMLInputFactory: Error reading configuration file
	at java.util.ServiceLoader.fail(ServiceLoader.java:232)
	at java.util.ServiceLoader.parse(ServiceLoader.java:309)
	at java.util.ServiceLoader.access$200(ServiceLoader.java:185)
	at java.util.ServiceLoader$LazyIterator.hasNextService(ServiceLoader.java:357)
	at java.util.ServiceLoader$LazyIterator.hasNext(ServiceLoader.java:393)
	at java.util.ServiceLoader$1.hasNext(ServiceLoader.java:474)
	at javax.xml.stream.FactoryFinder$1.run(FactoryFinder.java:352)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.xml.stream.FactoryFinder.findServiceProvider(FactoryFinder.java:341)
... 15 more
Caused by: java.io.FileNotFoundException: /var/task/META-INF/services/javax.xml.stream.XMLInputFactory (Too many open files)
	at java.io.FileInputStream.open0(Native Method)
	at java.io.FileInputStream.open(FileInputStream.java:195)
	at java.io.FileInputStream.<init>(FileInputStream.java:138)
	at java.io.FileInputStream.<init>(FileInputStream.java:93)
	at sun.net.www.protocol.file.FileURLConnection.connect(FileURLConnection.java:90)
	at sun.net.www.protocol.file.FileURLConnection.getInputStream(FileURLConnection.java:188)
	at java.net.URL.openStream(URL.java:1093)
	at java.util.ServiceLoader.parse(ServiceLoader.java:304)
	... 22 more

Snap of my pom.xml

		<dependency>
			<groupId>com.fasterxml.jackson.dataformat</groupId>
			<artifactId>jackson-dataformat-xml</artifactId>
			<version>2.11.0</version>
		</dependency> 
		<dependency>
			<groupId>com.fasterxml.jackson.core</groupId>
			<artifactId>jackson-core</artifactId>
			<version>2.11.0</version>
		</dependency>
		<dependency>
			<groupId>com.fasterxml.jackson.core</groupId>
			<artifactId>jackson-databind</artifactId>
			<version>2.11.0</version>
		</dependency>
		<dependency>
			<groupId>com.fasterxml.jackson.core</groupId>
			<artifactId>jackson-annotations</artifactId>
			<version>2.11.0</version>
		</dependency>

My code ....

public class PicturesXML {

	private static final Logger LOG = LogManager.getLogger(PicturesXML.class);
 
    public static Pictures parse(byte[] picturesByteArray) throws JsonParseException, JsonMappingException, IOException { 
		ObjectMapper objectMapper = new XmlMapper();
		Pictures toReturn;
		toReturn = objectMapper.readValue(picturesByteArray, Pictures.class);
		return toReturn;
    }

    public static Pictures read(String bktName, String xmlLoc) 
										throws JsonParseException, JsonMappingException, IOException { 
	    byte[] data = S3Utilities.getByteArray(bktName, xmlLoc); //, logger);
	    Pictures pics = parse(data);
	    pics = Pictures.filterOutNonPictures(pics.getPicture());
	    return pics;
	}
}

thanks Jeevan

[cowtowncoder]

Wrong repo, will transfer.

[cowtowncoder]

I am sorry but this is not enough to really reproduce the problem. I am not aware of file handle leakage, and code included does not have anything obvious.

One thing I would suggest, however, would be moving this line:

ObjectMapper objectMapper = new XmlMapper();

from within method into static initializer like

final static ObjectMapper objectMapper = new XmlMapper();

I am not sure if that helps in Lambda environment -- it really depends on whether parse is only called once per JVM setup or so. But if it is called multiple times this makes a HUGE difference: construction of ObjectMapper is a very heavy operation (or more specifically, first read/write call as much of initialization happens lazily).

Other than that I cannot do much without some form of reproduction of the issue.