osm-lib
osm-lib copied to clipboard
Array overrun while writing planet.vex
While writing planet.vex:
19:27:55.007 [Thread-0] DEBUG com.conveyal.osmlib.VEXBlock - Contained 5263 entities with type 2.
Exception in thread "main" java.lang.ArrayIndexOutOfBoundsException: 1048576
at com.conveyal.osmlib.DeflatedBlockWriter.write(DeflatedBlockWriter.java:129)
at java.io.OutputStream.write(OutputStream.java:116)
at java.io.OutputStream.write(OutputStream.java:75)
at com.conveyal.osmlib.VarIntOutputStream.writeBytes(VarIntOutputStream.java:52)
at com.conveyal.osmlib.VarIntOutputStream.writeString(VarIntOutputStream.java:58)
at com.conveyal.osmlib.VexOutput.writeRelation(VexOutput.java:168)
at com.conveyal.osmlib.PBFInput.parseRelations(PBFInput.java:190)
at org.openstreetmap.osmosis.osmbinary.BinaryParser.parse(BinaryParser.java:122)
at org.openstreetmap.osmosis.osmbinary.BinaryParser.handleBlock(BinaryParser.java:68)
at org.openstreetmap.osmosis.osmbinary.file.FileBlock.process(FileBlock.java:135)
at org.openstreetmap.osmosis.osmbinary.file.BlockInputStream.process(BlockInputStream.java:34)
at com.conveyal.osmlib.PBFInput.copyTo(PBFInput.java:237)
at com.conveyal.osmlib.main.Converter.main(Converter.java:32)
This is because there are OSM entities bigger than com.conveyal.osmlib.DeflatedBlockWriter#MAX_MESSAGE_SIZE
Temporary fix in 26deed74bb2caeedc13a1e1e2854046af1162ccc (just made the buffer bigger). A better solution would be to copy any unfinished message to the beginning of the next block buffer.