[Feature Request]: Support Jackson 2.15

[Abacn]

What would you like to happen?

Known breaking change

• https://github.com/FasterXML/jackson-core/issues/863

causing

com.fasterxml.jackson.core.exc.StreamConstraintsException: String length (5046272) exceeds the maximum length
(5000000)
at com.fasterxml.jackson.core.StreamReadConstraints.validateStringLength(StreamReadConstraints.java:290)
at com.fasterxml.jackson.core.util.ReadConstrainedTextBuffer.validateStringLength(ReadConstrainedTextBuffer.java:27)
at com.fasterxml.jackson.core.util.TextBuffer.finishCurrentSegment(TextBuffer.java:931)

Issue Priority

Priority: 2 (default / most feature requests should be filed as P2)

Issue Components

• [ ] Component: Python SDK
• [X] Component: Java SDK
• [ ] Component: Go SDK
• [ ] Component: Typescript SDK
• [ ] Component: IO connector
• [ ] Component: Beam examples
• [ ] Component: Beam playground
• [ ] Component: Beam katas
• [ ] Component: Website
• [ ] Component: Spark Runner
• [ ] Component: Flink Runner
• [ ] Component: Samza Runner
• [ ] Component: Twister2 Runner
• [ ] Component: Hazelcast Jet Runner
• [ ] Component: Google Cloud Dataflow Runner

[masahitojp]

In jackson-databind, I have found that the following changes are made for the same event. https://github.com/search?q=repo%3AFasterXML%2Fjackson-databind+maxStringLength&type=code

In apache/beam, com.fasterxml.jackson.core.JsonFactory is used in the following.

https://github.com/search?q=repo%3Aapache%2Fbeam+com.fasterxml.jackson.core.JsonFactory&type=code

[masahitojp]

In v2.15.1, the default string length was changed from 5M to 20M. If you use jackson version 2.15 or later, it would be better to upgrade to 2.15.1. :-) https://github.com/FasterXML/jackson-core/pull/1020

[Abacn]

@masahitojp Thanks for the information. iiuc does this means the breaking change of 2.15.0 is (partly) fixed in 2.15.1 ?

I also checked tha stacktrace the jackson parser is handled by google-api-client-jackson2 dependency (not directly by Beam). It may require the update from this dependency to support Jackson 2.15.

[Abacn]

As of July 2023, the latest version of google-http-client-jackson2 still uses jackson 2.14.2: https://mvnrepository.com/artifact/com.google.http-client/google-http-client-jackson2/1.43.3

[chamikaramj]

Noting that Jackson 2.14.1 has a conflict with SnakeYaml 2.0: https://github.com/spring-projects/spring-boot/issues/34405

SnakeYaml 1.x has a known vulnerability (https://mvnrepository.com/artifact/org.yaml/snakeyaml/1.33), so we need to upgrade Jackson to properly resolve this conflict.

Also, Beam Java core recently added a hard dependency to SnakeYaml 2.0: https://github.com/apache/beam/blob/611676d108b26ee378a2b0c128c855017d162772/sdks/java/core/build.gradle#L101

[chamikaramj]

cc: @ahmedabu98 since we added the SnakeYaml 2.0 dependency in https://github.com/apache/beam/pull/30808.

[chamikaramj]

@Abacn @kennknowles should we consider making this a release blocker ?

[kennknowles]

Yes, that could make sense if we currently have a dep conflict that will block important uses. What is the resolution proposal?

[chamikaramj]

So we can either downgrade SnakeYAML to 1.33 or upgrade Jackson to 2.15. SnakeYAML 1.33 has a a known vulnerability so upgrading Jackson is probably preferable. I'm not sure if we'll run into other conflicts due to other dependencies transitively depending on Jackson 2.14 though.

Assigning to @Abacn to comment on the feasibility / best approach on resolving this conflict.

[chamikaramj]

cc: @liferoad

[Abacn]

There is indeed transient dependency on snakeyaml 1.x, for cassandra-all 3.x used in hadoop-format test. see CASSANDRA-19653

[Abacn]

For those who upgraded to Jackson 2.15 and see the problem in the Issue description, https://github.com/spring-projects/spring-boot/issues/34709#issuecomment-1957121251 may be solution (going to get reproduce on my own)

[flop-signing]

I still can't find any proper way and solve the same issue that I had been faced in recent days. The file that I processed that size was 15 MB. But got the same error. java.lang.RuntimeException: com.fasterxml.jackson.core.exc.StreamConstraintsException: String value length (20054016) exceeds the maximum allowed (20000000, from StreamReadConstraints.getMaxStringLength())