data-prepper icon indicating copy to clipboard operation
data-prepper copied to clipboard

Published 20 hours ago • verified publisher icon opensearch-project

[BUG] DynamoDB Source doesn't support parsing data with Control Characters

Open dinujoh opened this issue 4 months ago • 0 comments

Describe the bug The DynamoDB Source doesn't support parsing data with Control Characters

Object deserialization/serialization doesn't handle Control Characters:

https://github.com/opensearch-project/data-prepper/blob/main/data-prepper-plugins/dynamodb-source/src/main/java/org/opensearch/dataprepper/plugins/source/dynamodb/converter/StreamRecordConverter.java#L93

[pool-18-thread-104] ERROR org.opensearch.dataprepper.plugins.source.dynamodb.converter.StreamRecordConverter - Failed to parse and convert data from stream due to Illegal unquoted character ((CTRL-CHAR, code 4)): has to be escaped using backslash to be included in string value
 at [Source: REDACTED (`StreamReadFeature.INCLUDE_SOURCE_IN_LOCATION` disabled); line: 1, column: 720]

Expected behavior DynamoDb source should escape the control characters and process the data.

dinujoh avatar Oct 07 '24 22:10 dinujoh