kafka-connect-fs icon indicating copy to clipboard operation
kafka-connect-fs copied to clipboard

Source connector handling formatted JSON type records in different way

Open deepakrghv opened this issue 4 years ago • 1 comments

Hi

Can source connector handle formatted JSON records. I have file contains JSON records like this

{ "Employee":{ "header":{ "username":"abc", "reference":"1d0d5113-3d22-44c0-99f5-f6ff642f0233" }, "payload":{ "name":"hello", "sal":"" } } } { "Employee":{ "header":{ "username":"abc", "reference":"1d0d5113-3d22-44c0-99f5-f6ff642f0299" }, "payload":{ "name":"hello", "sal":"10000" } } }

It is considering each line as new record i.e. here 24 records and what I found out that it supports only minify JSON. Please let me know if my understanding is correct. ?

Note : I am using latest version i.e. v1.2.0 and it is processing correctly if my files contains JSON with below style:

{"Employee":{"header":{"username":"abc","reference":"1d0d5113-3d22-44c0-99f5-f6ff642f0233"},"payload":{"name":"hello","sal":""}}} {"Employee":{"header":{"username":"abc","reference":"1d0d5113-3d22-44c0-99f5-f6ff642f0299"},"payload":{"name":"hello","sal":"10000"}}}

Regards Deepak

deepakrghv avatar Feb 09 '21 08:02 deepakrghv

Hi! You can set the property file_reader.json.record_per_line to configure this. All config options are in the documentation here.

mmolimar avatar Feb 09 '21 15:02 mmolimar