kafka-connect-file-pulse icon indicating copy to clipboard operation
kafka-connect-file-pulse copied to clipboard

Published 20 hours ago • verified publisher icon streamthoughts

Wrong mapping of field names to their index for DelimitedRowFilter

Open loquisgon opened this issue 1 year ago • 5 comments

When using a DelimitedRowFilter and the input schema has fields that are not in alphabetical order then the Schema will be configured with the wrong fieldName to indexPositionForField mapping. That is, imagine your schema is XField:Long, AField:Float, BField:String then the mapping that will be produced is AField:Long, BField:Float, XField:String which is wrong.

To Reproduce See unit test in https://github.com/streamthoughts/kafka-connect-file-pulse/pull/439.

loquisgon avatar Apr 04 '23 23:04 loquisgon

This was blocking my use case, I built a new connector based on this patch and my use case now is unblocked. It'd be great to put this into a release soon (the PR referenced above).

loquisgon avatar Apr 05 '23 18:04 loquisgon

Hi @loquisgon, thanks for your PR. I've made a review on it. Could you please check the requested changes so that I can merge your fix ?

Thank you

fhussonnois avatar Apr 06 '23 10:04 fhussonnois

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

github-actions[bot] avatar Jul 06 '23 02:07 github-actions[bot]

The above fix, seems to be duplicating all fields on the list while re-adding them.

We have raised an improvement based on the suggested implementation on https://github.com/streamthoughts/kafka-connect-file-pulse/pull/534 FYI @fhussonnois @loquisgon

theworkshopcom avatar Sep 13 '23 09:09 theworkshopcom

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

github-actions[bot] avatar Feb 28 '24 01:02 github-actions[bot]