authorjapps
Problem working with Kafka messages with Record keys in Zerocode version 1.3.28 and later
Hi.
I have been executing kafka tests on older versions of kafka without having any issues but recently tried upgrading to more recent version - but have hit a problem when kafka messages with a key (containing dashes) is present.
ava.lang.RuntimeException: ZeroCode Step execution failed. Details:java.lang.RuntimeException: com.fasterxml.jackson.core.JsonParseException: Unexpected character ('a' (code 97)): Expected space separating root-level values at [Source: (String)"326e9aff-9767-46a9-a5e9-22c339389da7"; line: 1, column: 7]
The record has uses a request id as the key
Record Key - 326e9aff-9767-46a9-a5e9-22c339389da7 , Record value - {"request_id": "326e9aff-9767-46a9-a5e9-22c339389da7" .... }
The messages are processed fine with 1.3.27 - so something must have changed in the 1.3.28 version which is leading to this exception.
Could anyone advise what might be the problem?
I found something online which possibly might be related (similar error returned) https://discuss.aerospike.com/t/root-level-value-error-when-using-aerospike-inbound-kafka-connector-with-librdkafka/6903
Hi @bfarrell-ibm Is this issue is still active? I am interested in working on this issue
@BharatBaweja
Yes - the problem is still there as far as I am aware. I have not retested in recent days - but assume nothing has changed since i reported the issue.
Hi @bfarrell-ibm
I'm a third-year undergraduate Computer Science major and I'd like to help out with this issue or a similar one. Do you think that this is probably doable for me? I haven't looked at the code yet but I will soon.
Hi @bfarrell-ibm Is this issue is still active? I am interested in working on this issue
Hello, if you identified the problem and the fix, can you raise a PR? or If you are busy, can you advise about the problem to @JPLahoda and about the fix(if already found) ?
@bfarrell-ibm , can you attach the request/response JSON scenario and the log output to the ticket description please?
Hi - sorry for the delayed response - i'm not working on this issue at present and needed to dig up some info.
It appears to still be an issue. My test runs on up to zerocode version 1.3.27 - but after that the kafka records are not processed.
2023-11-17 12:13:38,667 [main] ERROR org.jsmart.zerocode.core.kafka.client.BasicKafkaClient - Exception during operation:CONSUME, topicName:AnsibleLifecycleDriver_lifecycle_request_queue, error:Unrecognized token 'f36eeb7f': was expecting (JSON String, Number, Array, Object or token 'null', 'true' or 'false')
Where the record is as follows
Record Key - f36eeb7f-2ace-43b4-9467-fd3bbde08ea3 , Record value - {"request_id": "f36eeb7f-2ace-43b4-9467-fd3bbde08ea3", "lifecycle_name": "Stop", "driver_files": "UEsDBBQACAgIAAAAIQAAAAAAAAAAAAAAAAAgABEAY29uZmlnL2hvc3RfdmFycy9yZW1vdGUtaG9zdC55bWxVVA0ABwAAAAAAAAAAAAAAANPV1eVKzCvOTMpJjc/ILy6xUlCqrlYoKMovSC0qyUwt1itKzc0vgcgp1NYqwRWXFqcW4VIMkkNRXFycEV+QWFyMSwNIDqyBCwBQSwcI34+WwUkAAACQAAAAUEsDBBQACAgIAAAAIQAAAAAAAAAAAAAAAAAQABEAY29uZmlnL2ludmVudG9yeVVUDQAHAAAAAAAAAAAAAAAAiy5Kzc0vSY3lgtC6GfnFJVwAUEsHCB/HAbISAAAAFQAAAFBLAwQUAAgICAAAACEAAAAAAAAAAAAAAAAAEgARAHNjcmlwdHMvQWRvcHQueWFtbFVUDQAHAAAAAAAAAAAAAAAAXYwxDoAwDAN3XmGx9wPZWPgGCiW0Ei1BpPyflJHtbJ8ccHIVwrTp1QYgqzUjFI1cOnu1StSunOohcctyLzvHrs1cTLxtbIeRQ/jdAZusT6IPgWqJMH7j+AJQSwcIKwqrZFkAAAB6AAAAUEsDBBQACAgIAAAAIQAAAAAAAAAAAAAAAAAWABEAc2NyaXB0cy9Db25maWd1cmUueWFtbFVUDQAHAAAAAAAAAAAAAAAAbYwxDoAwDAN3XmGx9wNZkfgGCiUURGmkpv0/LQMT29k62yHxLYRJ036GmmUADrVihKieY+dWreK1a0lbCFwOycvOvmszR+urwnYZNXA/l8Amaw30InBbIIyfMD5QSwcIYtQ3ZFwAAACGAAAAUEsDBBQACAgIAAAAIQAAAAAAAAAAAAAAAAATABEAc2NyaXB0cy9DcmVhdGUueWFtbFVUDQAHAAAAAAAAAAAAAAAAXYw5DoAgEEV7T/FDzwWmNfEaZsARElkSBu8vaGf3/mpROAthbcJdFiBW7UpI1XOaPCwnvs5OqUME7lHafrKftY2TzlVnvZQG2P8fcIi7A70IZA0E86XmAVBLBwjknRJOWAAAAH0AAABQSwMEFAAICAgAAAAhAAAAAAAAAAAAAAAAABMAEQBzY3JpcHRzL0RlbGV0ZS55YW1sVVQNAAcAAAAAAAAAAAAAAABdjLkNgDAMRXum+KLPAq4RayAnmEQih4TN/iTQ0b1/OlQuQlgki8kEpKamhNwC58Hd8hLa6NTWRWRLcm0Hh1FbOetYGeup1MH9/4Bd/B3pRaBoJMxfOj9QSwcIpQ/M/1cAAAB9AAAAUEsDBBQACAgIAAAAIQAAAAAAAAAAAAAAAAAUABEAc2NyaXB0cy9JbnN0YWxsLnlhbWxVVA0ABwAAAAAAAAAAAAAAAGWMwQ2AIBAE/1ax8U8DV4CJVZgDT0g8IPGwf8H48ze7O1mHwlkIa7HGqhOQqjUjaA2sg3vlJdQhldpD5Jbk2g4OQ1tYTXrb2E6jDu53COzi70gvAtkiYf7m+QFQSwcID3II1VsAAACAAAAAUEsDBBQACAgIAAAAIQAAAAAAAAAAAAAAAAASABEAc2NyaXB0cy9TdGFydC55YW1sVVQNAAcAAAAAAAAAAAAAAABdjMENgDAMA/9MYfHvAhmABRgApSW0EoVITdiflie/s31ywM2XEFbn5hNQ1NwIVRPXwb2KknQot/aQ2Yu07eA0tIWrSW+d7TTqEH53wC7xyfQhcFkmzN84v1BLBwjWD7asWQAAAHoAAABQSwMEFAAICAgAAAAhAAAAAAAAAAAAAAAAABEAEQBzY3JpcHRzL1N0b3AueWFtbFVUDQAHAAAAAAAAAAAAAAAAVYzBDYAgEAT/VrHxTwNXgA1YgDnwhETgjIf9C/78ze5O1qFyEcLa9JqApNaMkDVwHtwrL0GHUbWHyC3JvR0chrZwNultYzuNOrj/G7CLfyJ9CBSLhHls8wtQSwcI9F9rrFkAAAB3AAAAUEsBAhQAFAAICAgAAAAhAN+PlsFJAAAAkAAAACAACQAAAAAAAAAAAAAAAAAAAGNvbmZpZy9ob3N0X3ZhcnMvcmVtb3RlLWhvc3QueW1sVVQFAAcAAAAAUEsBAhQAFAAICAgAAAAhAB/HAbISAAAAFQAAABAACQAAAAAAAAAAAAAAqAAAAGNvbmZpZy9pbnZlbnRvcnlVVAUABwAAAABQSwECFAAUAAgICAAAACEAKwqrZFkAAAB6AAAAEgAJAAAAAAAAAAAAAAAJAQAAc2NyaXB0cy9BZG9wdC55YW1sVVQFAAcAAAAAUEsBAhQAFAAICAgAAAAhAGLUN2RcAAAAhgAAABYACQAAAAAAAAAAAAAAswEAAHNjcmlwdHMvQ29uZmlndXJlLnlhbWxVVAUABwAAAABQSwECFAAUAAgICAAAACEA5J0STlgAAAB9AAAAEwAJAAAAAAAAAAAAAABkAgAAc2NyaXB0cy9DcmVhdGUueWFtbFVUBQAHAAAAAFBLAQIUABQACAgIAAAAIQClD8z/VwAAAH0AAAATAAkAAAAAAAAAAAAAAA4DAABzY3JpcHRzL0RlbGV0ZS55YW1sVVQFAAcAAAAAUEsBAhQAFAAICAgAAAAhAA9yCNVbAAAAgAAAABQACQAAAAAAAAAAAAAAtwMAAHNjcmlwdHMvSW5zdGFsbC55YW1sVVQFAAcAAAAAUEsBAhQAFAAICAgAAAAhANYPtqxZAAAAegAAABIACQAAAAAAAAAAAAAAZQQAAHNjcmlwdHMvU3RhcnQueWFtbFVUBQAHAAAAAFBLAQIUABQACAgIAAAAIQD0X2usWQAAAHcAAAARAAkAAAAAAAAAAAAAAA8FAABzY3JpcHRzL1N0b3AueWFtbFVUBQAHAAAAAFBLBQYAAAAACQAJAKQCAAC4BQAAAAA=", "system_properties": {"resourceManagerId": {"type": "string", "value": "brent"}, "resourceId": {"type": "string", "value": "406ad03b-400a-42e1-9bb2-ea26fa6ab5bc"}, "metricKey": {"type": "string", "value": "9e4ae98a-a45f-44dc-ad77-003ea90f3792"}, "requestId": {"type": "string", "value": "fcf42564-3894-4d13-a18e-8e2e19fc24f5"}, "resourceName": {"type": "string", "value": "CreateSimpleDummy1__dummy__1"}, "deploymentLocation": {"type": "string", "value": "RHOSP"}, "resourceType": {"type": "string", "value": "resource::dummy-vnfc-dummy::1.0"}}, "resource_properties": {}, "request_properties": {}, "associated_topology": {}, "deployment_location": {"objectGroupId": "bb111383-be83-486d-870f-fccde62ffbe5", "resourceManager": "brent", "name": "RHOSP", "type": "Openstack", "properties": {}}, "tenant_id": "ac0b2851-22e0-45d5-ba2a-4f4b8765f910", "logging_context": {"tracectx.tenantid": "ac0b2851-22e0-45d5-ba2a-4f4b8765f910", "tracectx.processid": "a1cb8b8b-0e58-40a1-aa40-3e66f3a0ab85", "tracectx.transactionid": "6db673e7-e6e6-40db-b026-6a186e979186", "tracectx.taskid": "6"}}, Record partition - 0, Record offset - 10, Headers - RecordHeaders(headers = [], isReadOnly = false)
I tried on the latest ZC version - 1.3.35 - and it is still the same Zerocodeversion1.3.23.log Zerocodeversion1.3.28.log Zerocodeversion1.3.35.log
Okay no worries. I am checking the issues list to see which of those I can work on :)