kafka-connect-transform-xml icon indicating copy to clipboard operation
kafka-connect-transform-xml copied to clipboard

org.xml.sax.SAXParseException; Exception thrown while processing: Exception thrown while building field / org.apache.kafka.connect.errors.ConnectException: Schema compiler could not bind schema

Open rmoff opened this issue 3 years ago • 4 comments

Using this XML and an XSD auto-generated, I get a Transform failure

Fails

        transforms = [xml]
        transforms.xml.negate = false
        transforms.xml.package = com.github.jcustenborder.kafka.connect.transform.xml.model
        transforms.xml.predicate =
        transforms.xml.schema.path = [https://rmoff.net/files/livecyclehireupdates.xsd]
        transforms.xml.type = class com.github.jcustenborder.kafka.connect.transform.xml.FromXml$Value
        transforms.xml.xjc.options.automatic.name.conflict.resolution.enabled = true
        transforms.xml.xjc.options.strict.check.enabled = true
        transforms.xml.xjc.options.verbose.enabled = true
        value.converter = null
 (org.apache.kafka.connect.runtime.ConnectorConfig$EnrichedConnectorConfig:354)
[2020-09-28 14:00:03,127] INFO [source-http-xml-02|task-0] FromXmlConfig values:
        package = com.github.jcustenborder.kafka.connect.transform.xml.model
        schema.path = [https://rmoff.net/files/livecyclehireupdates.xsd]
        xjc.options.automatic.name.conflict.resolution.enabled = true
        xjc.options.strict.check.enabled = true
        xjc.options.verbose.enabled = true
 (com.github.jcustenborder.kafka.connect.transform.xml.FromXmlConfig:354)
[2020-09-28 14:00:03,128] INFO [source-http-xml-02|task-0] compileContext() - Generating source for https://rmoff.net/files/livecyclehireupdates.xsd (com.github.jcustenborder.kafka.connect.t
ransform.xml.XSDCompiler:99)
[2020-09-28 14:00:03,593] ERROR [source-http-xml-02|task-0] Error (com.github.jcustenborder.kafka.connect.transform.xml.XSDCompiler:31)
org.xml.sax.SAXParseException; Exception thrown while processing: Exception thrown while building field 'station'. Stations
        at com.github.jcustenborder.kafka.connect.xml.KafkaConnectPlugin.run(KafkaConnectPlugin.java:486)
        at com.sun.tools.xjc.model.Model.generateCode(Model.java:292)
        at com.sun.tools.xjc.api.impl.s2j.SchemaCompilerImpl.bind(SchemaCompilerImpl.java:284)
        at com.sun.tools.xjc.api.impl.s2j.SchemaCompilerImpl.bind(SchemaCompilerImpl.java:95)
        at com.github.jcustenborder.kafka.connect.transform.xml.XSDCompiler.compileContext(XSDCompiler.java:106)
        at com.github.jcustenborder.kafka.connect.transform.xml.FromXml.configure(FromXml.java:130)
        at org.apache.kafka.connect.runtime.ConnectorConfig.transformations(ConnectorConfig.java:285)
        at org.apache.kafka.connect.runtime.Worker.buildWorkerTask(Worker.java:605)
        at org.apache.kafka.connect.runtime.Worker.startTask(Worker.java:555)
        at org.apache.kafka.connect.runtime.distributed.DistributedHerder.startTask(DistributedHerder.java:1251)
        at org.apache.kafka.connect.runtime.distributed.DistributedHerder.access$1700(DistributedHerder.java:127)
        at org.apache.kafka.connect.runtime.distributed.DistributedHerder$10.call(DistributedHerder.java:1266)
        at org.apache.kafka.connect.runtime.distributed.DistributedHerder$10.call(DistributedHerder.java:1262)
        at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base/java.lang.Thread.run(Thread.java:834)
Caused by: java.lang.IllegalStateException: Exception thrown while building field 'station'. Stations
        at com.github.jcustenborder.kafka.connect.xml.KafkaConnectPlugin.field(KafkaConnectPlugin.java:420)
        at com.github.jcustenborder.kafka.connect.xml.KafkaConnectPlugin.fields(KafkaConnectPlugin.java:452)
        at com.github.jcustenborder.kafka.connect.xml.KafkaConnectPlugin.fields(KafkaConnectPlugin.java:463)
        at com.github.jcustenborder.kafka.connect.xml.KafkaConnectPlugin.run(KafkaConnectPlugin.java:477)
        ... 16 more
Caused by: java.util.ConcurrentModificationException
        at java.base/java.util.HashMap.computeIfAbsent(HashMap.java:1134)
        at com.github.jcustenborder.kafka.connect.xml.KafkaConnectPlugin.type(KafkaConnectPlugin.java:293)
        at com.github.jcustenborder.kafka.connect.xml.KafkaConnectPlugin.field(KafkaConnectPlugin.java:401)
        ... 19 more
[2020-09-28 14:00:03,594] ERROR [source-http-xml-02|task-0] Failed to start task source-http-xml-02-0 (org.apache.kafka.connect.runtime.Worker:560)
org.apache.kafka.connect.errors.ConnectException: org.apache.kafka.connect.errors.ConnectException: Schema compiler could not bind schema.
        at org.apache.kafka.connect.runtime.ConnectorConfig.transformations(ConnectorConfig.java:296)
        at org.apache.kafka.connect.runtime.Worker.buildWorkerTask(Worker.java:605)
        at org.apache.kafka.connect.runtime.Worker.startTask(Worker.java:555)
        at org.apache.kafka.connect.runtime.distributed.DistributedHerder.startTask(DistributedHerder.java:1251)
        at org.apache.kafka.connect.runtime.distributed.DistributedHerder.access$1700(DistributedHerder.java:127)
        at org.apache.kafka.connect.runtime.distributed.DistributedHerder$10.call(DistributedHerder.java:1266)
        at org.apache.kafka.connect.runtime.distributed.DistributedHerder$10.call(DistributedHerder.java:1262)
        at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base/java.lang.Thread.run(Thread.java:834)
Caused by: org.apache.kafka.connect.errors.ConnectException: Schema compiler could not bind schema.
        at com.github.jcustenborder.kafka.connect.transform.xml.XSDCompiler.compileContext(XSDCompiler.java:109)
        at com.github.jcustenborder.kafka.connect.transform.xml.FromXml.configure(FromXml.java:130)
        at org.apache.kafka.connect.runtime.ConnectorConfig.transformations(ConnectorConfig.java:285)
        ... 10 more

If you install the HTTP source connector you can run this:

curl -i -X PUT -H "Accept:application/json" \
    -H  "Content-Type:application/json" http://localhost:8083/connectors/source-http-xml-02/config \
    -d ' {
        "connector.class": "com.github.castorm.kafka.connect.http.HttpSourceConnector",
        "tasks.max": "1",
        "http.request.url": "https://tfl.gov.uk/tfl/syndication/feeds/cycle-hire/livecyclehireupdates.xml",
        "http.timer.interval.millis": "600000",
        "kafka.topic": "livecyclehireupdates",
        "transforms": "xml",
        "transforms.xml.type": "com.github.jcustenborder.kafka.connect.transform.xml.FromXml$Value",
        "transforms.xml.schema.path": "https://rmoff.net/files/livecyclehireupdates.xsd",
        "transforms.xml.xjc.options.verbose.enabled": "true",
        "transforms.xml.xjc.options.automatic.name.conflict.resolution.enabled":"true"
    }'

Versions:

  • Confluent Platform 6.0
  • jcustenborder/kafka-connect-transform-xml:0.1.0.18

rmoff avatar Sep 28 '20 14:09 rmoff