camel-k
camel-k copied to clipboard
apache
The Yaml Parser diverges for Kamelets and Integration Resources
What happened?
While the following Yaml Syntax:
- constructors:
'0': '{{influx-url}}'
'1': '{{influx-username}}'
'2': '{{influx-password}}'
factoryBean: org.influxdb.InfluxDBFactory
factoryMethod: connect
name: influxClient
type: org.influxdb.InfluxDB
Works perfectly fine when applied in a Camel Integration resource directly. If it is used in a Kamelet, that is referenced in a Pipe, the Integration that is build and executed on the cluster runs into an unsupported field exception.
The unsupported Field Exception will appear with any of the fields factoryBean, factoryMethod and constructors, depending on the order they are used in.
Relevant log output
Caused by: Unsupported field: constructors
in file:/etc/camel/kamelets/kamelets-bundle-influx-action-binding-001/write-influx-action.kamelet.yaml, line 58, column 9:
"0": '{{influx-url}}'
^
at org.apache.camel.dsl.yaml.common.YamlDeserializerBase.handleUnknownProperty(YamlDeserializerBase.java:133)
at org.apache.camel.dsl.yaml.common.YamlDeserializerBase.setProperties(YamlDeserializerBase.java:127)
at org.apache.camel.dsl.yaml.common.YamlDeserializerBase.construct(YamlDeserializerBase.java:66)
at org.apache.camel.dsl.yaml.common.YamlDeserializationContext.lambda$resolve$0(YamlDeserializationContext.java:157)
at org.apache.camel.dsl.yaml.common.YamlDeserializerSupport.asType(YamlDeserializerSupport.java:344)
Camel K version
2.2.0 (Camel 4.2.0, Apache Camel K Runtime 3.6.0)
Do you mind providing a reproducer with basic sources? It would help troubleshooting. Also, mind that Camel K 2.2.0 is running by default with Camel Quarkus 3.2.0 runtime.
Sure, sorry for the late reply.
This is a basic correctly working integration:
---
# yamllint disable-line rule:line-length
# camel-k: language=yaml dependency="camel:influxdb" trait=camel.runtime-version="3.6.0"
- beans:
- type: org.influxdb.InfluxDB
name: influxClient
factoryBean: org.influxdb.InfluxDBFactory
factoryMethod: connect
constructors:
0: '{{influx-url}}'
1: '{{influx-username}}'
2: '{{influx-password}}'
- route:
id: influx-reader
description: >
This route reads data from influxdb for testing purposes.
from:
uri: timer://read?repeatCount=1
steps:
- setHeader:
id: setInfluxQuery
description: >
Set the header 'influxQuery' to the query that should be executed.
name: camelInfluxDB.query
simple: "SELECT * FROM assets"
- to:
id: retrieveAssetFromInfluxDb
description: >
Query the influxdb for assets.
uri: 'influxdb:influxClient'
parameters:
databaseName: "{{influx-database}}"
retentionPolicy: "{{influx-retentionpolicy}}"
operation: 'query'
- log:
id: logSuccess
loggingLevel: 'INFO'
message: '$simple{id}: Data has been retrieved from the influxdb.'
And now two basic Kamelets and Pipe that lead to the aforementioned error:
apiVersion: camel.apache.org/v1
kind: Kamelet
metadata:
name: influx-query-action
annotations:
trait.camel.apache.org/camel.runtime-version: 3.6.0
camel.apache.org/kamelet.support.level: "Stable"
camel.apache.org/catalog.version: "4.2.0"
camel.apache.org/kamelet.icon: "data:image/svg+xml;base64,PD94bWwgdmVyc2lvbj0iMS4wIiBlbmNvZGluZz0iaXNvLTg4NTktMSI/Pg0KPCEtLSBHZW5lcmF0b3I6IEFkb2JlIElsbHVzdHJhdG9yIDE2LjAuMCwgU1ZHIEV4cG9ydCBQbHVnLUluIC4gU1ZHIFZlcnNpb246IDYuMDAgQnVpbGQgMCkgIC0tPg0KPCFET0NUWVBFIHN2ZyBQVUJMSUMgIi0vL1czQy8vRFREIFNWRyAxLjEvL0VOIiAiaHR0cDovL3d3dy53My5vcmcvR3JhcGhpY3MvU1ZHLzEuMS9EVEQvc3ZnMTEuZHRkIj4NCjxzdmcgdmVyc2lvbj0iMS4xIiBpZD0iQ2FwYV8xIiB4bWxucz0iaHR0cDovL3d3dy53My5vcmcvMjAwMC9zdmciIHhtbG5zOnhsaW5rPSJodHRwOi8vd3d3LnczLm9yZy8xOTk5L3hsaW5rIiB4PSIwcHgiIHk9IjBweCINCgkgd2lkdGg9IjUxMnB4IiBoZWlnaHQ9IjUxMnB4IiB2aWV3Qm94PSIwIDAgNTEyIDUxMiIgc3R5bGU9ImVuYWJsZS1iYWNrZ3JvdW5kOm5ldyAwIDAgNTEyIDUxMjsiIHhtbDpzcGFjZT0icHJlc2VydmUiPg0KPGc+DQoJPHBhdGggZD0iTTQ0OCwwSDY0QzQ2LjMyOCwwLDMyLDE0LjMxMywzMiwzMnY0NDhjMCwxNy42ODgsMTQuMzI4LDMyLDMyLDMyaDM4NGMxNy42ODgsMCwzMi0xNC4zMTIsMzItMzJWMzINCgkJQzQ4MCwxNC4zMTMsNDY1LjY4OCwwLDQ0OCwweiBNNjQsNDgwVjEyOGg4MHY2NEg5NnYxNmg0OHY0OEg5NnYxNmg0OHY0OEg5NnYxNmg0OHY0OEg5NnYxNmg0OHY4MEg2NHogTTQ0OCw0ODBIMTYwdi04MGgyNTZ2LTE2DQoJCUgxNjB2LTQ4aDI1NnYtMTZIMTYwdi00OGgyNTZ2LTE2SDE2MHYtNDhoMjU2di0xNkgxNjB2LTY0aDI4OFY0ODB6Ii8+DQo8L2c+DQo8Zz4NCjwvZz4NCjxnPg0KPC9nPg0KPGc+DQo8L2c+DQo8Zz4NCjwvZz4NCjxnPg0KPC9nPg0KPGc+DQo8L2c+DQo8Zz4NCjwvZz4NCjxnPg0KPC9nPg0KPGc+DQo8L2c+DQo8Zz4NCjwvZz4NCjxnPg0KPC9nPg0KPGc+DQo8L2c+DQo8Zz4NCjwvZz4NCjxnPg0KPC9nPg0KPGc+DQo8L2c+DQo8L3N2Zz4NCg=="
camel.apache.org/provider: "Apache Software Foundation"
camel.apache.org/kamelet.group: „company“
labels:
camel.apache.org/kamelet.type: "action"
# The name of the application.
app.kubernetes.io/name: influx-action
# The component within the architecture.
app.kubernetes.io/component: test
# The name of the higher level application this application is part of.
app.kubernetes.io/part-of: test
spec:
dependencies:
- "mvn:org.apache.camel:camel-influxdb:4.2.0"
dataTypes:
out:
default: json
types:
json:
mediaType: application/json
definition:
title: "Influx-Query-Action"
description: |-
This action retrieves assets from an influxDB.
type: object
required:
- influx-url
- influx-username
- influx-password
- influx-database
- influx-retentionpolicy
properties:
influx-url:
title: Address to locate the influxdb
example: http://localhost:8086
type: string
influx-database:
title: The database to query from
example: primary
type: string
influx-username:
title: The username used to access the influx db
example: testuser
type: string
influx-password:
title: The token used to access the influx db
example: asdwqe23
type: string
influx-retentionpolicy:
title: The retention policy applied to the influx db
example: autogen
type: string
template:
beans:
- type: org.influxdb.InfluxDB
name: influxClient
factoryBean: org.influxdb.InfluxDBFactory
factoryMethod: connect
constructors:
0: '{{influx-url}}'
1: '{{influx-username}}'
2: '{{influx-password}}'
from:
uri: "kamelet:source"
steps:
- to:
id: queryInfluxDb
description: >
Query the influx database.
uri: influxdb:influxClient
parameters:
query: "$simple{body}"
databaseName: "{{influx-database}}"
retentionPolicy: "{{influx-retentionpolicy}}"
operation: QUERY
- to: "log:info"
apiVersion: camel.apache.org/v1
kind: Kamelet
metadata:
name: influx-generator-source
labels:
camel.apache.org/kamelet.type: "source"
annotations:
trait.camel.apache.org/logging.level: DEBUG
camel.apache.org/kamelet.support.level: "Stable"
camel.apache.org/kamelet.icon: "data:image/svg+xml;base64,PHN2ZyBpZD0iTGF5ZXJfMSIgZGF0YS1uYW1lPSJMYXllciAxIiB4bWxucz0iaHR0cDovL3d3dy53My5vcmcvMjAwMC9zdmciIHZpZXdCb3g9IjAgMCAyNTYgMzA4LjIzNDAxIj48dGl0bGU+YXdzLWtpbmVzaXM8L3RpdGxlPjxwYXRoIGQ9Ik0wLDE3Mi4wODdsMTI3Ljc1NCw1OC44MSwxMjcuNzUyLTU4LjgxLTEyNy43NTItNS4yOTNaIiB0cmFuc2Zvcm09InRyYW5zbGF0ZSgwIDAuMDAwMDUpIiBmaWxsPSIjZmNiZjkyIi8+PHBhdGggZD0iTTEyOC4xNDcsMCwuMDU5LDYzLjg4MXY5MC4xMzZIMTUzLjY0OFYxMi43NTFaIiB0cmFuc2Zvcm09InRyYW5zbGF0ZSgwIDAuMDAwMDUpIiBmaWxsPSIjOWQ1MDI1Ii8+PHBhdGggZD0iTS4wNTksMjE3LjU1OWwxMjguMTYyLDkwLjY3NUwyNTYsMjE3LjU1OSwxMjcuOTQ1LDE5OC45MjZaIiB0cmFuc2Zvcm09InRyYW5zbGF0ZSgwIDAuMDAwMDUpIiBmaWxsPSIjZmNiZjkyIi8+PHBhdGggZD0iTTEyOC4xNDYsMTU0LjAxN2g2Ny41NzdWNTcuODM2TDE3NS45OSw0OS45NDMsMTI4LjE0Niw2My44OThaIiB0cmFuc2Zvcm09InRyYW5zbGF0ZSgwIDAuMDAwMDUpIiBmaWxsPSIjOWQ1MDI1Ii8+PHBhdGggZD0iTTE3NS45OSwxNTQuMDE3aDUyLjIzM1Y5MS42MzJsLTE0Ljk0LTQuNDgxLTM3LjI5Myw2LjMzWiIgdHJhbnNmb3JtPSJ0cmFuc2xhdGUoMCAwLjAwMDA1KSIgZmlsbD0iIzlkNTAyNSIvPjxwYXRoIGQ9Ik0yMTMuMjgyLDgyLjI2djcxLjc1N2g0Mi4yMjRMMjU2LDgxLjk0MWwtMTIuODI2LTUuMTI0WiIgdHJhbnNmb3JtPSJ0cmFuc2xhdGUoMCAwLjAwMDA1KSIgZmlsbD0iIzlkNTAyNSIvPjxwYXRoIGQ9Ik0xMjguMTQ3LDBWMTU0LjAxN2gyNS41VjEyLjc1MVoiIHRyYW5zZm9ybT0idHJhbnNsYXRlKDAgMC4wMDAwNSkiIGZpbGw9IiNmNjg1MzQiLz48cGF0aCBkPSJNMTk1LjcyNCw1Ny44MzZsLTE5LjczMy03Ljg5NFYxNTQuMDE3aDE5LjczMloiIHRyYW5zZm9ybT0idHJhbnNsYXRlKDAgMC4wMDAwNSkiIGZpbGw9IiNmNjg1MzQiLz48cGF0aCBkPSJNMjI4LjIyNCw5MS42MzJsLTE0Ljk0MS00LjQ4djY2Ljg2NWgxNC45NFoiIHRyYW5zZm9ybT0idHJhbnNsYXRlKDAgMC4wMDAwNSkiIGZpbGw9IiNmNjg1MzQiLz48cGF0aCBkPSJNMjQzLjE3NCwxNTQuMDE3SDI1NlY4MS45NDFsLTEyLjgyNi01LjEyNFoiIHRyYW5zZm9ybT0idHJhbnNsYXRlKDAgMC4wMDAwNSkiIGZpbGw9IiNmNjg1MzQiLz48cGF0aCBkPSJNMTI3Ljc1NCwxODQuODYzdjQ2LjAzM2wxMjcuNzUyLTMxLjg0NFYxNzIuMDg3WiIgdHJhbnNmb3JtPSJ0cmFuc2xhdGUoMCAwLjAwMDA1KSIgZmlsbD0iI2Y2ODUzNCIvPjxwYXRoIGQ9Ik0xMjcuNzU0LDI2Mi43ODF2NDUuNDUzTDI1NiwyNDQuMTE0VjIxNy41NloiIHRyYW5zZm9ybT0idHJhbnNsYXRlKDAgMC4wMDAwNSkiIGZpbGw9IiNmNjg1MzQiLz48cGF0aCBkPSJNLjA1OSwyNDQuMzlsMTI3LjY5NSw2My44NDRWMjYyLjQ0OEwuMDU4LDIxNy41NThaIiB0cmFuc2Zvcm09InRyYW5zbGF0ZSgwIDAuMDAwMDUpIiBmaWxsPSIjOWQ1MDI1Ii8+PHBhdGggZD0iTTAsMTk5LjA1MWwxMjcuNzU0LDMxLjg0NVYxODQuODYyTDAsMTcyLjA4NloiIHRyYW5zZm9ybT0idHJhbnNsYXRlKDAgMC4wMDAwNSkiIGZpbGw9IiM5ZDUwMjUiLz48L3N2Zz4="
camel.apache.org/provider: "test Gmbh"
camel.apache.org/kamelet.group: "test"
spec:
dependencies:
- "camel:kamelet"
- "camel:core"
- "camel:log"
definition:
title: "Report-generator"
description: "Sends a test input to another kamelet"
required:
- query
properties:
query:
title: queries
description: The query map to split up into separate influx queries
type: string
example: '"SELECT * from Hosts"'
dataTypes:
out:
default: text
types:
text:
mediaType: text/plain
template:
from:
uri: timer:tick
steps:
- setBody:
constant: "{{query}}"
- to: "log:info"
apiVersion: camel.apache.org/v1
kind: Pipe
metadata:
name: influx-action-binding
spec:
source:
ref:
kind: Kamelet
apiVersion: camel.apache.org/v1
name: influx-generator-source
properties:
query: "from(bucket:\"company\") |> range(start: 0)"
steps:
- ref:
kind: Kamelet
apiVersion: camel.apache.org/v1
name: influx-query-action
properties:
influx-database: test
influx-url: "http://my-influxdb.test.svc.cluster.local:8086"
influx-username: "test"
influx-password: "test12345"
influx-retentionpolicy: "autogen"
sink:
uri: http://influx-service.test/queryresult
The integration for the influx-query-action will build without error and will be executed, but runtime errors due to unsupported fields will occur. Setting up an influxDB for reproduction should not be necessary, as the given runtime error occurs before the specified Bean is constructed.
I think it could be because in the yaml spec [1], the two objects are really different. One is type TemplatedRouteBeanDefinition and the other is type RouteTemplateBeanDefinition. I am not sure if the expected format and related implementation is different or it is some possible bug on Camel. @davsclaus @oscerd @lburgazzoli wdyt?
[1] https://github.com/apache/camel/blob/main/dsl/camel-yaml-dsl/camel-yaml-dsl/src/generated/resources/schema/camelYamlDsl.json
@lburgazzoli @davsclaus @oscerd bump. Please, confirm if this could be a bug on the core or something else we may need to fix in Camel K. Thanks in advance.
