Integration Extract JSON key value pair

[jaybrto]

Trying to use the Integration feature but variables come in empty. I can only bring in values from the header into the playbook but not if the data comes in from the Body. I tried changing the body type to string and sending in just text in the body and that doesn't work either. No other combination of content-type work. Not sure what else to try.

Here is the extraction data from API

[
    {
        "id": 12,
        "name": "aptype",
        "integration_id": 3,
        "value_source": "body",
        "body_data_type": "json",
        "key": "aptype",
        "variable": "auto_node_type"
    },
    {
        "id": 11,
        "name": "apval",
        "integration_id": 3,
        "value_source": "body",
        "body_data_type": "json",
        "key": "apval",
        "variable": "auto_node_val"
    }
]

Here is how I call the integration endpoint. Matcher on header x-req-type == apn

curl --location 'http://semaphore:3000/api/integrations/ny4lq0dnfccbxmyq' \
--header 'x-req-type: apn' \
--header 'Content-Type: application/json' \
--data '{
    "apval": "val 123",
    "aptype": "type 123"
}'

Here are the logs from the server

time="2024-04-08T03:49:27Z" level=info msg="Receiving Integration from: 192.168.1.252:60859"
time="2024-04-08T03:49:27Z" level=info msg="1 integrations found for alias ny4lq0dnfccbxmyq"
time="2024-04-08T03:49:27Z" level=info msg="Running integration 3"
time="2024-04-08T03:49:27Z" level=info msg="Task 87 added to queue"
time="2024-04-08T03:49:28Z" level=info msg="Set resource locker with TaskRunner 87"
time="2024-04-08T03:49:28Z" level=info msg="Task 87 removed from queue"
time="2024-04-08T03:49:33Z" level=info msg="Stopped running TaskRunner 87"
time="2024-04-08T03:49:33Z" level=info msg="Release resource locker with TaskRunner 87"

Here are the variables that the playbook gets "auto_node_type":"\\u003cnil\\u003e","auto_node_val":"\\u003cnil\\u003e"

These are from hostvars debug

"auto_node_type": "",
"auto_node_val": "",

Most of these tests were from Postman, but also straight curls. I have also tried removing the integration and re-creating it from scratch. Any help on this is appreciated. Thanks!

[lefortig]

Hi, i have the same problem.

The matcher is OK. The ExtractValue Json return "" when i choose "json" on Body Data type and '' (empty i guess) when i choose "string".

This feature seem to be cool!

[tboerger]

Not sure exactly but could be https://github.com/semaphoreui/semaphore/pull/1987 related to that?

[lefortig]

i thinks so

but it's not on the v2.9.75 ?

[tboerger]

This PR is not merged yet.

[reisenbauer]

This PR is not merged yet.

Hi Thomas, is there any eta for merging this PR?

[fiftin]

@reisenbauer merged.

[h4ckilles]

I'm running version v2.10.22 and the problem still exist. The PR is already in the code but it didn't fix the issue. Any other fix coming?

[reisenbauer]

@h4ckilles it is indeed already working, you have to be very carefull with the http-headers (content-type).

Running Version: 2.10.22-e44910d-1721658299

Configuration of the Integration:

Curl Request to trigger it: curl -H "Content-Type: application/json" https://xxxxx/api/integrations/9qlzybysn8xz1kug -d '{ "type": "xoxo", "xoxo": "1234" }'

Result:

[h4ckilles]

@reisenbauer Thanks for the hint. I've tried to trigger my integration from curl like you've suggested but still no extracted values. Here is the curl command I use:

curl -H "Content-Type: application/json" -H 'Authorization: ******************' https://xxxxxxxx/api/integrations/drkn294l6ehy02jp -d '{ "os_type": "X", "requester": "deploymate" }'

Here is the integration:

Here is the called ansible playbook:

- hosts: localhost
  gather_facts: true
  connection: local
  vars_files:
    - ./credentials.yml
  tasks:
    - name : debug env var
      ansible.builtin.debug:
        msg: "{{ lookup('ansible.builtin.env', 'OS_TYPE') }}"
    - name : Linux
      include_role:
        name: linux
      when: lookup('ansible.builtin.env', 'OS_TYPE') == "L"
    - name : Windows
      include_role: 
        name: windows
      when: lookup('ansible.builtin.env', 'OS_TYPE') == "W"

The playbook did run on curl execution but no environment variable:

[reisenbauer]

Hey @h4ckilles, probably found your issue:

- hosts: localhost
  gather_facts: true
  connection: local
  vars_files:
    - ./credentials.yml
  tasks:
    - name : debug env var
      ansible.builtin.debug:
        msg: "{{ lookup('ansible.builtin.env', '{{OS_TYPE}}') }}"
    - name : Linux
      include_role:
        name: linux
      when: lookup('ansible.builtin.env', '{{OS_TYPE}}') == "L"
    - name : Windows
      include_role: 
        name: windows
      when: lookup('ansible.builtin.env', '{{OS_TYPE}}') == "W"

try it this way :)

[h4ckilles]

@reisenbauer Thanks a lot it was indeed the playbook causing the problem. This is the correct version:

---
- hosts: localhost
  gather_facts: true
  connection: local
  vars_files:
    - ./credentials.yml
  tasks:
    - name : debug OS_TYPE
      ansible.builtin.debug:
        msg: "{{ OS_TYPE }}"
    - name : Linux
      include_role:
        name: linux
      when: OS_TYPE == "L"
    - name : Windows
      include_role: 
        name: windows
      when: OS_TYPE == "W"

[reisenbauer]

@h4ckilles glad you got it working :-)