hurl-6.1.1 return an empty list instead nothing on jsonpath selectors

[lu-zero]

Example:

13 | jsonpath "$.array_that_exists[?(@.field_that_exists == 'Not exists')].notExists" not exists
   |   actual:   list <[]>
   |   expected: not something
14 | jsonpath "$.array_that_exists[?(@.field_that_exists == 'Exists')].notExists" not exists
   |   actual:   list <[]>
   |   expected: not something

hurl-6.0.0 works as expected.

[fabricereix]

Thanks @lu-zero for reporting the bug. Could you please give us the JSON response so that we can reproduce it?

[ekynoxe]

I have the exact same issue. Here's a simple repro:

json data returned by a small node server:

{
    arr: [
      {id: '123', enabled: true},
      {id: '456'},
    ]
  }

hurl test:

GET http://localhost:5005
Content-Type: application/json
Accept: application/json
HTTP 200
[Asserts]
jsonpath "$.arr[?(@.id=='123')]" exists
jsonpath "$.arr[?(@.id=='123')].enabled" exists
jsonpath "$.arr[?(@.id=='456')]" exists
jsonpath "$.arr[?(@.id=='456')].enabled" not exists

output with hurl 6.0.0:

* Request:
* GET http://localhost:5005
* Content-Type: application/json
* Accept: application/json
*
* Request can be run with the following curl command:
* curl --header 'Content-Type: application/json' --header 'Accept: application/json' --timeout 30 'http://localhost:5005'
*
> GET / HTTP/1.1
> Host: localhost:5005
> Content-Type: application/json
> Accept: application/json
>
* Response: (received 50 bytes in 0 ms)
*
< HTTP/1.1 200 OK
< Content-Type: application/json
< Date: Mon, 14 Apr 2025 10:08:55 GMT
< Connection: keep-alive
< Keep-Alive: timeout=5
< Transfer-Encoding: chunked
<
*
tests/exists.hurl: Success (1 request(s) in 1 ms)
* Writing HTML report to /var/folders/3l/2gnvxyqx0j161b13l7lzwvfr0000gp/T/tmp.veTnOOyhIK
--------------------------------------------------------------------------------
Executed files:    1
Executed requests: 1 (1000.0/s)
Succeeded files:   1 (100.0%)
Failed files:      0 (0.0%)
Duration:          1 ms

output with hurl 6.1.1:

* Request:
* GET http://localhost:5005
* Content-Type: application/json
* Accept: application/json
*
* Request can be run with the following curl command:
* curl --header 'Content-Type: application/json' --header 'Accept: application/json' --timeout 30 'http://localhost:5005'
*
> GET / HTTP/1.1
> Host: localhost:5005
> Content-Type: application/json
> Accept: application/json
>
* Response: (received 50 bytes in 8 ms)
*
< HTTP/1.1 200 OK
< Content-Type: application/json
< Date: Mon, 14 Apr 2025 10:09:31 GMT
< Connection: keep-alive
< Keep-Alive: timeout=5
< Transfer-Encoding: chunked
<
*
error: Assert failure
  --> tests/exists.hurl:9:0
   |
   | GET http://localhost:5005
   | ...
 9 | jsonpath "$.arr[?(@.id=='456')].enabled" not exists
   |   actual:   list <[]>
   |   expected: not something
   |

tests/exists.hurl: Failure (1 request(s) in 9 ms)
* Writing HTML report to /var/folders/3l/2gnvxyqx0j161b13l7lzwvfr0000gp/T/tmp.jZLZntBIFD
--------------------------------------------------------------------------------
Executed files:    1
Executed requests: 1 (100.0/s)
Succeeded files:   0 (0.0%)
Failed files:      1 (100.0%)
Duration:          10 ms

Error:

9 | jsonpath "$.arr[?(@.id=='456')].enabled" not exists
   |   actual:   list <[]>
   |   expected: not something

[fabricereix]

I confirm that the behavior is different between 6.1.1 and 6.0.0. But the correct one is in 6.1.1 (fix with the issue #3859) When selecting a non-existing field, jsonpath should return an empty list.

Here is the example above with jsonpath.com http://jsonpath.com/#eJxtjcEKwjAQRH8lDEIVSkFrBQNFP8F700Ns0hqoqabJQUL-3S316F52d_bNTsSsvTd2AI_QdjBWg8P13bkqK-SYgn8Ff5P-MYP3cpx1yvEO2n0I2xTSueayvRZG1XV2rE7Zri20lfdRKzKrqQtPbT2hUVhGJUAOAc6adWcsChi1KAL7QymQ0_D7sKjeBUr8A1OYQFoP7dIS0heo6DzA

[Thomasdezeeuw]

@fabricereix am I correct in saying it's a breaking change that actually fixed a bug in 6.0.0?

[ekynoxe]

Thanks @fabricereix that makes sense.

However, how would you check for the absence of a property like in my example above?

jsonpath "$.arr[?(@.id=='456')].enabled" == [] is invalid (invalid predicate value) jsonpath "$.arr[?(@.id=='456')].enabled" == "[]" is incorrect (actual: list <[]> expected: string <[]>) jsonpath "$.arr[?(@.id=='456')].enabled" isEmpty also fails with actual: list <[]> expected: not something

As per @Thomasdezeeuw comment above (and I suspect @lu-zero ), we see these issues after a basic install of hurl 6, where our tests were written for hurl 5, pass with hurl 6.0.0, and now fail after the minor update to hurl 6.1.1.

So whilst I agree there was a bug with the handling of json path return value, this feels like a breaking change.

[fabricereix]

@ekynoxe, you could use

jsonpath "$.arr[?(@.id=='456')].enabled" count == 0

@Thomasdezeeuw, I'm not sure about the breaking change. The syntax has not been changed, the Hurl file can still be parsed with hurlfmt.

[ekynoxe]

@fabricereix oh, yes, good shout.

I think the issue I'm having is that I am thinking in a different language, and need to re-frame the way I look at assertions.

This suggestion makes sense. Thank you!

[Thomasdezeeuw]

@Thomasdezeeuw, I'm not sure about the breaking change. The syntax has not been changed, the Hurl file can still be parsed with hurlfmt.

If it worked with Hurl v6.0.0 and doesn't with 6.1.1 and the file wasn't changed... then that's a breaking change to me.

[ekynoxe]

I'll also add that now we have an issue where some new devs in our team have hurl 6.1.1 installed (standard brew install hurl on mac osx), and our pipelines still use 6.0.0, and the changes suggested above fail on hurl 6.0.0:

jsonpath "$.arr[?(@.id=='456')].enabled" count == 0
    |                                                 ^^^^^ missing value to apply filter
    |

So this is breaking quite a few things now as it's not backwards compatible it seems...

[lepapareil]

I'll also add that now we have an issue where some new devs in our team have hurl 6.1.1 installed (standard brew install hurl on mac osx), and our pipelines still use 6.0.0, and the changes suggested above fail on hurl 6.0.0:

jsonpath "$.arr[?(@.id=='456')].enabled" count == 0
    |                                                 ^^^^^ missing value to apply filter
    |

So this is breaking quite a few things now as it's not backwards compatible it seems...

Hi @ekynoxe, keeping local and pipeline cli versions out of sync is risky, if devs use new features from 6.1.x, then 6.0.0 pipeline will also fail 😢.

[ekynoxe]

I'll also add that now we have an issue where some new devs in our team have hurl 6.1.1 installed (standard brew install hurl on mac osx), and our pipelines still use 6.0.0, and the changes suggested above fail on hurl 6.0.0:

jsonpath "$.arr[?(@.id=='456')].enabled" count == 0
    |                                                 ^^^^^ missing value to apply filter
    |

So this is breaking quite a few things now as it's not backwards compatible it seems...

Hi @ekynoxe, keeping local and pipeline cli versions out of sync is risky, if devs use new features from 6.1.x, then 6.0.0 pipeline will also fail 😢.

I don't disagree in principle, but that's an irrelevant point to the issue here.

Businesses with large infrastructure and slow processes can't "simply upgrade tools" like that. I'd love to, but it's not how it really woks.

Right now, we're stuck in between these two bad situations because 6.1.1 isn't backwards compatible with 6.0.0, despite being only a minor version away, which, as @Thomasdezeeuw said, is a breaking change.