venom icon indicating copy to clipboard operation
venom copied to clipboard

1.2.0 bug when accessing `result.bodyjson.__type__` and `result.bodyjson.__len__`

Open eyevanovich opened this issue 2 months ago • 1 comments

Description

Summary

In venom v1.2.0, the result.bodyjson.__type__ and result.bodyjson.__len__ assertions fail with <nil> when the HTTP response returns a top-level JSON array. This appears to be a regression from v1.1.0 and is not documented in the v1.2.0 release notes.

Environment

  • Venom version: v1.2.0
  • Go version: 1.25.1
  • OS: macOS (darwin/arm64)

Expected Behavior

Based on the HTTP executor documentation, the following assertions should work for top-level JSON arrays:

assertions:
  - result.bodyjson.__type__ ShouldEqual Array
  - result.bodyjson.__len__ ShouldEqual 2

Actual Behavior

When the HTTP response body is a top-level JSON array like [{"address":"[email protected]","id":"123"}], the assertions fail with:

Assertion "result.bodyjson.__type__ ShouldEqual Array" failed. expected: Array  got: <nil>
Assertion "result.bodyjson.__len__ ShouldEqual 2" failed. expected: 2  got: <nil>

Reproduction Steps

  1. Start a simple HTTP server that returns a JSON array:
app.Get("/emails", func(c *fiber.Ctx) error {
    return c.JSON([]*Email{
        {Address: "[email protected]", ID: "id1"},
        {Address: "[email protected]", ID: "id2"},
    })
})
  1. Create a venom test:
name: "Test Top-Level Array"
testcases:
- name: TestArrayTypeAssertion
  steps:
  - type: http
    method: GET
    url: http://localhost:3000/emails
    assertions:
    - result.statuscode ShouldEqual 200
    - result.bodyjson.__type__ ShouldEqual Array  # FAILS
    - result.bodyjson.__len__ ShouldEqual 2       # FAILS
  1. Run: venom run test.yml

Result: Both __type__ and __len__ assertions fail with <nil>.

What Still Works

✅ Accessing array elements directly:

assertions:
  - result.bodyjson.bodyjson0.address ShouldEqual [email protected]

✅ Using string-based assertions:

assertions:
  - result.body ShouldContainSubstring [email protected]

✅ Top-level arrays work correctly in v1.1.0 (not confirmed but suspected based on documentation)

Additional Context

  • This issue specifically affects top-level JSON arrays
  • Arrays nested within objects (like {"emails": [...]}) appear to have similar issues
  • The response body is valid JSON and can be parsed correctly
  • The result.body contains the correct JSON string
  • This was discovered while upgrading from v1.1.0 to v1.2.0

Workarounds

  1. Access array elements directly instead of checking type/length
  2. Use string-based assertions on result.body
  3. Modify API to return wrapped arrays: {"data": [...]}

Questions

  1. Is this an intentional breaking change? If so, it should be documented in release notes
  2. Should __type__ and __len__ work for top-level arrays?
  3. Is there an alternative way to assert on array types and lengths in v1.2.0?

eyevanovich avatar Oct 03 '25 16:10 eyevanovich

Thank you for detailed issue, we'll have a look

yesnault avatar Oct 20 '25 07:10 yesnault

Try using __Type__ and __Len__ instead I think this is due to the fact that 1.2 changed to be case sensitive by default so the old syntax doesn't work anymore

You can see in the tests: https://github.com/ovh/venom/blob/db5709b2c50adc01905136d1c27ed69edd5c1b05/tests/http.yml#L65-L67

But the doc are indeed outdated with the old syntax

lowlighter avatar Dec 23 '25 15:12 lowlighter