Karate contains only shortcut fails in v1.4.1

[mapwake]

According to Karate documentation for contains short-cuts The following match arguments should pass only for the first expectation:

 * def cat = 
  """
  {
    name: 'Billie',
    kittens: [
      { id: 23, name: 'Bob', bla: [{ b: '1'}] },
      { id: 42, name: 'Wild' }
    ]
  }
  """
# this will pass and would be my expectation
* def expected = [{ id: 42, name: 'Wild' }, { id: 23, name: 'Bob', bla: [{ b: '1'}]}]
* match cat == { name: 'Billie', kittens: '#(^^expected)' }

# this also pass but i would assume, that it does not pass
* def expected = [{ id: 42, name: 'Wild' }, { id: 23, name: 'Bob', bla: { b: '1'} }]
* match cat == { name: 'Billie', kittens: '#(^^expected)' }

The second one should fail, because the expectation for the parameter "bla" is an object and not an array with an object in it.

I'm using Karate v1.4.1. The same scenario does work as i would assume with v1.2.0.

[ptrthomas]

probably related to changes here, needs investigation https://github.com/karatelabs/karate/issues/2170

[f-delahaye]

That's an interesting one ...

My understanding is that:

• a MatchOperation with type 'contains only' is created to validate kittens: '#(^^expected)'
• since kittens is a list, type is converted into 'contains only deep'
• 'contains only deep' is passed to the MatchOperation that validates each item in the list, since they are objects and the deep operator applies to nested lists or objects, per doc
• then it is passed to the MatchOperation that validates bla, since it is a list
• and match [{ b: '1'}] contains deep { b: '1'} passes.

There are probably a few steps I missed/misunderstood, so feel free to correct me. But hopefully, the general idea is the right one.

Basically, as soon as contains only is converted into contains only deep, there's no way back. If line 278 is commented so as to keep contains only, the test fails as OP expects. I suspect that if line 580 is changed so that contains only deep is reverted back to contains only, it would fail too. Obviously, neither change is possible as it would break backwards compatibility.

I can't think of any other way to fix the issue, but maybe someone else will have an idea...

[ptrthomas]

@f-delahaye yes this is a complicated one :|

[f-delahaye]

Actually ... there might be a solution.

Line 278 is key, IMO.

This method handles macros, which presumably are the same thing as shortcuts. So when ^^ (contains only) is specified, it gets converted to contains only deep. Is this by design? This does not sound like the expected behaviour.

@ptrthomas we could comment out line 278, so that nestedType remains contains only? The test then fails, as expected by the user. This would be a breaking change, but we could add a new shortcut for contains only deep, if needed.

With the line commented out, and using the standard syntax as opposed to the shortcuts: match cat contains only deep { name: 'Billie', kittens: [{ id: 42, name: 'Wild' }, { id: 23, name: 'Bob', bla: { b: '1'} }]} passes, but we did specify contains only deep so that sounds reasonable.