jmespath.js icon indicating copy to clipboard operation
jmespath.js copied to clipboard

filtering flattened projection does not seem to follow projection rules

Open daniefer opened this issue 5 years ago • 4 comments

Scenario

If we have an object like so:

{
    "persons": [
        {
            "name": "bob",
            "addresses": [
                {
                    "current": true,
                    "address": "132 Main Ave"
                },
                {
                    "current": false,
                    "address": "99 1st St"
                }
            ]
        },
        {
            "name": "linda",
            "addresses": [
                {
                    "current": true,
                    "address": "132 Main Ave"
                },
                {
                    "current": false,
                    "address": "99 1st St"
                }
            ]
        }
    ]
}

and the query:

persons[?name=='bob'].addresses[]

I get the result

[
  {
    "current": true,
    "address": "132 Main Ave"
  },
  {
    "current": false,
    "address": "99 1st St"
  }
]

This seems to line up just fine with the documentation of what a "projection" is in JMES. It is my understanding that Filter and Flatten are both types of projections. So if I then filter the flattened projection like so

persons[?name=='bob'].addresses[][?current]

I would expect this

[
  {
    "current": true,
    "address": "132 Main Ave"
  }
]

but instead I have to pipe the result to a filter projection in order to get this to work:

persons[?name=='bob'].addresses[] | [?current]

Question / Issue

I would not expect that I have to first use the pipe operator in order to achieve that. Why is it that the filter is not applied as a projection of each result of the flatten operation? Am I misunderstanding something about what is/is not a projection and how the projection algorithm is supposed to work?

daniefer avatar Aug 14 '20 15:08 daniefer

I think without the pipe the second filter is being applied to each of the results, rather than the whole result set.

darrenmothersele avatar Aug 15 '20 07:08 darrenmothersele

I'm still confused on why the interpreter sees a difference between filtering a array property and a flattened array? They are both projections are they not? If so, what am I missing in the documentation on vs this implementation? The implementation reads like the [][?<filter expression>] should work.

daniefer avatar Aug 24 '20 13:08 daniefer

NB: Sorry, I know this doesn't answer your question but I found it interesting, so I thought I would share.

I sometimes find it useful to look at the parsed AST to see what's going on: https://codesandbox.io/s/angry-water-kzrcm?file=/src/index.js

If you compare expr3 and expr4 where I've used an index expression, you can see how it nests differently.

darrenmothersele avatar Aug 24 '20 13:08 darrenmothersele

@daniefer I think this detailed explanation of the concept of projection may help you.

@darrenmothersele is correct, you need a | to stop the projection. The reason for that is that the flatten-expression [] returns a projection, whereas the subsequent filter-expression [%…] expects an array. So you need to stop the projection in between, otherwize, the filter-expression will be triggered multiple-times, once per items in the projection returned by the upstream flatten-expression.

springcomp avatar Aug 09 '22 11:08 springcomp