Allow rename to different depths

[philrz]

The following doesn't work, but probably should.

$ echo '{id:{orig_h:10.164.94.120,orig_p:39681(port=uint16),resp_h:10.47.3.155,resp_p:3389(port)}}' | zq -Z 'rename dst:=id.resp_h' -
rename: left-hand side and right-hand side must have the same depth (dst vs id.resp_h)

Details

Repro is with Zed commit f530788.

The repro above is from the rename-error-move.yaml test. When I came across this recently, I was puzzled because it feels pretty trivial to do this in two steps at the Zed layer by combining put and drop.

$ zq -version
Version: v1.11.1-9-gf530788f

$ echo '{id:{orig_h:10.164.94.120,orig_p:39681(port=uint16),resp_h:10.47.3.155,resp_p:3389(port)}}' | zq -Z 'put dst:=id.resp_h | drop id.resp_h' -
{
    id: {
        orig_h: 10.164.94.120,
        orig_p: 39681 (port=uint16),
        resp_p: 3389 (port)
    },
    dst: 10.47.3.155
}

I bounced this off @nwt and he explained that the current limitation is due how the rename implementation surgically alters the type to avoid rebuilding the whole value. This seems clever/efficient, but in a situation like the one above it may leave a user scratching their head to come up with the same workaround I did and then wonder why it didn't just work that way in the first place.

In the same discussion @nwt wondered aloud where the field-at-a-different-depth would land (e.g., dst in this case: would the user expect it to be somehow adjacent to the nested record it was pulled from? as the first or last field at the destination depth?) I'm personally fine with it doing what put does (i.e., put it at the end) and @nwt seemed fine with this too.

[zmajeed]

Seems like there should be a move operator

[zmajeed]

Separately why is the assignment operator := needed for rename - couldn't a comma work for a separator?

rename x,w,y,z

[philrz]

@zmajeed: RE: "move operator". I think this turns out to be another language design topic. Similar to how mv in UNIX/Linux can be used to rename files in the same directory or move them between directories while potentially giving them new names in their destination location, Zed could arguably have one operator that allows both rename-in-place (like the current rename is limited to) as well as renaming fields while moving them between levels (what's proposed here). Upon seeing your remarks one of our developers @mattnibs also noted that he'd wondered in the past if we should call the current operator mv or move instead of rename, and that argument would seem to become even more valid if we adapt the current operator to handle field moves between levels. But the naming is kind of a second-order topic. The bigger blocker is that the code to adapt the current rename implementation to cover this is non-trivial and I doubt the language designers would want to introduce yet another operator just for the "move between levels" use case.

Regarding your rename x,w,y,z comment, I don't think I understand. The := assignment syntax is used to express the "from" and "to" and also allowing for multiple renames at once by separating with commas, e.g.:

$ echo '{"x": 1, "y": 2}' | zq -Z 'rename a:=x,b:=y' -
{
    a: 1,
    b: 2
}

Since your x,w,y,z is just a flat comma-separated list, it's not clear to me what you had in mind in terms of how that would rename what-to-what. Could you write out an example with inputs & outputs just to make sure we're not missing out on a good idea? Thanks!

[zmajeed]

It'd be rename a,x,b,y - commas between and within pairs - to me := is visual clutter here - and feels like a needless confusing overload of its primary use for assignment Actually I think I prefer rename from,to - so rename x,a,y,b would rename x to a, y to b