Assignment to array element

[philrz]

Repro is with Zed commit 63d6b29. This variation was actually first called out in https://github.com/brimdata/zed/issues/4651#issuecomment-1622343020.

Here's an example of something that's easy in Python, which might be familiar to our data-centric users due to how Python is often used with Pandas et al.

$ python3
Python 3.11.5 (main, Aug 24 2023, 15:18:16) [Clang 14.0.3 (clang-1403.0.22.14.1)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> numbers = [4,5,6]
>>> numbers[0]=1
>>> print(numbers)
[1, 5, 6]

Attempting the same using Zed with what I'd think of as the equivalent straightforward syntax:

$ zq -version
Version: v1.10.0-5-g63d6b299

$ echo '{numbers:[4,5,6]}' | zq -z 'numbers[0]:=1' -
illegal left-hand side of assignment

It's been pointed out that it's possible to get there today if the user is familiar with slices & spread:

$ echo '{numbers:[4,5,6]}' | zq -z 'cut numbers:=[1,...numbers[1:]]' -
{numbers:[1,5,6]}

However, given how the simple reference to the element on the left-hand side is common in other languages, it seems like it would be convenient to the user if it worked similarly in Zed.

FWIW, I tried to make use of this syntax in https://github.com/brimdata/challenges/blob/aace23e1ef85b7f411323d776977afc4cfa9935e/ames-shaping/shaper.zed#L3-L23.

[philrz]

Here's an example from within the Zed language itself where I imagine users could likely bump into this.

Let's say a user has this test data.

{"Hello": "world", "Number": 1}

Now imagine they want to change the field names to all lowercase, such as was one of the first steps in this Python/Pandas shaping exercise. Right now in Zed, iterating through record fields by name is done via the over operator, so let's say they start by examining the output of that alone.

$ echo '{"Hello": "world", "Number": 1}' | zq -z 'over this' -
{key:["Hello"],value:"world"}
{key:["Number"],value:1}

Notice that key is an array. Therefore, it seems natural a user would first attempt the following:

$ echo '{"Hello": "world", "Number": 1}' | zq -z 'over this => (key[0] := lower(key[0]) | collect(this) | unflatten(this) )' -
illegal left-hand side of assignment

[philrz]

I happened to revisit this issue just to see if the changes in #4795 affected what's described here, and it turns out this did indeed happen. Starting with Zed commit ef9695f which is associated with the changes in #4795, the simple repro shown in the opening of this issue no longer results in the illegal left-hand side of assignment error, but instead does this:

$ zq -version
Version: v1.10.0-16-gef9695f2

$ echo '{numbers:[4,5,6]}' | zq -z 'numbers[0]:=1' -
{numbers:{"0":1}}

I guess it's doing something like converting the 0 to a string value and then treating this as a valid left-hand side indexing reference to a field in a record, hence dropping the array formerly known as numbers and replacing it with a newly-constructed record. But given the Python example shown above and how many other languages treat bare numeric references in contexts like this as indexing into arrays, I'm guessing this may not be what most users would expect to have happen. That might boost the priority of addressing this issue.

[philrz]

A community user recently attempted to perform an assignment to an array element and encountered the effect described in https://github.com/brimdata/super/issues/4798#issuecomment-1789618359 above. Separate issue #6246 has been opened to track that we may surface this as an error until we manage to address this issue.