CQL2: Thoughts on arrays and `IN`

[jerstlouis]

After completing an in-depth review of CQL2 (focused on cql2-text so far) as part of our implementation, the following thoughts on arrays and IN occurred to me:

As discussed in this issue, several of the conformance classes for extensions could actually have been defined separately by using functions as an extension mechanism.

One thing that stands out as not possible to define this way are the array literals which use the [ and ] syntax.

While I do love the [ and ] syntax for arrays (this is how we define arrays in JSON, eC, ECCSS/CMSS), I realized in CQL2(text) it feels somewhat out of place, because we actually have list of elements elsewhere in the language which use parentheses:

• Following the IN operator
  • NOTE: It would also be useful to be able to use IN with an array on the right-hand expression
• In the WKT geometry definitions (points, linestrings, polygons, etc.)

I realize it's very late in the game, but I am throwing these suggestions for consideration just in case:

• Would it make sense to use ARRAY(1, 2, 3) instead of [ 1, 2, 3 ]?
  • Alternatively, should IN use the array syntax IN [ 1, 2, 3 ]?
• Should IN be usable with array types as a right-hand expressions, where you check containment of a single element inside a set?

I am also curious whether the original CQL had array and array operators, or if this is a recent addition? If it is a recent addition, I imagine it is not widely implemented yet, so it might be worthwhile to make this change for greater consistency.

[pvretano]

No. The original CQL did not include arrays. I think arrays were added as a result of issue #384.

[jerstlouis]

@pvretano In that case, I would strongly suggest that:

• The syntax in cql2-text be ARRAY(1, 2, 3) to be consistent with everything else such as WKT geometry, time intervals, IN, and does not require the introduction of a completely new syntactical element (square brackets).
• IN accepts array expressions as a right-hand operand when both Advanced Comparison and Array Operators conformance classes are supported

P.S.: Actually perhaps what is even more consistent with WKT and IN is to not require ARRAY and support simply (1,2,3).

P.P.S.: And WKT uses tuples not separated by commas for coordinates inside those array literals e.g., 10 20 in POLYGON((10 30, 20 0)), or directly as arguments e.g., 30 10 in LINESTRING(30 10, 10 30, 40 40)

i.e., I think it is possible to have a minimalistic grammar (very few rules) also covering embedded WKT literals by supporting array literals using ( ) and tuples of space-separated expressions.

[pvretano]

Not opposed to it but we will need to discuss in the SWG. As you pointed out, I was trying to go for an array syntax similar to JSON.

[jerstlouis]

The (1,2,3) syntax is nice because it could be consistent with WKT and the in syntax.

However, I am not sure how we could distinguish a sub-expression(123) from an array of a single element (123). Perhaps by limiting sub-expressions to be operation expressions e.g., ExpSub: ( ExpOperation ).

[cportele]

Meeting 2022-07-04: @pvretano will investigate. We might also test this in the July Code Sprint.

[cportele]

Meeting 2022-09-26:

• General agreement to make the change and always use parenthesis (without the "ARRAY" keyword). Arrays are used in the IN operator. @pvretano will create a PR.
• Currently IN cannot take an array-values property on the right hand side. Should this be allowed? @pvretano will check and also look at the support for this in SQL.

[jerstlouis]

The one potential grammar ambiguity resulting from this change is the distinction between an array with a single element and a sub-expression. Perhaps the context (expected type) is enough to disambiguate these?