Function/operator/doc improvements for "net" type
A community user's recent question helped us realize some needed improvements to the helper functions, operators, and docs for working with values of Zed's net type.
Details
At the time this issue is being opened, Zed it at commit c9cf7f1.
A community user had the following inquiry in a recent Slack thread:
Is there a possibility to extract ip and netmask from existing
data type? I'm trying to process an input list of IP addresses (CIDR and IP only) to query a pool of nets to find matching ones. Challenge I'm facing is, network_of doesn't accept net as arguments to query of net in net kind of queries. While network_of() supports ip + ip|int|uint as arguments. What is a recommended approach to perform querying net in net? How to extract ip and netmask of any given net value?
To get them going with the building blocks that are currently in place, I suggested they use the following user-defined function.
func net_extract(network): (
{addr: ip(split(string(network), '/')[0]), masklen: uint8(split(string(network), '/')[1])}
)
If that's stored in file net_extract.zed and they have the following input data in file networks.zson:
{MyNet: 192.168.5.0/24}
{MyNet: 172.16.33.64/26}
it can be used like:
$ zq -version
Version: v1.12.0-39-gc9cf7f15
$ zq -Z -I net_extract.zed 'yield {MyNet,...net_extract(MyNet)}' networks.zson
{
MyNet: 192.168.5.0/24,
addr: 192.168.5.0,
masklen: 24 (uint8)
}
{
MyNet: 172.16.33.64/26,
addr: 172.16.33.64,
masklen: 26 (uint8)
}
That's better than nothing, but this made it clear that there should be more direct ways. We discussed and a few improvements came to mind:
- A cast of a
netvalue toiptype could return just the IP address portion, e.g., instead of the current:
$ echo '192.168.1.0/24' | zq -z 'ip(this)' -
error({message:"cannot cast to ip",on:192.168.1.0/24})
That could return 192.168.1.0.
-
We could have a helper function that returns just the mask length portion of a
netas an integer value. There was some thought of maybe overloading the existinglen()function to cover this, but perhaps a dedicated function with a name likenetmask_len()might be justified so it's easier for users to find in doc searches. -
Right now creating a
netvalue dynamically based on input is needlessly complex, e.g.,
$ echo '{MyIp: 192.168.1.0, MaskLen: 24}' | zq -z 'MyNet:=net(string(MyIp) + "/" + string(MaskLen))' -
{MyIp:192.168.1.0,MaskLen:24,MyNet:192.168.1.0/24}
We could instead allow the / operator on an ip & numeric combo to yield the net value, e.g., MyNet:=MyIp/MaskLen in the example above.
Once we've got these pieces figured out, a small section in the Expressions doc might be justified to show these all working together. That way a network-centric user like the one that posed the question would have one place to look.