zed icon indicating copy to clipboard operation
zed copied to clipboard

Zeek format output should disambiguate "(empty)"

Open philrz opened this issue 5 years ago • 1 comments

As mentioned originally in a comment of https://github.com/brimsec/brim/issues/832, zq -f zeek currently outputs a string value of (empty) in a way that's out of sync with how Zeek itself behaves. We should change zq output to match what Zeek does.

Consider this Zeek script:

module Empty;

export {
    redef enum Log::ID += { LOG };

    type Info: record {
        ts:            time &log;
        my_str:        string &log;
        };
    }

event zeek_init()
    {
    Log::create_stream(Empty::LOG, [$columns=Empty::Info, $path="empty"]);

    Log::write( Empty::LOG, [$ts=network_time(),
                             $my_str="(empty)"]);
    }

Run through Zeek v3.1.3, it produces the following output empty.log:

#separator \x09
#set_separator	,
#empty_field	(empty)
#unset_field	-
#path	empty
#open	2020-05-31-15-41-52
#fields	ts	my_str
#types	time	string
1590964912.290580	\x28empty)
#close	2020-05-31-15-41-52

That is, since (empty) is a reserved word to be used specifically for the contexts of identifying empty sets and vectors, when appearing as a value of a string-type field, Zeek does the right thing and escapes it.

Meanwhile, zq as of commit 56727dd passes the value through as-is.

$ echo '{ "foo": "(empty)" }' | zq -f zeek -
#separator \x09
#set_separator	,
#empty_field	(empty)
#unset_field	-
#fields	foo
#types	string
(empty)

philrz avatar Jun 01 '20 23:06 philrz

This issue is still present as of current Zed commit ae1ac60.

$ zq -version
Version: v1.2.0-24-gae1ac60a

$ echo '{ "foo": "(empty)" }' | zq -f zeek -
#separator \x09
#set_separator	,
#empty_field	(empty)
#unset_field	-
#fields	foo
#types	string
(empty)

philrz avatar Aug 15 '22 19:08 philrz

Verified in Zed commit 2d52400.

Now when the test data is output in Zeek format, the string is escaped to disambiguate it from the true signifier of emptiness.

$ zq -version
Version: v1.2.0-29-g2d52400f

$ echo '{ "foo": "(empty)" }' | zq -f zeek -
#separator \x09
#set_separator	,
#empty_field	(empty)
#unset_field	-
#fields	foo
#types	string
\x28empty)

Thanks @nwt!

philrz avatar Aug 23 '22 00:08 philrz