trubka icon indicating copy to clipboard operation
trubka copied to clipboard

Add options to include topic metadata and offsets in list

Open retzkek opened this issue 3 years ago • 0 comments

Today I needed to review the retentions configured for all our topics. Rather than iterating over all the topics with the shell I thought it might make sense to have the list command grab the metadata too. I added -c and -o options, same as describe, that causes it to grab the config and offsets and stick it into a Metadata field.

Right now it only shows up in the json output, which probably makes sense, since the table would be unwieldy with all the metadata (but it would be really nice to be able to choose columns of interest).

You may have some better idea or preference on how this is done, I'm open to ideas, but this met my need so I wanted to offer it back.

Example usage:

$ trubka list topics  -c -t ingest\.dcache\.billing -f json | \
jq -c '.[] | {"name":.name, "retention_days":(.metadata.configurations | from_entries | ."retention.ms" | tonumber /1000/3600/24)}'
{"name":"ingest.dcache.billing","retention_days":7}
{"name":"ingest.dcache.billing.cms-disk","retention_days":7}
{"name":"ingest.dcache.billing.cms-tape","retention_days":30}

retzkek avatar Jul 24 '20 18:07 retzkek