trubka
trubka copied to clipboard
Add options to include topic metadata and offsets in list
Today I needed to review the retentions configured for all our topics. Rather than iterating over all the topics with the shell I thought it might make sense to have the list
command grab the metadata too. I added -c
and -o
options, same as describe
, that causes it to grab the config and offsets and stick it into a Metadata
field.
Right now it only shows up in the json
output, which probably makes sense, since the table would be unwieldy with all the metadata (but it would be really nice to be able to choose columns of interest).
You may have some better idea or preference on how this is done, I'm open to ideas, but this met my need so I wanted to offer it back.
Example usage:
$ trubka list topics -c -t ingest\.dcache\.billing -f json | \
jq -c '.[] | {"name":.name, "retention_days":(.metadata.configurations | from_entries | ."retention.ms" | tonumber /1000/3600/24)}'
{"name":"ingest.dcache.billing","retention_days":7}
{"name":"ingest.dcache.billing.cms-disk","retention_days":7}
{"name":"ingest.dcache.billing.cms-tape","retention_days":30}