Support JSON Lines output format for server req

[ripienaar]

Proposed change

Today people reach to nats server report connections --json to get a full fleet view of connections since it:

• Will fetch all the pages
• Will flatten them all into one enormous struct which is theoretically easier to use

The problem is 30k connections equal about 30GB of RAM when unmarshalled in order to flatten them - and then that has to be marshalled again. So its almost entirely unusable.

But this is unavoidable as I have to build the final structure.

A better approach is to support https://jsonlines.org/ for these enormous use cases, we'd then turn each conn into a json object with server and conn and emit them one line at a time. Thus we can use almost no RAM, just enough to hold one page of responses in memory and unmarshal one response at a time.

So we should:

• Add json lines to nats server req *
• Make a generic paged request helper and support paging in all of nats server req * when JSON lines is enabled
• Hard cap nats server report to some sane limit like 1000 and steer people to JSON lines support instead

Use case

Reporting on large infra

Contribution

No response