csv export aggregated search results, including aggregate field (like IPv4) and count

[kkplein]

There seems to be no way to export the results of an aggregated search, including the count (for number of occurrences of the aggregated field)

Example: I create a search for "failed authentication" over the last 60 minutes, including remote IPv4 address. The aggregated IPv4 search results look like this in my graylog dashboard:

IPv4	Count()
1.2.3.4	534
5.6.7.8	423
9.10.11.12	123
13.14.15.16	70

I would like to export those results (so only the IPv4 address and the count) to csv. But there is no 'export' button in the actions dropdown menu, see:

It would be very convenient for the actions menu to include the "export" option.

[tmacgbay]

+1 :)

[HungryHowies]

@kkplein I agree that would be nice to have. I tried a couple ways but was unable to get the count results.

[janheise]

My findings on how to implement this functionality:

• the existing CSV export is handled by ElasticsearchExportBackend which is not suited for aggregations. It is geared towards mass export of message lists.
• aggregations are usually are rather small result in comparison
• CSV export of other types than message lists should be done in the regular search backend
• it should only run the single search from the given widget
• it should decorate the result to CSV using a MessageBodyWriter
• Pivot tables can give results that are more like a table, especially with more than one header line. This has to be taken into consideration

[lucas-garcia-rubio]

This feature would be nice. I have to download all the messages I want by slices of time (because I don't have memory enough for so many messages) and aggregate them in my script. It takes too long, but unfortunately it is the only way I found.

[lucas-garcia-rubio]

I found a way out to this problem: inspect your page (F12 on Mozilla), go to network, clean requests, change the timerange on graylog and make the search again. It has to appear a "execution" request. This is the request that Graylog does to populate the tables. Click with the right button and copy the cURL. Paste in somewhere else and analyze the request. You can mimic it in your code and work with the response as a json or a dict.

[riskersen]

+1 I would really appreciate this feature. It doesn't make much sense to prepare nice dashboards and aggregration just to redo them again in Excel after the download.

Imagine also that proxy logs from a specific source might contains millions of rows but I'm only interested in the count per host... I have to download tons of rows just to do the count again.

[szimmer-dap]

+1 for this feature request. Given that the dashboard widgets lack basic functionality like rounding, it makes sense to be able to export aggregated data to another tool.

[fredsilvas]

+1. This would be a really nice feature.

[janheise]

Sorry all that it seems there hasn't been much progress with this ticket. Maybe for some, the new scripting API that exports csv is the solution? https://go2docs.graylog.org/5-1/making_sense_of_your_log_data/simple_search_scripting_api.htm?tocpath=Searching%20Your%20Log%20Data%7C_____5

[alex3ller]

+1

[drewmiranda-gl]

Noting that it looks like this is roadmapped for Graylog 6.1.

[sen-den]

+1

[dennisoelkers]

This has been implemented for 6.1, which is about to be released in October. Closing this issue. Thanks everyone for your input!

[kkplein]

Much appreciated! :-)