Cisco ISE - Hunting Queries generates E_RUNAWAY_QUERY error
I am working on implementing Sentinel for a new customer which is using Cisco ISE, and we would like to make use of the Cisco ISE Solution. However we are having a issues where queries using the CiscoISEEvent function provided in the solution fail will generate E_RUNAWAY_QUERY and E_LOW_MEMORY_CONDITION errors when we try and use the out of the box Workbooks or Hunting Rules
To Reproduce Steps to reproduce the behavior:
- Within Sentinel, Browse to the Cisco ISE Solution from the Content Hub
- Select the
CiscoISE - Authentication attempts to suspended user accountHunting Query, and click View Results. - An error is returned:
{
"code": "PartialError",
"message": "There were some errors when processing your query.",
"details": [
{
"code": "EngineError",
"message": "Something went wrong processing your query on the server.",
"innererror": {
"code": "-2133196799",
"message": "'sort' operator has exceeded the memory budget (5368709120) during evaluation. Results may be incorrect or incomplete (E_RUNAWAY_QUERY; see https://aka.ms/kustoquerylimits).",
"severity": 2,
"severityName": "Error"
}
},
{
"code": "EngineError",
"message": "Something went wrong processing your query on the server.",
"innererror": {
"code": "-2133196793",
"message": "Partial query failure: Low memory condition (E_LOW_MEMORY_CONDITION). (message: DefaultByteStreamResizeCallback: allocation failed: )",
"severity": 2,
"severityName": "Error"
}
}
]
}
Expected behavior The results of the query should be returned
Additional context I have tested some of the other hunting queries and get the same error. We have approximately 400 devices managed be ISE sending their RADIUS/TACACS logs, and the volume is approximately 70k - 80k entries per 24h. We have made sure that ISE is not sending debug messages or other which would be obvious low security value.
Hello @andrewj-t, thanks for flagging this issue. We will investigate this issue and get back to you with some updates. Thanks!
Thanks @v-utpalkumar I did some further testing by modifying the query in CiscoISEEvent to not try and reconstruct the multi-part message, and that does fix the performance issues, however it causes a number of fields to be missing from the messages, as the messages are no longer parsed fully. Let me know if you need any more information which could be useful