spark-monitoring icon indicating copy to clipboard operation
spark-monitoring copied to clipboard

Log Analytics query memory

Open cchenshu opened this issue 2 years ago • 0 comments

SparkMetric_CL
| where name_s contains "driver.jvm.total."
| where executorId_s == "driver"
| extend memUsed_GB = value_d / 1000000000
| project TimeGenerated, name_s, memUsed_GB
| summarize max(memUsed_GB) by tostring(name_s), bin(TimeGenerated, 1m)

Hi, I know Driver memory consumption could use the above one to query. I am now using Azure log analytics to monitor Databricks, and it has one driver and one worker node. So, how about worker node? How could I query memory usage for worker node(or the total memory usage)?

Many Thanks!

cchenshu avatar Jul 11 '22 06:07 cchenshu