Collecting metrics on dag file processing can lead to more outbound requests than desirable.

[BrandonTheBuilder]

Summary The scheduler process creates sub-processes to handle updates to dag files. When there are a large number of dags present many short lived processes will happen. If each process collects stats and sends them to New Relic there is a possibility that many more requests per second will be made than desirable.

Success Criteria Reporting of metrics is disabled for dag processing.

• No harvest thread is started from a dag processing process.
• No metrics are reported from a dag processing process.

Notes There is already a method in place to determine what the calling function for a stats call is.

Suggested Implementation

• Verify which methods report stats for dag proccessing
• Update the plugin to not create a recorder or register an atexit hook if the calling method is identified to be a part of dag processing.