[RDF] Add Skewness and Kurtosis actions to RDataFrame using Welford's algorithm

[edish-github]

This Pull Request:

Adds .Skewness() and .Kurtosis() actions to RDataFrame.

Motivation

In high-energy physics, looking beyond the mean and standard deviation is usually critical.

• Skewness helps in detecting asymmetry in energy distributions (which potentially indicates parity violation).
• Kurtosis helps identify "heavy tails" where rare events (like new heavy particles or dark matter candidates) might hide.

But currently it requires users to either bin data into TH1 (losing precision) or write manual loops. This action allows for exact calculation in a single pass.

Implementation Details

I used Welford's Online Algorithm, which allows us to calculate mean, variance, skewness, and kurtosis simultaneously in one pass. This is numerically stable and fits the RDataFrame parallel map-reduce pattern.

• Implementation Note: I initially explored an explicit SIMD implementation using ROOT::RVec.However, benchmarks on my local machine (M3, Clang -O3) showed that the compiler auto-vectorizes the scalar Welford loop very effectively, so i went for the scalar implementation

Changes or fixes:

• Added SkewnessHelper and KurtosisHelper in tree/dataframe/inc/ROOT/RDF/ActionHelpers.hxx.
• Exposed .Skewness() and .Kurtosis() in tree/dataframe/inc/ROOT/RDF/RInterface.hxx.
• Registered the new action tags in tree/dataframe/inc/ROOT/RDF/InterfaceUtils.hxx.

Checklist:

• [x] tested changes locally (Verified against a reference Welford implementation via test macro)
• [x] updated the docs (Added Doxygen comments matching the StdDev style)

[hahnjo]

Thanks for the pointers to the numerically stable algorithm, I think I will borrow this for the global statistics of ROOT's new histograms :wink:

[edish-github]

@hahnjo Thanks, It’s really cool that the algorithm might be useful for the new histograms too. 🚀

I’m currently working to implement the weighted version (West's algorithm) to support weighted skewness/kurtosis in RDF. The parallel merge steps get complex for cubic terms, but I verified the prototype against calculations, it holds up perfectly.

If you end up needing the weighted merge derivations (adapted from West/Chan) for the histogram work, please let me know—happy to share my prototype if it saves you some digging! 😊

[dpiparo]

Hello. A bit of context around this PR might be useful. Did you propose these changes to address an issue related to a task at hand or about which you learned? In other words, is this done to fix someone's problems?

[edish-github]

@dpiparo Hello, thanks for looking into this.

Motivation for this was primarily about the feature completeness. Since Mean and StdDev are already supported, adding Skewness and Kurtosis seemed like a logical step to bring RDataFrame closer to parity with other data analysis tools. I used Welford's algorithm to make sure the calculations are numerically stable, and I'm currently working on the statistical analysis for weighted distributions for these metrics as well.

I was also happy to see Jonas mention above that this approach might be helpful for the new histograms, so hopefully, this proves to be a useful addition to the codebase.

Best regards, Edish.

[hageboeck]

Hello @edish-github,

I could imagine extending RDF, but before that, I have a little request 😅: Could we separate whitespace changes and code changes? This tremendously helps the review.

We typically use git clang-format to format only the lines that have been touched. Like this, we progressively format the code base, but we don't introduce noise in "older" lines that were unaffected.

[edish-github]

Hello @hageboeck, thanks for reviewing!

I've cleaned up the diff by reverting the unrelated whitespace changes and applying clang-format strictly to the new code. I also added the example usage snippets to the documentation to match the style of the other actions.

On a side note, I am really enjoying working with the RDF internals. Currently I'm working on tackling weighted skewness/kurtosis next (e.g., .Skewness(col, weight_col)), as weighted events are standard in MC analysis. Would that be a welcome addition, or is there a higher priority area in RDF you would suggest?

Best regards, Edish

[edish-github]

Hello @hageboeck (and @vepadulano @martamaja10),

I've updated the PR and ran the tests locally. Could you please trigger the CI workflows when you have a moment? I'd like to ensure the tests pass on the official runners.

Best, Edish

[edish-github]

Hello @hageboeck (@vepadulano and @martamaja10)

It's been a little while, and I completely respect your time and busy schedules. I just wanted to gently bump this PR to see if we can get the workflows approved or if you have any initial feedback on the implementation.

I am standing by for any feedback or changes required. Thanks!