Daft icon indicating copy to clipboard operation
Daft copied to clipboard

feat: implement stddev with ddof parameter support

Open venkateshdb opened this issue 4 months ago • 5 comments

Changes Made

  • Added ddof param to stddev in DataFrame, GroupedDataFrame, and Expression (default 0 = population; 1 = sample).
  • Introduced StddevParams and updated AggExpr::Stddev to carry ddof through DSL, IR, logical, and physical plans.
  • Implemented population vs sample stddev formulas in planners, updated Python bindings to stddev(ddof) and SQL to support stddev (ddof=0) and stddev_samp (ddof=1).
  • Allowed Null input type for stddev supertype.

Related Issues

Checklist

  • [ ] Documented in API Docs (if applicable)
  • [ ] Documented in User Guide (if applicable)
  • [ ] If adding a new documentation page, doc is added to docs/mkdocs.yml navigation
  • [ ] Documentation builds and is formatted properly (tag @/ccmao1130 for docs review)

venkateshdb avatar Aug 11 '25 09:08 venkateshdb

@venkateshdb I'm going to make this PR a draft since CI is failing. Can you open up the PR for review once you're ready?

srilman avatar Aug 11 '25 18:08 srilman

Codecov Report

:x: Patch coverage is 76.57658% with 26 lines in your changes missing coverage. Please review. :white_check_mark: Project coverage is 74.26%. Comparing base (1f59628) to head (98bc7b9).

Files with missing lines Patch % Lines
src/daft-logical-plan/src/ops/project.rs 0.00% 9 Missing :warning:
src/daft-core/src/array/ops/stddev.rs 50.00% 4 Missing :warning:
daft/dataframe/dataframe.py 50.00% 3 Missing :warning:
src/daft-core/src/python/series.rs 0.00% 3 Missing :warning:
src/daft-core/src/series/ops/agg.rs 57.14% 3 Missing :warning:
src/daft-core/src/utils/stats.rs 85.71% 2 Missing :warning:
src/daft-dsl/src/expr/mod.rs 85.71% 2 Missing :warning:
Additional details and impacted files

Impacted file tree graph

@@           Coverage Diff           @@
##             main    #4949   +/-   ##
=======================================
  Coverage   74.26%   74.26%           
=======================================
  Files         956      956           
  Lines      123101   123184   +83     
=======================================
+ Hits        91424    91487   +63     
- Misses      31677    31697   +20     
Files with missing lines Coverage Δ
daft/expressions/expressions.py 96.55% <100.00%> (ø)
src/daft-core/src/datatypes/agg_ops.rs 62.06% <100.00%> (+0.66%) :arrow_up:
src/daft-dsl/src/python.rs 90.46% <100.00%> (ø)
...ft-physical-plan/src/physical_planner/translate.rs 85.92% <100.00%> (+0.16%) :arrow_up:
src/daft-recordbatch/src/lib.rs 78.61% <100.00%> (ø)
src/daft-sql/src/modules/aggs.rs 72.00% <100.00%> (+2.70%) :arrow_up:
src/daft-core/src/utils/stats.rs 94.59% <85.71%> (-3.80%) :arrow_down:
src/daft-dsl/src/expr/mod.rs 79.55% <85.71%> (-0.14%) :arrow_down:
daft/dataframe/dataframe.py 78.72% <50.00%> (-0.14%) :arrow_down:
src/daft-core/src/python/series.rs 84.69% <0.00%> (-0.84%) :arrow_down:
... and 3 more

... and 8 files with indirect coverage changes

:rocket: New features to boost your workflow:
  • :snowflake: Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • :package: JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

codecov[bot] avatar Sep 06 '25 23:09 codecov[bot]

Hi @srilman, Can you review this PR and can you let me know what changes are needed.

venkateshdb avatar Sep 08 '25 18:09 venkateshdb

@venkateshdb Do you mind responding to Greptiles comments first, whether or not they are reasonable or not? If they are, can you address them?

srilman avatar Sep 08 '25 22:09 srilman

👋 Hi @venkateshdb ! Thanks for making this contribution. Just checking in here -- how is it going addressing the review feedback?

malcolmgreaves avatar Oct 21 '25 21:10 malcolmgreaves

@venkateshdb Thanks again for contributing. Would you be able to address the reptile feedback, so we can then review this?

madvart avatar Nov 19 '25 01:11 madvart

Hi @venkateshdb - Checking to see if you are still planning to work on this?

madvart avatar Nov 25 '25 21:11 madvart

@madvart, Yes

venkateshdb avatar Nov 26 '25 17:11 venkateshdb