datafusion icon indicating copy to clipboard operation
datafusion copied to clipboard

Make CI test error if a function is not documented

Open alamb opened this issue 1 year ago • 1 comments

Draft until we have completely migrated all function documentation. The CI will fail until all currently existing functions are documented

Which issue does this PR close?

Closes https://github.com/apache/datafusion/issues/12872

Rationale for this change

The idea here is to ensure that all functions are documented, and ensure this is the case via a CI test.

What changes are included in this PR?

Changes: make the ./dev/update_function_docs.sh script error if there are undocumented functions

Here is an example failure:

(venv) andrewlamb@Andrews-MacBook-Pro-2:~/Software/datafusion$ ./dev/update_function_docs.sh
/Users/andrewlamb/Software/datafusion
Inserting header
Running CLI and inserting aggregate function docs table
   Compiling datafusion v42.0.0 (/Users/andrewlamb/Software/datafusion/datafusion/core)
    Finished `dev` profile [unoptimized + debuginfo] target(s) in 0.77s
     Running `target/debug/print_functions_docs aggregate`
INFO: The following functions do not have documentation:
  - regr_count
  - regr_avgx
  - regr_syy
  - regr_intercept
  - regr_slope
  - regr_sxy
  - regr_r2
  - regr_sxx
  - regr_avgy
Error: NotImplemented("Some functions do not have documentation. Please implement `documentation` for: {\"regr_count\", \"regr_avgx\", \"regr_syy\", \"regr_intercept\", \"regr_slope\", \"regr_sxy\", \"regr_r2\", \"regr_sxx\", \"regr_avgy\"}")
(venv) andrewlamb@Andrews-MacBook-Pro-2:~/Software/datafusion$

Are these changes tested?

Yes, I tested it manually and here is an example of it failing

Are there any user-facing changes?

alamb avatar Oct 15 '24 11:10 alamb

Example CI failure: https://github.com/apache/datafusion/actions/runs/11344725082/job/31550591277?pr=12938

alamb avatar Oct 15 '24 15:10 alamb

Bam ! Thank you for the reviews @comphead @Omega359 and @timsaucer -- We are so close to getting the docs all migrated over

alamb avatar Oct 25 '24 21:10 alamb