pyhf icon indicating copy to clipboard operation
pyhf copied to clipboard

fix: Check parameter shapes for pdf API calls

Open kratsg opened this issue 4 years ago • 3 comments

Pull Request Description

Resolves #1459. Check that expected_data, expected_auxdata, and expected_actualdata are being called with the right shape for pars (the parameters) before evaluating. This is usually caught by lower-level code, however the error is not as user-friendly.

A quick note: it is ok to add if/raise in these API calls as they're not meant to be "fast" in our code -- but this could potentially be a problem if we need to allow for autodiff capabilities. For now, we're only enforcing that logpdf() calls are the performant ones.

Checklist Before Requesting Reviewer

  • [x] Tests are passing
  • [x] "WIP" removed from the title of the pull request
  • [x] Selected an Assignee for the PR to be responsible for the log summary

Before Merging

For the PR Assignees:

  • [x] Summarize commit messages into a comprehensive review of the PR
* Add checks on input parameter shapes for the pdf API calls to expected_auxdata,
expected_actualdata, and expected_data.
* Add tests for parameter shape checks to properly raise exceptions.InvalidPdfParameters

kratsg avatar May 17 '21 12:05 kratsg

Codecov Report

Merging #1461 (390b100) into master (ce70574) will increase coverage by 0.00%. The diff coverage is 100.00%.

Impacted file tree graph

@@           Coverage Diff           @@
##           master    #1461   +/-   ##
=======================================
  Coverage   98.12%   98.12%           
=======================================
  Files          64       64           
  Lines        4270     4278    +8     
  Branches      683      687    +4     
=======================================
+ Hits         4190     4198    +8     
  Misses         46       46           
  Partials       34       34           
Flag Coverage Δ
contrib 26.20% <0.00%> (-0.05%) :arrow_down:
doctest 60.47% <0.00%> (-0.12%) :arrow_down:
unittests 96.18% <100.00%> (+<0.01%) :arrow_up:

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
src/pyhf/pdf.py 97.85% <100.00%> (+0.05%) :arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more Δ = absolute <relative> (impact), ø = not affected, ? = missing data Powered by Codecov. Last update ce70574...390b100. Read the comment docs.

codecov[bot] avatar May 17 '21 12:05 codecov[bot]

I wonder whether for these types of API we shsould have a pattern

def method(self,...):
    # check inputs
    ....
   self._method(...)

such that for performance critical or internal paths (i.e. where pyhf itself calls method we're able to call the "unsafe" _method - relying on prior checks, while the user gets a friendly API with safety checks

lukasheinrich avatar May 17 '21 12:05 lukasheinrich

I wonder whether for these types of API we shsould have a pattern

I had this idea where I wanted to use decorators. It can introspect the arguments and do the checks based on something consistent, rather than copy/paste code around a lot. In this case, something like @pyhf.checks.pars that checks the number of parameters passed in for pdf calls, and would become a passthrough based on some pyhf.config configuration or similar.

kratsg avatar May 17 '21 12:05 kratsg