CBO baseline update

[bodiyang]

This PR serves as a follow up work after Tax-Data CBO baseline update.

New input data files will be updated in Tax-Calculator to match the update in Tax-Data

[codecov[bot]]

Codecov Report

Merging #2662 (c2ac9fb) into master (98b23dc) will not change coverage. The diff coverage is 100.00%.

Additional details and impacted files

@@           Coverage Diff           @@
##           master    #2662   +/-   ##
=======================================
  Coverage   98.54%   98.54%           
=======================================
  Files          14       14           
  Lines        2609     2609           
=======================================
  Hits         2571     2571           
  Misses         38       38           

Flag	Coverage Δ
unittests	98.54% <100.00%> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files	Coverage Δ
taxcalc/policy.py	100.00% <100.00%> (ø)
taxcalc/utils.py	97.79% <100.00%> (ø)

[jdebacker]

@bodiyang, when I run the unit tests on this branch on my machine, I get the following failures:

FAILED taxcalc/tests/test_calculator.py::test_itemded_component_amounts[2017-c18300-ID_AllTaxes_hc]
FAILED taxcalc/tests/test_calculator.py::test_itemded_component_amounts[2017-c19700-ID_Charity_hc]
FAILED taxcalc/tests/test_compare.py::test_itax_compare[True] - ValueError: C...
FAILED taxcalc/tests/test_puf_var_stats.py::test_puf_var_stats - ValueError: ...
FAILED taxcalc/tests/test_pufcsv.py::test_agg - ValueError: PUFCSV AGG RESULT...
ERROR taxcalc/tests/test_utils.py::test_table_columns_labels - ValueError: AC...
== 5 failed, 346 passed, 44 skipped, 1 warning, 1 error in 1517.22s (0:25:17) ==

The PUF tests are not run on GH Actions. Are you getting everything to pass on your machine?

[bodiyang]

@jdebacker all tests have been passed locally, including the PUF tests. However, the online built in tests failed now (before this commit, the tests were passing online).

[jdebacker]

@bodiyang Thanks for the latest commits. Test failures seem to be related to an issue in the latest release of bokeh (see here). Unfortunately, I don't see a solution yet.

[jdebacker]

@bodiyang Have you pushed all your changes to this branch? I synced and ran tests again locally and still found failures.

[bodiyang]

@jdebacker yes I have pushed all changes

https://github.com/PSLmodels/Tax-Calculator/pull/2662/commits/56708fb9637262e6d9042750e6f4319420fe686f

[jdebacker]

@bodiyang Thanks for confirming. I created a new PUF from the most recent taxdata and then ran the unit test. All test, except for those using bokeh passed, confirming your results.

My last comment then, is that you should update the cps.csv.gz file with the the version produced from the most recent taxdata (and any associated files that should be updated) with this PR (e.g., as was done in PR #2599).

[bodiyang]

@bodiyang Thanks for confirming. I created a new PUF from the most recent taxdata and then ran the unit test. All test, except for those using bokeh passed, confirming your results.

My last comment then, is that you should update the cps.csv.gz file with the the version produced from the most recent taxdata (and any associated files that should be updated) with this PR (e.g., as was done in PR #2599).

I've run the make all and make cps-files commend in Tax-Data, to generate all cps files. However only cps_raw.csv.gz is updated, while cps.csv.gz is not updated. Checked stage-II.py, there is no code do the update for cps.csv.gz. Then I try to run createcps.py script, still trying to generate the cps.csv.gz file. But there is no change made to the cps.csv.gz file ~ the time of modify of the file looks changed, but the file itself is not. So possibly the cps.csv.gz file has no update from this CBO baseline update. Any other possibilities?

[jdebacker]

@bodiyang Ok, I confirmed my cps.csv.gz produced by taxdata:master gives exactly the same data as what is in the cps.csv.gz file in Tax-Calculator.

In tax data PR #412, @andersonfrailey notes that it was odd that the CPS file did not change, but could not identify any incorrect steps in that PR so it was merged anyway.

Thus, I suppose it's ok that there are no changes to the cps.csv.gz file.

[jdebacker]

@bodiyang I just opened a PR to your CBO-baseline branch which should take care of the test failures here. Please give it a look and merge it in and we'll see if that's all we need.

[jdebacker]

@bodiyang Thanks or merging my PR. It seems there is still one test failure when using bokeh >= 3.0.0 with a serialization error. I found an issue related to that, but it seems to suggest it's been resolved. I have unsuccessfully tried to find a work around in our case.

Let's leave this sit for another day or so and if nothing turns up, perhaps we just pin to bokeh < 3.0.0 in the environment.yml file.

[jdebacker]

@bodiyang Can you change environment.yml to replace - "bokeh>=1.4.0, <3.0.0"

[bodiyang]

@bodiyang Can you change environment.yml to replace - "bokeh>=1.4.0, <3.0.0"

Have updated the package, now passed all tests with PUF locally and also the online testing

[jdebacker]

LGTM - thanks @bodiyang!