Tax-Calculator icon indicating copy to clipboard operation
Tax-Calculator copied to clipboard

Error when calculating n2164 intermediate variable using PUF

Open Peter-Metz opened this issue 4 years ago • 4 comments

@MaxGhenis's work in #2463 splits UBI_21 into UBI_2164 and UBI_65 by calculating an intermediate variable n2164 that counts the number of people in a filing unit between the ages of 21 and 64. Reasonably, he includes this check: assert n2164 >= 0 because a filing unit should not have negative people in it. Unfortunately, that is not the case with the PUF. There are numerous filing units with n2164 values of -1, so calc_all() fails when using PUF.

My guess is that this issue will have to be sorted out in taxdata (cc @andersonfrailey), so it might make sense to revert #2463 until we come up with a solution.

cc @MattHJensen

# code to see filing units with negative people
import pandas as pd
import numpy as np

puf_df = pd.read_csv('puf.csv')
puf_df['n65'] = np.where(puf_df.age_head >= 65,1,0) +
    np.where(puf_df.age_spouse >= 65,1,0) +
    puf_df['elderly_dependents']
puf_df['n2164'] = puf_df['n21'] - puf_df['n65']
puf_df[puf_df['n2164']<0]

Peter-Metz avatar Aug 21 '20 13:08 Peter-Metz

Ah turns out I identified this in June 2019: https://github.com/PSLmodels/taxdata/issues/318

Since #2463 is backwards-incompatible, I'd prefer to drop the assert for now so that it can get in the major release, or maybe set n2164 to zero in the negative cases, until https://github.com/PSLmodels/taxdata/issues/318 can be fixed.

MaxGhenis avatar Aug 21 '20 14:08 MaxGhenis

@MaxGhenis, we could do a phased deprecation: in a minor release, add n2164, n65, and a warning that n21 is being removed in the next major release, please use the other two instead. I've been thinking we need to add this kind of phased deprecation approach anyways (see, e.g., https://github.com/PSLmodels/Tax-Calculator/pull/2413#issuecomment-652593182)

That said, I'm also open to your suggestion of setting n2164 to zero in the negative cases and proceeding with these changes in 3.0.0 for now.

MattHJensen avatar Aug 21 '20 14:08 MattHJensen

Sure, there isn't a rush on this, so reverting #2463 and re-implementing as a phased deprecation sounds good.

MaxGhenis avatar Aug 21 '20 15:08 MaxGhenis

Sounds good, thanks @MaxGhenis. I'll leave this issue open after #2471 reverts #2463.

If this becomes urgent before the TaxData patch is ready, I'll be open to setting n2164 to zero in the negative cases as a temp fix.

MattHJensen avatar Aug 21 '20 16:08 MattHJensen