taxdata icon indicating copy to clipboard operation
taxdata copied to clipboard

Add 2019 CPS

Open MaxGhenis opened this issue 5 years ago • 2 comments

Looks like this is blocked by NBER not yet having published their SAS script at http://data.nber.org/data/cps_progs.html. Is that right @andersonfrailey?

MaxGhenis avatar Jul 20 '20 19:07 MaxGhenis

Actually I'd bet they don't have SAS scripts because the 2019 files are just CSVs: http://data.nber.org/cps/asecpub19csv.zip

Will this require merging the person/family/household CSVs then? e.g. ffpos isn't in the person record.

MaxGhenis avatar Jul 20 '20 19:07 MaxGhenis

The Census Bureau published the 2019 ASEC as a fixed-width file. They don't have SAS input statements like NBER does (only for the replicate weights), but I tried just applying the taxdata 2018 logic on it using #345, which works correctly for 2018 data. But it looks like the file structure changed:

(base) mghenis@penguin:~/PSLmodels/taxdata/cps_data/pycps$ python create.py 
C-TAM imputed benefits are not available for 2018. Creating file with benefits reported in the CPS.
Reading DAT file
Creating Records
2018:   0%|                                                                                                                                                                                                       | 0/354345 [00:00<?, ?it/s]
Traceback (most recent call last):
  File "create.py", line 158, in <module>
    benefits=True, verbose=True
  File "create.py", line 78, in create
    benefits=_benefits, exportpkl=exportpkl, exportcsv=exportcsv
  File "/home/mghenis/PSLmodels/taxdata/cps_data/pycps/cpsmar.py", line 147, in create_cps
    house = parse(record, parsing_dict["household"])
  File "/home/mghenis/PSLmodels/taxdata/cps_data/pycps/cpsmar.py", line 102, in parse
    value = int(rec[start: end])
ValueError: invalid literal for int() with base 10: ''

I'll try merging the CSVs instead.

MaxGhenis avatar Jul 21 '20 00:07 MaxGhenis