USA Spending Data Not Added for Counties

[tthibo]

Attempted to use the CA donor data from the Examples directory and adding USA Spending Fed Assistance data.

Description says it will add "137 columns" -- which seems more problem than help, really.

But in fact, only 5 columns were added, with no values:

[cathydeng]

This is happening b/c the USA Spending mancer is the only mancer where the # of columns that can be added is dependent upon the geography selected. Because we store all the columns ahead of time (b/c it takes a long time to grab all these tables on the spot), the USA Spending mancer will show all possible columns for all geographies.

There isn't a clear fix for this - we'll think on it - but since it doesn't ultimately break or produce errors (the listing is the inaccurate part, but on the download page, it'll correctly say 5 columns added), we'll prioritize the other issues first and then get back to this one if we have time.

[cathydeng]

Some updates:

This issue is actually 2 issues: (1) county data doesn't exist for USA spending, & instead of returning an error or no data, the api returns zeros & (2) the columns that are added are dependent upon the geography type (my last issue comment). The solution to (1) would just be to remove County as a valid geography for USA spending data. The solution to (2) would be to break out each of the datasets (fpds, faads, fsrs) into a separate dataset for each geography type, since the columns available are different for each geography.

But anyways, a bigger issue is that the USA Spending API is unreliable - it was just updated & broke the existing code, & the new API fails to handle requests without record limits (& even fails on low record limits, pretty much anything over 10 records at a time). Since there also aren't clear use cases for joining USA spending data on geography, we'll just drop this mancer, as agreed on the call.