open-grid-emissions icon indicating copy to clipboard operation
open-grid-emissions copied to clipboard

Re-assess cleaning of EIA-930 data

Open gailin-p opened this issue 1 year ago • 4 comments

Physics-based cleaning

We use gridemissions to clean EIA 930 data. Currently we use 930 generation, however, this causes issues during the calculation of consumed emission rates. (see #214)

A better solution may be to do physics-based cleaning using our own generated MWh. This would require modifying the optimization algorithm to prioritize changing interchange and demand values over net generation values, since we trust our net generation estimates over the 930 values.

We would also need to provide user guidance on compiling cvxpy, since it can be extremely slow for some users and this change would require running optimization during every run of data_pipeline.py.

Systematic issues

The initial correction of systematic issues in 930 data depended on comparison of interchanges between BAs. Now that we have the OGE dataset, we can do a more thorough comparison.

One example of this is AZPS, where there is a sign flip on June 1, 2020, that was not caught in the initial analysis.

Screen Shot 2022-09-06 at 9 47 37 PM Red is our `net_generation_mwh`, blue is total interchange as reported to EIA-930 after `gridemissions` cleaning

gailin-p avatar Sep 07 '22 02:09 gailin-p