WIP: Allocate fuel consumption

[cmgosnell]

Mostly amended the interim outputs for the existing net generation allocation methodology to be compatible for the fuel allocation. This involved including the boiler fuel table and the necessary groupby/sums for the boiler for allocation into associate_generator_tables() and prep_allocation_fraction().

There are two main options for allocating fuel:

• "in BF": records that have data in the boiler_fuel_eia923 table.
  • The BF table is reported at the plant/boiler/fuel type level. Boilers are connected in a m:1 relationship to units. And units are connected to generators in a 1:m relationship. So we aggregate the BF boiler data to the unit level. Then associate the unit data with generators and allocated to individual generators within each unit based on capacity.
• "not in BF": records that do not have data in the boiler_fuel_eia923 table.
  • When records do not have associated data in the BF table, we have to use the generation_fuel_eia923 (GF) table. The GF table is reported at the plant/prime mover/fuel type level. Because some plants have some portion of the plant that does report to the BF table, we first determine the "remaining" fuel to report to the "not in BF records". Then we allocate the reminder based on capacity.

Known issues as of 7/29:

• The GF and BF table are not always internally consistent. We generally assume that GF table is more complete, thus the BF table should never have more fuel being reported for any individual plant. That is not always the case and is causing at least some issues with the existing tests.

[review-notebook-app[bot]]

Check out this pull request on 

See visual diffs & provide feedback on Jupyter Notebooks.

---

Powered by ReviewNB

[codecov[bot]]

Codecov Report

Base: 83.0% // Head: 83.3% // Increases project coverage by +0.2% :tada:

Coverage data is based on head (955cf21) compared to base (5bd7db8). Patch coverage: 95.8% of modified lines in pull request are covered.

Additional details and impacted files

@@           Coverage Diff           @@
##             dev   #1096     +/-   ##
=======================================
+ Coverage   83.0%   83.3%   +0.2%     
=======================================
  Files         67      67             
  Lines       7141    7308    +167     
=======================================
+ Hits        5930    6090    +160     
- Misses      1211    1218      +7     

Impacted Files	Coverage Δ
src/pudl/analysis/mcoe.py	90.4% <ø> (ø)
src/pudl/analysis/allocate_net_gen.py	96.7% <95.7%> (-0.6%)	:arrow_down:
src/pudl/output/eia923.py	98.1% <100.0%> (+<0.1%)	:arrow_up:
src/pudl/output/pudltabl.py	87.9% <100.0%> (-0.3%)	:arrow_down:

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

:umbrella: View full report at Codecov.
:loudspeaker: Do you have feedback about the report comment? Let us know in this issue.

[zaneselvans]

Some columns that were previously named fuel_type or fuel_type_code have been re-named to energy_source_code because they use the same set of EIA codes as the energy_source_code_N columns in the generators_eia860 table. This is breaking the allocate_net_gen process because it directly relies on those columns and manipulates them, including renaming them internally sometimes (I think to get the old fuel_type_code and fuel_type columns harmonized between the boiler fuel and fuel receipts and costs tables.) I don't know exactly what needs to happen to fix this, but I suspect it'll be pretty obvious to @cmgosnell

[grgmiller]

Hi @zaneselvans and @cmgosnell thanks for your work on the fuel allocation. I've been working on adapting analysis.allocate_net_gen for use in the eGRID project I'm currently working on. Do you have an expected timeline for when the code for allocating fuel will be functioning? For now, I've been just copying the code from the master branch analysis.allocate_net_gen and adapting it for use in my data pipeline, rather than installing pudl. If I were to copy the code as it exists in the allocate_fuel branch, would it work properly or are there still issues that need to be resolved (like https://github.com/catalyst-cooperative/pudl/pull/1096#issuecomment-1068280137)

[cmgosnell]

hey @grgmiller ! I successfully got dev merged into this branch and did the little bit of work to actually integrate the preliminary results into the main outputs of pudl.analysis.allocate_net_gen.allocate_gen_fuel_by_generator_energy_source()

tbh, I don't have a great beat on what the main remaining issues are, but there is a small section in the notebook in this pr where I was exploring some issues. iirc my intention was to explore the plants where the sum of the plant was not adding up to the total plant fuel in the gf table. I was going to categorize issues and try to implement solutions for them. I'm very happy to help with design or just thinking through weird bugs.

[grgmiller]

Thank you! I'll work on getting this up and running on my computer and start testing it out.