otoole icon indicating copy to clipboard operation
otoole copied to clipboard

[FEATURE]: Allow reading in of result data via `read` function

Open trevorb1 opened this issue 10 months ago • 0 comments

Feature Request

It would be nice to use the Python API read() function to read in already processed result csv data. For example, if I process my solution results once to a folder of csvs, I can then uses these results in a variety of downstream scripts.

Is your feature related to a bug?

#169, #179,

Suggested Solution

#169 proposes a input flag to skip the name check. But may be easier to just read in all names and check that all read names intersect with all defined names in the config, and leave it at that.

Additional Info

For example, this code works for reading input data

from otoole import read
input_data, input_deafults = read("config.yaml", "csv", "data")
print(input_data["SpecifiedAnnualDemand"])
$ python read_data.py 

                      VALUE
REGION     FUEL YEAR       
SIMPLICITY FEL1 2014  2.214
           FEL2 2014  0.000
           FEL1 2015  2.225
           FEL2 2015  0.000

But this code fails when reading in already processed result data.

from otoole import read
input_data, input_deafults = read("config.yaml", "csv", "results")
print(input_data["Demand"])
$ python read_results.py 
Traceback (most recent call last):
  File "/home/trevor/repos/simplicity/otoole_issue.py", line 3, in <module>
    input_data, input_deafults = read("config.yaml", "csv", "results")
                                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/trevor/repos/otoole/src/otoole/convert.py", line 417, in read
    return read_strategy.read(from_path)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/trevor/repos/otoole/src/otoole/read_strategies.py", line 214, in read
    self._compare_read_to_expected(names=names)
  File "/home/trevor/repos/otoole/src/otoole/input.py", line 531, in _compare_read_to_expected
    raise OtooleNameMismatchError(name=errors)
otoole.exceptions.OtooleNameMismatchError: Names not consistent between data and config file:

AccumulatedAnnualDemand, AccumulatedNewCapacity, AnnualEmissionLimit, AnnualEmissions, AnnualExogenousEmission, AnnualFixedOperatingCost, AnnualTechnologyEmission, AnnualTechnologyEmissionByMode, AnnualVariableOperatingCost, AvailabilityFactor, CapacityFactor, CapacityOfOneTechnologyUnit, CapacityToActivityUnit, CapitalCost, CapitalCostStorage, CapitalInvestment, Conversionld, Conversionlh, Conversionls, DAILYTIMEBRACKET, DAYTYPE, DaySplit, DaysInDayType, Demand, DepreciationMethod, DiscountRate, DiscountRateStorage, DiscountedCapitalInvestment, DiscountedCostByTechnology, DiscountedOperationalCost, DiscountedSalvageValue, DiscountedTechnologyEmissionsPenalty, EMISSION, EmissionActivityRatio, EmissionsPenalty, FUEL, FixedCost, InputActivityRatio, MODE_OF_OPERATION, MinStorageCharge, ModelPeriodEmissionLimit, ModelPeriodExogenousEmission, NewCapacity, OperationalLife, OperationalLifeStorage, OutputActivityRatio, ProductionByTechnology, ProductionByTechnologyAnnual, REGION, REMinProductionTarget, RETagFuel, RETagTechnology, RateOfActivity, RateOfProductionByTechnology, RateOfProductionByTechnologyByMode, RateOfUseByTechnology, RateOfUseByTechnologyByMode, ReserveMargin, ReserveMarginTagFuel, ReserveMarginTagTechnology, ResidualCapacity, ResidualStorageCapacity, SEASON, STORAGE, SalvageValue, SpecifiedAnnualDemand, SpecifiedDemandProfile, StorageLevelStart, StorageMaxChargeRate, StorageMaxDischargeRate, TECHNOLOGY, TIMESLICE, TechnologyFromStorage, TechnologyToStorage, TotalAnnualMaxCapacity, TotalAnnualMaxCapacityInvestment, TotalAnnualMinCapacity, TotalAnnualMinCapacityInvestment, TotalAnnualTechnologyActivityByMode, TotalCapacityAnnual, TotalDiscountedCost, TotalTechnologyAnnualActivity, TotalTechnologyAnnualActivityLowerLimit, TotalTechnologyAnnualActivityUpperLimit, TotalTechnologyModelPeriodActivity, TotalTechnologyModelPeriodActivityLowerLimit, TotalTechnologyModelPeriodActivityUpperLimit, TradeRoute, UseByTechnology, VariableCost, YEAR, YearSplit.

Update config or data with matching names.

trevorb1 avatar Feb 15 '25 00:02 trevorb1