message_ix icon indicating copy to clipboard operation
message_ix copied to clipboard

Extend user-defined relations to include the index of `time`

Open behnam-zakeri opened this issue 3 years ago • 3 comments

This PR re-introduces user-defined relations (sets, parameters, equations, variables) with the inclusion of sub-annual time index. Background: The user-defined relations in the GAMS formulation do not represent index of time, which can be an issue when working on models with sub-annual time slices (see #191). For example, it is not possible to represent time slices in parameter relation_activity, which relates to the activity of technologies and must be defined at the sub-annual time level. The addition of time to existing sets and parameters means re-initializing these items, which will create backward incompatibility. Via #631 a workaround was suggested, which was implemented in PR#633. The improvements proposed in this PR could use the same workaround as #633, however, a few sets here, namely, is_relation_upper and is_relation_lower are populated in the Java Backend side, which may make re-initializing these sets impossible or complex on the message_ix side, i.e., in models.py.

Solution: Until a robust solution will be in place for re-initializing sets and parameters that are at the moment being populated at the Java side in message_ix, this PR proposes an interim solution as follows:

  • creating mirror sets and parameters for adding the missing index: For example, parameter relation_activity_time is introduced as the mirror of existing relation_activity, and similarly relation_cost_time, relation_lower_time and relation_upper_time are added.
  • creating mapping (or masking) sets on the fly in models.py. This includes is_relation_lower_time and is_relation_upper_time instead of existing is_relation_lower and is_relation_upper, respectively, which are produced on the Java backend.
  • adding duplicate blocks of equations (RELATION_EQUIVALENCE_TIME, RELATION_CONSTRAINT_UP_TIME, and RELATION_CONSTRAINT_LO_TIME) for enhancing the mathematical formulation.

This PR remains as draft for the time being. Merging this PR in the main branch shouldn't create any problems for running existing scenarios. However, having duplicate items (set, parameter, variable and equations) proposed in this PR may not be an optimal solution from a design perspective.

How to review

  1. Check out this branch and run your existing scenarios, and they should work without a flaw.
  2. Load model/scenario "MESSAGEix-CAS/relations_with_time_index" from "ene_ixmp" database and solve it using the code in this PR. This scenario does include the new items proposed in this PR. There shouldn't be any error when loading and solving that.

PR checklist

  • [ ] Continuous integration checks all ✅
  • [ ] Add or expand tests; coverage checks both ✅
  • [x] Add, expand, or update documentation.
  • [x] Update release notes.

behnam-zakeri avatar Dec 01 '22 13:12 behnam-zakeri

Codecov Report

Merging #680 (63a4b0c) into main (cec4092) will not change coverage. The diff coverage is 100.0%.

@@          Coverage Diff          @@
##            main    #680   +/-   ##
=====================================
  Coverage   94.4%   94.4%           
=====================================
  Files         43      43           
  Lines       3448    3448           
=====================================
  Hits        3257    3257           
  Misses       191     191           
Impacted Files Coverage Δ
message_ix/tests/test_tutorials.py 100.0% <ø> (ø)
message_ix/models.py 98.2% <100.0%> (ø)
message_ix/tests/test_reporting.py 100.0% <100.0%> (ø)

codecov[bot] avatar Mar 10 '23 10:03 codecov[bot]

#705 already includes the content of commit 08ef0c75f441e29d9281e56faf78f6c0b293a49c, so when you continue working on this PR, you can/have to exclude this commit. See this comment for how to do that.

glatterf42 avatar Apr 18 '23 13:04 glatterf42