Extend user-defined relations to include the index of `time`
This PR re-introduces user-defined relations (sets, parameters, equations, variables) with the inclusion of sub-annual time index.
Background: The user-defined relations in the GAMS formulation do not represent index of time, which can be an issue when working on models with sub-annual time slices (see #191). For example, it is not possible to represent time slices in parameter relation_activity, which relates to the activity of technologies and must be defined at the sub-annual time level. The addition of time to existing sets and parameters means re-initializing these items, which will create backward incompatibility. Via #631 a workaround was suggested, which was implemented in PR#633. The improvements proposed in this PR could use the same workaround as #633, however, a few sets here, namely, is_relation_upper and is_relation_lower are populated in the Java Backend side, which may make re-initializing these sets impossible or complex on the message_ix side, i.e., in models.py.
Solution: Until a robust solution will be in place for re-initializing sets and parameters that are at the moment being populated at the Java side in message_ix, this PR proposes an interim solution as follows:
- creating mirror sets and parameters for adding the missing index: For example, parameter
relation_activity_timeis introduced as the mirror of existingrelation_activity, and similarlyrelation_cost_time,relation_lower_timeandrelation_upper_timeare added. - creating mapping (or masking) sets on the fly in
models.py. This includesis_relation_lower_timeandis_relation_upper_timeinstead of existingis_relation_lowerandis_relation_upper, respectively, which are produced on the Java backend. - adding duplicate blocks of equations (
RELATION_EQUIVALENCE_TIME,RELATION_CONSTRAINT_UP_TIME, andRELATION_CONSTRAINT_LO_TIME) for enhancing the mathematical formulation.
This PR remains as draft for the time being. Merging this PR in the main branch shouldn't create any problems for running existing scenarios. However, having duplicate items (set, parameter, variable and equations) proposed in this PR may not be an optimal solution from a design perspective.
How to review
- Check out this branch and run your existing scenarios, and they should work without a flaw.
- Load model/scenario "MESSAGEix-CAS/relations_with_time_index" from "ene_ixmp" database and solve it using the code in this PR. This scenario does include the new items proposed in this PR. There shouldn't be any error when loading and solving that.
PR checklist
- [ ] Continuous integration checks all ✅
- [ ] Add or expand tests; coverage checks both ✅
- [x] Add, expand, or update documentation.
- [x] Update release notes.
Codecov Report
Merging #680 (63a4b0c) into main (cec4092) will not change coverage. The diff coverage is
100.0%.
@@ Coverage Diff @@
## main #680 +/- ##
=====================================
Coverage 94.4% 94.4%
=====================================
Files 43 43
Lines 3448 3448
=====================================
Hits 3257 3257
Misses 191 191
| Impacted Files | Coverage Δ | |
|---|---|---|
| message_ix/tests/test_tutorials.py | 100.0% <ø> (ø) |
|
| message_ix/models.py | 98.2% <100.0%> (ø) |
|
| message_ix/tests/test_reporting.py | 100.0% <100.0%> (ø) |
#705 already includes the content of commit 08ef0c75f441e29d9281e56faf78f6c0b293a49c, so when you continue working on this PR, you can/have to exclude this commit. See this comment for how to do that.