init sankey

[mabudz]

Create sankey diagram and add tutorial.

The approach here creates the sankey diagram by using the sankey plot function of the pyam package. This function requires a mapping dictionary. The mapping dict is created by using the newly created automatic report message::sankey and a utility function ~sankey_mapper()~map_for_sankey.

This pull request is related to the draft pull request in message_data , which can be removed, since the sankey diagram has been implemented in the pyam package.

[!NOTE] The new key is now only added when calling Reporter.add_sankey().

How to review

General approach should be reviewed. Also, I am not sure if the util function ~sankey_mapper()~ map_for_sankey() is at the right place.

PR checklist

• [x] Continuous integration checks all ✅
• [x] Add or expand tests; coverage checks both ✅
• [x] Add, expand, or update documentation.
• [x] Update release notes.

[CLAassistant]

All committers have signed the CLA.

[glatterf42]

Thanks for this PR :) Getting the tests to pass is a bit tricky since the API for pyam changed when they went from version 1.9.0 to 2.0.0 (as you might expect) and our code needs to satisfy both versions since we test on Python <= 3.9, which is not supported for pyam 2.0.0. So let me take a closer look, maybe we'd want to use the function that pyam's sankey function wraps around directly, after all.

[glatterf42]

Thanks @daymontas1 for stepping up here, much appreciated :)

You should be able to test the PR as is and see how you like it's usefulness, but there are a few things that I would like to see happen before we merge it:

• Please rebase on main to keep it up to date with current developments.
• Please migrate the main functionality from util/__init__.py to somewhere more appropriate. E.g. report/sankey.py if it's a reporting tool or util/sankey.py if there are more general use cases.
• Please check if we can use plotly directly instead of pyam's wrapper functionality. This should allow us to utilize more design options and avoids some dependency constraints (see above). See some examples and their docs.
• Please confirm that we want to integrate the sankey functionality in the reporting as it currently stands. I'd probably ask at least @khaeru for his opinion.
• Please write a test if possible to confirm that sankey_mapper maps the correct values to the correct keys. We might also need to adapt one or two reporting tests that check the number of reporting steps if this stays in the default reporting workflow.

Please feel free to ask about any of these steps if you need help :)

[gidden]

Hi all, to simplify here, I would simply limit this functionality to pyam > 2 - we can do a version check for this on the fly and raise an error.

[glatterf42]

Rebased onto current main and expanded the PR a bit :) Thanks for jumping in here, @gidden :) @daymontas1, please pull these latest changes before making further edits. If you already have some edits locally, you can git stash them, git pull these changes and git stash pop your work on top of this.

The missing type hints (from the failing code quality check) probably indicate that we need to add pyam.* to our mypy config exclude list.

[glatterf42]

The other main error I'm seeing just indicates that by adding this reporter step, we need to adapt the expected value of reporter steps in another test, which is no problem :)

[glatterf42]

Is there any update here, @daymontas1? Since we want to publish a new release still this week (most likely), I'll postpone this to the 3.10 milestone.

[daymontas1]

Hi @glatterf42, I tried to push the changes, but I didn't have permission to do so. So, you need to grant me access. This was my latest comment. Thanks.

[glatterf42]

Alright, that's good to know. @mabudz, could you please give @daymontas1 write access to your fork? You can do that under Settings -> Collaborators and Teams -> Add people. If you do that, please confirm here so we know we can proceed.

If you don't want to wait, @daymontas1, please consider pushing your changes to a branch on your fork or on the main repository: git push -u <name of the repo> <name of the branch>. So you need to pick a repo name from the list produced by git remote -v and can choose any branch name.

[mabudz]

Hi @glatterf42 and @daymontas1 ,

i have added @daymontas1 to the repo ☺️

[glatterf42]

Thanks for the quick reaction, much appreciated :)

[khaeru]

FYI all, see the GitHub docs about the “Allow edits from maintainers” option. AFAIK this is on by default, so an equivalent solution would be to add @daymontas1 to the team @iiasa/messageix-devs. Since that team has (at least) "maintainer" permissions on this repo, he would then have write access to the specific branch used in this PR (so long as @mabudz left the default value for the above option).

This avoids the work of having to (maybe) add every maintainer to every contributor's fork.

[glatterf42]

Thanks, I've also added @daymontas1 to @iiasa/messageix-devs :)

[codecov[bot]]

Codecov Report

Attention: Patch coverage is 26.66667% with 22 lines in your changes missing coverage. Please review.

Project coverage is 23.5%. Comparing base (023b91a) to head (d358873). Report is 63 commits behind head on main.

Additional details and impacted files

@@           Coverage Diff            @@
##            main    #770      +/-   ##
========================================
- Coverage   95.4%   23.5%   -71.9%     
========================================
  Files         46      47       +1     
  Lines       4351    4363      +12     
========================================
- Hits        4153    1029    -3124     
- Misses       198    3334    +3136     

Files	Coverage Δ
message_ix/report/__init__.py	45.8% <ø> (-54.2%)	:arrow_down:
message_ix/util/__init__.py	19.4% <ø> (-79.3%)	:arrow_down:
message_ix/tests/test_util.py	25.0% <23.0%> (-75.0%)	:arrow_down:
message_ix/util/sankey.py	29.4% <29.4%> (ø)

... and 41 files with indirect coverage changes