Lightkurve plots sometimes do not show in jupyter notebooks

[christinahedges]

Problem description

Lightkurve's plot methods usually show plots inline in jupyter notebooks. Since v2.3 was released, I've noticed that sometimes plots do not render, and I have to use %matplotlib inline or plt.show() to get plots to show up. This is quite hard to debug because it doesn't always happen.

Example

import lightkurve
# insert code here ...

Expected behavior

Plots should always show in a jupyter environment.

Environment

• platform OSx, TIKE
• lightkurve version 2.4
• installation method pip

[vvilhelmus]

I have the same issue when I use Lightkurve via Poetry. The first plot shows perfectly fine, but the plots that follow that initial plot do not show up at all. Same thing is happening now in the 'Quickstart' tutorial: https://docs.lightkurve.org/quickstart.html ...as well as with some of the other tutorials on the website, like for example: https://docs.lightkurve.org/tutorials/2-creating-light-curves/2-1-cutting-out-tpfs.html

Ken Mghell uses the plt.close() function after the plt.show() function as shown on his GitHub webpage: https://github.com/KenMighell/mkpy3 Perhaps this is what is missing in the Lightkurve codes? Maybe the code thinks it is still in the first plot, because it hasn't been closed properly?

[christinahedges]

@vvilhelmus thanks for commenting and corroborating, that's a really good point about the tutorials. I hadn't seen this issue in v2.3. We need to do some digging to figure out what's changed and why this isn't working.

[tylerapritchard]

I've been taking a look at this issue this afternoon @christinahedges and wanted to give an update - my thoughts so far:

• I can reproduce this with a new conda environment and pip install lightkurve

• plt.show() does cause the missing plots to render when called

• including previously unrendered plots if plot() was called multiple times

• plt.close() doesn't allow the next .plot() call to render as we might expect, since the first plot() call in a notebook works

• Looking through the lightkurve changelog, if this behavior wasn't seen in 2.3, I'm not seeing any changes in lightkurve that I would reasonably expect to cause this change in behavior

• Forcibly deprecating dependent packages does seem to resolve this issue - e.g. New environment with Jupyter=1.0.0 & matplotlib=3.1 seems to allow serial lk.plot() calls to render, suggesting that this change in behavior is possibly due to changes of behavior in these packages

• Adding a plt.show() call to the last line of lightcurve._create_plot() does allow serial plot calls to render.

I had a discussion with @Nschanche in the office, and the question was raised if this was intended behavior as the %matplotlib inline magic command exists for this reason?

Assuming that I'm on the right track here, I think the next steps are: To Do:

• I'll spend some time looking through change-logs for the jupyter/ipython/matplotlib modules to see if I can find what triggered this change in behaviour, and if there is a better way to approach this
• Investigate if there's any expected/demonstrable adverse effects from including the plt.show command, or if there is a preferred method of forcing a render in a module that is calling matplotlib

[christinahedges]

@tylerapritchard great investigation. If you rollback your lightkurve version to 2.3 does the issue go away? I'm finding on TIKE this issue doesn't happen, and TIKE uses 2.3.

@Nschanche is right that %matplotlib inline exists for this reason, but it is odd that this behavior has changed suddenly and unexpectedly. It's also odd that the first plot is fine, but subsequent plots are not. Does using that magic remove this problem?

To add my two cents, the downside of plt.show is that if you e.g. execute 3 figures and then call plt.show() it will then show them all at the same time, rather than at each execution.

[tylerapritchard]

@Nschanche is right that %matplotlib inline exists for this reason, but it is odd that this behavior has changed suddenly and unexpectedly. It's also odd that the first plot is fine, but subsequent plots are not.

Yeah, absolutely, very much agreed! Both of these are concerning.

%matplotlib inline did resolve this issue from my quick testing

Installing Lightkurve 2.3 in a fresh python/jupyter/matplotlib environmoent did NOT resolve this issue for me

open questions then - I'll try and get a copy of the TIKE environment and compare versions against what I'm running which may help narrow things down if it is an upstream/environment issue, and do some more digging in the lightkurve 2.3->2.4 change-log

[tylerapritchard]

After some more sleuthing with @christinahedges, it looks like we're both seeing the same behavior and that it is probably upstream of lightkurve or an environment issue. What we've found:

• Running a regular matplotlib plot first seems to stop this from happening, subsequent lightcurve.plot() calls work as. expected
• %matplotlib inline solves the issues for both of us as well
• It looks like this might be related to ipython/matplotlib-inline/issues/22 as the behavior seems very similar to what is discussed there
• One question was if this was related to a matplotlib rendering backend - both in notebook and in lightcurve.plot() (when called from a notebook) the module://matplotlib_inline.backend_inline backend is being used. In theory this means that this should be working without requiring a %matplotlib inline command I think.
• This issue is wierdly stochastic on which versions of matplotlib this works for. The TIKE TESS environment is running matplotlib 3.6.3 & lightkurve 2.3 and isn't evidencing this issue, 3.7 and 3.5 were showing this issue, 3.1 was not if my understanding of our testing versions is correct. ipython and jupyter/lab packages varied a bit across our tests and didn't seem to affect this. matplotlib-inline seemed to be at 0.16 for them all.
• Is this related to plotting in a specific context with a style? lightcurve._create_plot() returns an axes with with plt.style.context(style): and the matplotlib-inline issue above is plotting in a seaborn style context with plt.style.context('seaborn'):. This might be worth looking into, uncertain.
• We further discussed the insertion of a plt.show() command at the end of lightcurve._create_plot() and decided against this. As Christina pointed out that while this would fix this jupyter notebook/lab functionality, it would break currently existing iPython functionality.

In the short term, it does look like the %matplotlib inline magic funtion is working across multiple environments and this could be used to fix tutorials easily so that they aren't breaking for users

[isabelangelo]

We reproduced this error at the lightkurve sprint for a number of different matplotlib/jupyter versions. For some reason matplotlib 3.5 seems to work. Because the source of the issue seems to be more upstream than the lightkurve source code, adding %matplotlib inline (or %matplotlib notebook) in the tutorials is probably the simplest solution.