Performance improvements in glue and echo related to subsets

[astrofrog]

I've been investigating performance issues in jdaviz, starting from the massive slowdown in cubeviz after making just a few subsets. This is a meta-issue to keep track of the work on this.

Changes

I found several places in echo where it was possible to avoid repeated/duplicate calls:

• https://github.com/glue-viz/echo/pull/46
• https://github.com/glue-viz/echo/pull/48
• https://github.com/glue-viz/echo/pull/50

With these changes, I sped up cubeviz with 10 subsets by a factor of 4.5x. I have now released echo 0.11 which contains these changes. The changes are pretty generic and will likely result in speedup in other jdaviz applications too.

I have also made some improvements in glue-core for a further factor of 2x speedup:

• https://github.com/glue-viz/glue/pull/2544 (speeds cubeviz up by ~30%)
• https://github.com/glue-viz/glue/pull/2545 (speeds cubeviz up by ~25%)

And I've also found some bottlenecks in jdaviz:

• https://github.com/spacetelescope/jdaviz/pull/3626
• https://github.com/spacetelescope/jdaviz/pull/3628

Test code

I've been testing performance using:

from regions import PixCoord, CirclePixelRegion
from jdaviz import Cubeviz
cubeviz = Cubeviz()
uri = "mast:JWST/product/jw02732-c1001_t004_miri_ch1-short_s3d.fits"
cubeviz.load_data(uri, cache=True)
N=12
for iter in range(N):
    my_reg = CirclePixelRegion(center=PixCoord(x=35, y=35), radius=10)
    cubeviz.plugins['Subset Tools'].import_region(my_reg, combination_mode='new')

Results

Here is the time to add N subsets by varying N above with the original code, with echo 0.11, and with echo 0.11 and the jdaviz changes above. This doesn't yet include improvements related to glue-core PRs:

and in log:

and here is the final speedup: