jdaviz
jdaviz copied to clipboard
Published
20 hours ago •
spacetelescope
spacetelescope
[BUG] TypeError when fitting a model to MaNGA Cubes
Jdaviz component
Cubeviz
Description
Perhaps a lingering issue from https://github.com/spacetelescope/jdaviz/pull/3307: An issue when trying to model fit a MaNGA cube that throws a TypeError in the console log. It seems model fitting still runs (produces WARNING: Model is linear in parameters; consider using linear fitting methods. [astropy.modeling.fitting] warnings) but gives bad results.
How to Reproduce
- Import a MaNGA Cube (e.g.
cubeviz.load_data(f"https://data.sdss.org/sas/dr17/manga/spectro/redux/v3_1_1/7443/stack/manga-7443-12703-LOGCUBE.fits.gz")) - Open Model Fitting plug-in
- Toggle on Cube Fit
- Error Triggered
Full trace:
---------------------------------------------------------------------------
TypeError Traceback (most recent call last)
File [~/Documents/Code/strauss_jdat/vslim/lib/python3.11/site-packages/ipywidgets/widgets/widget.py:773](http://localhost:8891/lab/tree/notebook/~/Documents/Code/strauss_jdat/vslim/lib/python3.11/site-packages/ipywidgets/widgets/widget.py#line=772), in Widget._handle_msg(self, msg)
771 if 'buffer_paths' in data:
772 _put_buffers(state, data['buffer_paths'], msg['buffers'])
--> 773 self.set_state(state)
775 # Handle a state request.
776 elif method == 'request_state':
File [~/Documents/Code/strauss_jdat/vslim/lib/python3.11/site-packages/ipywidgets/widgets/widget.py:650](http://localhost:8891/lab/tree/notebook/~/Documents/Code/strauss_jdat/vslim/lib/python3.11/site-packages/ipywidgets/widgets/widget.py#line=649), in Widget.set_state(self, sync_data)
645 self._send(msg, buffers=echo_buffers)
647 # The order of these context managers is important. Properties must
648 # be locked when the hold_trait_notification context manager is
649 # released and notifications are fired.
--> 650 with self._lock_property(**sync_data), self.hold_trait_notifications():
651 for name in sync_data:
652 if name in self.keys:
File [/opt/homebrew/Cellar/python](http://localhost:8891/opt/homebrew/Cellar/python)@3.11[/3.11.9/Frameworks/Python.framework/Versions/3.11/lib/python3.11/contextlib.py:144](http://localhost:8891/3.11.9/Frameworks/Python.framework/Versions/3.11/lib/python3.11/contextlib.py#line=143), in _GeneratorContextManager.__exit__(self, typ, value, traceback)
142 if typ is None:
143 try:
--> 144 next(self.gen)
145 except StopIteration:
146 return False
File [~/Documents/Code/strauss_jdat/vslim/lib/python3.11/site-packages/traitlets/traitlets.py:1510](http://localhost:8891/lab/tree/notebook/~/Documents/Code/strauss_jdat/vslim/lib/python3.11/site-packages/traitlets/traitlets.py#line=1509), in HasTraits.hold_trait_notifications(self)
1508 for changes in cache.values():
1509 for change in changes:
-> 1510 self.notify_change(change)
File [~/Documents/Code/strauss_jdat/vslim/lib/python3.11/site-packages/ipywidgets/widgets/widget.py:701](http://localhost:8891/lab/tree/notebook/~/Documents/Code/strauss_jdat/vslim/lib/python3.11/site-packages/ipywidgets/widgets/widget.py#line=700), in Widget.notify_change(self, change)
698 if name in self.keys and self._should_send_property(name, getattr(self, name)):
699 # Send new state to front-end
700 self.send_state(key=name)
--> 701 super().notify_change(change)
File [~/Documents/Code/strauss_jdat/vslim/lib/python3.11/site-packages/traitlets/traitlets.py:1525](http://localhost:8891/lab/tree/notebook/~/Documents/Code/strauss_jdat/vslim/lib/python3.11/site-packages/traitlets/traitlets.py#line=1524), in HasTraits.notify_change(self, change)
1523 def notify_change(self, change: Bunch) -> None:
1524 """Notify observers of a change event"""
-> 1525 return self._notify_observers(change)
File [~/Documents/Code/strauss_jdat/vslim/lib/python3.11/site-packages/traitlets/traitlets.py:1568](http://localhost:8891/lab/tree/notebook/~/Documents/Code/strauss_jdat/vslim/lib/python3.11/site-packages/traitlets/traitlets.py#line=1567), in HasTraits._notify_observers(self, event)
1565 elif isinstance(c, EventHandler) and c.name is not None:
1566 c = getattr(self, c.name)
-> 1568 c(event)
File [~/Documents/Code/jdaviz/jdaviz/configs/default/plugins/model_fitting/model_fitting.py:324](http://localhost:8891/lab/tree/notebook/~/Documents/Code/jdaviz/jdaviz/configs/default/plugins/model_fitting/model_fitting.py#line=323), in ModelFitting._cube_fit_changed(self, event)
322 self._units['y'] = sb_unit
323 self.dataset.add_filter('is_flux_cube')
--> 324 self.dataset.remove_filter('layer_in_spectrum_viewer')
325 else:
326 self._units['y'] = spectral_y_unit
File [~/Documents/Code/jdaviz/jdaviz/core/template_mixin.py:906](http://localhost:8891/lab/tree/notebook/~/Documents/Code/jdaviz/jdaviz/core/template_mixin.py#line=905), in SelectPluginComponent.remove_filter(self, *filters)
905 def remove_filter(self, *filters):
--> 906 self.filters = [f for f in self.filters
907 if (f not in filters and getattr(f, '__name__', '') not in filters)]
File [~/Documents/Code/jdaviz/jdaviz/core/template_mixin.py:685](http://localhost:8891/lab/tree/notebook/~/Documents/Code/jdaviz/jdaviz/core/template_mixin.py#line=684), in BasePluginComponent.__setattr__(self, attr, value, force_super)
683 def __setattr__(self, attr, value, force_super=False):
684 if attr[0] == '_' or force_super or attr not in self._plugin_traitlets.keys():
--> 685 return super().__setattr__(attr, value)
687 return setattr(self._plugin, self._plugin_traitlets.get(attr), value)
File [~/Documents/Code/strauss_jdat/vslim/lib/python3.11/site-packages/traitlets/traitlets.py:716](http://localhost:8891/lab/tree/notebook/~/Documents/Code/strauss_jdat/vslim/lib/python3.11/site-packages/traitlets/traitlets.py#line=715), in TraitType.__set__(self, obj, value)
714 if self.read_only:
715 raise TraitError('The "%s" trait is read-only.' % self.name)
--> 716 self.set(obj, value)
File [~/Documents/Code/strauss_jdat/vslim/lib/python3.11/site-packages/traitlets/traitlets.py:3635](http://localhost:8891/lab/tree/notebook/~/Documents/Code/strauss_jdat/vslim/lib/python3.11/site-packages/traitlets/traitlets.py#line=3634), in List.set(self, obj, value)
3633 return super().set(obj, [value]) # type:ignore[list-item]
3634 else:
-> 3635 return super().set(obj, value)
File [~/Documents/Code/strauss_jdat/vslim/lib/python3.11/site-packages/traitlets/traitlets.py:706](http://localhost:8891/lab/tree/notebook/~/Documents/Code/strauss_jdat/vslim/lib/python3.11/site-packages/traitlets/traitlets.py#line=705), in TraitType.set(self, obj, value)
702 silent = False
703 if silent is not True:
704 # we explicitly compare silent to True just in case the equality
705 # comparison above returns something other than True[/False](http://localhost:8891/False)
--> 706 obj._notify_trait(self.name, old_value, new_value)
File [~/Documents/Code/strauss_jdat/vslim/lib/python3.11/site-packages/traitlets/traitlets.py:1513](http://localhost:8891/lab/tree/notebook/~/Documents/Code/strauss_jdat/vslim/lib/python3.11/site-packages/traitlets/traitlets.py#line=1512), in HasTraits._notify_trait(self, name, old_value, new_value)
1512 def _notify_trait(self, name: str, old_value: t.Any, new_value: t.Any) -> None:
-> 1513 self.notify_change(
1514 Bunch(
1515 name=name,
1516 old=old_value,
1517 new=new_value,
1518 owner=self,
1519 type="change",
1520 )
1521 )
File [~/Documents/Code/strauss_jdat/vslim/lib/python3.11/site-packages/traitlets/traitlets.py:1525](http://localhost:8891/lab/tree/notebook/~/Documents/Code/strauss_jdat/vslim/lib/python3.11/site-packages/traitlets/traitlets.py#line=1524), in HasTraits.notify_change(self, change)
1523 def notify_change(self, change: Bunch) -> None:
1524 """Notify observers of a change event"""
-> 1525 return self._notify_observers(change)
File [~/Documents/Code/strauss_jdat/vslim/lib/python3.11/site-packages/traitlets/traitlets.py:1568](http://localhost:8891/lab/tree/notebook/~/Documents/Code/strauss_jdat/vslim/lib/python3.11/site-packages/traitlets/traitlets.py#line=1567), in HasTraits._notify_observers(self, event)
1565 elif isinstance(c, EventHandler) and c.name is not None:
1566 c = getattr(self, c.name)
-> 1568 c(event)
File [~/Documents/Code/jdaviz/jdaviz/core/template_mixin.py:3671](http://localhost:8891/lab/tree/notebook/~/Documents/Code/jdaviz/jdaviz/core/template_mixin.py#line=3670), in DatasetSelect._on_data_changed(self, msg)
3668 manual_items = [{'label': label} for label in self.manual_options]
3669 self.items = manual_items + [_dc_to_dict(data) for data in self.app.data_collection
3670 if self._is_valid_item(data)]
-> 3671 self._apply_default_selection()
3672 # future improvement: only clear cache if the selected data entry was changed?
3673 self._clear_cache(*self._cached_properties)
File [~/Documents/Code/jdaviz/jdaviz/core/template_mixin.py:984](http://localhost:8891/lab/tree/notebook/~/Documents/Code/jdaviz/jdaviz/core/template_mixin.py#line=983), in SelectPluginComponent._apply_default_selection(self, skip_if_current_valid)
982 default_empty = [] if self.is_multiselect else ''
983 if self.default_mode == 'first':
--> 984 self.selected = self.labels[0] if len(self.labels) else default_empty
985 elif self.default_mode == 'default_text':
986 self.selected = self._default_text if self._default_text else default_empty
File [~/Documents/Code/jdaviz/jdaviz/core/template_mixin.py:687](http://localhost:8891/lab/tree/notebook/~/Documents/Code/jdaviz/jdaviz/core/template_mixin.py#line=686), in BasePluginComponent.__setattr__(self, attr, value, force_super)
684 if attr[0] == '_' or force_super or attr not in self._plugin_traitlets.keys():
685 return super().__setattr__(attr, value)
--> 687 return setattr(self._plugin, self._plugin_traitlets.get(attr), value)
File [~/Documents/Code/strauss_jdat/vslim/lib/python3.11/site-packages/traitlets/traitlets.py:716](http://localhost:8891/lab/tree/notebook/~/Documents/Code/strauss_jdat/vslim/lib/python3.11/site-packages/traitlets/traitlets.py#line=715), in TraitType.__set__(self, obj, value)
714 if self.read_only:
715 raise TraitError('The "%s" trait is read-only.' % self.name)
--> 716 self.set(obj, value)
File [~/Documents/Code/strauss_jdat/vslim/lib/python3.11/site-packages/traitlets/traitlets.py:706](http://localhost:8891/lab/tree/notebook/~/Documents/Code/strauss_jdat/vslim/lib/python3.11/site-packages/traitlets/traitlets.py#line=705), in TraitType.set(self, obj, value)
702 silent = False
703 if silent is not True:
704 # we explicitly compare silent to True just in case the equality
705 # comparison above returns something other than True[/False](http://localhost:8891/False)
--> 706 obj._notify_trait(self.name, old_value, new_value)
File [~/Documents/Code/strauss_jdat/vslim/lib/python3.11/site-packages/traitlets/traitlets.py:1513](http://localhost:8891/lab/tree/notebook/~/Documents/Code/strauss_jdat/vslim/lib/python3.11/site-packages/traitlets/traitlets.py#line=1512), in HasTraits._notify_trait(self, name, old_value, new_value)
1512 def _notify_trait(self, name: str, old_value: t.Any, new_value: t.Any) -> None:
-> 1513 self.notify_change(
1514 Bunch(
1515 name=name,
1516 old=old_value,
1517 new=new_value,
1518 owner=self,
1519 type="change",
1520 )
1521 )
File [~/Documents/Code/strauss_jdat/vslim/lib/python3.11/site-packages/ipywidgets/widgets/widget.py:701](http://localhost:8891/lab/tree/notebook/~/Documents/Code/strauss_jdat/vslim/lib/python3.11/site-packages/ipywidgets/widgets/widget.py#line=700), in Widget.notify_change(self, change)
698 if name in self.keys and self._should_send_property(name, getattr(self, name)):
699 # Send new state to front-end
700 self.send_state(key=name)
--> 701 super().notify_change(change)
File [~/Documents/Code/strauss_jdat/vslim/lib/python3.11/site-packages/traitlets/traitlets.py:1525](http://localhost:8891/lab/tree/notebook/~/Documents/Code/strauss_jdat/vslim/lib/python3.11/site-packages/traitlets/traitlets.py#line=1524), in HasTraits.notify_change(self, change)
1523 def notify_change(self, change: Bunch) -> None:
1524 """Notify observers of a change event"""
-> 1525 return self._notify_observers(change)
File [~/Documents/Code/strauss_jdat/vslim/lib/python3.11/site-packages/traitlets/traitlets.py:1568](http://localhost:8891/lab/tree/notebook/~/Documents/Code/strauss_jdat/vslim/lib/python3.11/site-packages/traitlets/traitlets.py#line=1567), in HasTraits._notify_observers(self, event)
1565 elif isinstance(c, EventHandler) and c.name is not None:
1566 c = getattr(self, c.name)
-> 1568 c(event)
File [~/Documents/Code/jdaviz/jdaviz/core/template_mixin.py:2940](http://localhost:8891/lab/tree/notebook/~/Documents/Code/jdaviz/jdaviz/core/template_mixin.py#line=2939), in NonFiniteUncertaintyMismatchMixin._check_non_finite_uncertainty_mismatch(self, event)
2936 if not hasattr(self, 'dataset') or self.dataset_selected == '':
2937 # during initial init, this can trigger before the component is initialized
2938 return
-> 2940 spec = self.dataset.selected_spectrum
2942 if spec.uncertainty is None:
2943 self.non_finite_uncertainty_mismatch = False
File [/opt/homebrew/Cellar/python](http://localhost:8891/opt/homebrew/Cellar/python)@3.11[/3.11.9/Frameworks/Python.framework/Versions/3.11/lib/python3.11/functools.py:1001](http://localhost:8891/3.11.9/Frameworks/Python.framework/Versions/3.11/lib/python3.11/functools.py#line=1000), in cached_property.__get__(self, instance, owner)
999 val = cache.get(self.attrname, _NOT_FOUND)
1000 if val is _NOT_FOUND:
-> 1001 val = self.func(instance)
1002 try:
1003 cache[self.attrname] = val
File [~/Documents/Code/jdaviz/jdaviz/core/template_mixin.py:3551](http://localhost:8891/lab/tree/notebook/~/Documents/Code/jdaviz/jdaviz/core/template_mixin.py#line=3550), in DatasetSelect.selected_spectrum(self)
3549 @cached_property
3550 def selected_spectrum(self):
-> 3551 return self.get_selected_spectrum(use_display_units=True)
File [~/Documents/Code/jdaviz/jdaviz/core/template_mixin.py:3545](http://localhost:8891/lab/tree/notebook/~/Documents/Code/jdaviz/jdaviz/core/template_mixin.py#line=3544), in DatasetSelect.get_selected_spectrum(self, use_display_units)
3541 spec_extract = self.app._jdaviz_helper.plugins['Spectral Extraction']._obj
3542 sp = spec_extract._extract_in_new_instance(self.selected,
3543 function=self._spectral_extraction_function,
3544 add_data=False)
-> 3545 return self.plugin._specviz_helper._handle_display_units(sp, use_display_units)
3546 return self.plugin._specviz_helper.get_data(data_label=self.selected,
3547 use_display_units=use_display_units)
File [~/Documents/Code/jdaviz/jdaviz/core/helpers.py:479](http://localhost:8891/lab/tree/notebook/~/Documents/Code/jdaviz/jdaviz/core/helpers.py#line=478), in ConfigHelper._handle_display_units(self, data, use_display_units)
477 new_uncert = uncertainty
478 if ('_pixel_scale_factor' in data.meta):
--> 479 new_uncert_converted = flux_conversion(new_uncert.quantity.value,
480 new_uncert.unit, y_unit, spec=data)
481 new_uncert = StdDevUncertainty(new_uncert_converted, unit=y_unit)
482 else:
File [~/Documents/Code/jdaviz/jdaviz/utils.py:442](http://localhost:8891/lab/tree/notebook/~/Documents/Code/jdaviz/jdaviz/utils.py#line=441), in flux_conversion(values, original_units, target_units, spec, eqv, slice)
434 eqv += _eqv_flux_to_sb_pixel()
436 # when angle<>pixel translations are enabled
437 # eqv += _eqv_sb_per_pixel_to_per_angle(u.Jy)
438
439 # indirect units cannot be directly converted, and require
440 # additional conversions to reach the desired end unit.
441 # if spec_unit in [original_units, target_units]:
--> 442 result = _indirect_conversion(
443 values=values, orig_units=orig_units, targ_units=targ_units,
444 eqv=eqv, spec_unit=spec_unit
445 )
447 if result and len(result) == 2:
448 values, updated_units = result
File [~/Documents/Code/jdaviz/jdaviz/utils.py:512](http://localhost:8891/lab/tree/notebook/~/Documents/Code/jdaviz/jdaviz/utils.py#line=511), in _indirect_conversion(values, orig_units, targ_units, eqv, spec_unit, image_data)
508 if ((u.Unit(targ_units) in indirect_units()) or
509 (u.Unit(orig_units) in indirect_units())):
510 # SB -> Flux -> Flux -> SB
511 temp_orig = orig_units * solid_angle_in_orig
--> 512 temp_targ = targ_units * solid_angle_in_targ
514 # Convert Surface Brightness to Flux, then Flux to Flux
515 values = (values * orig_units).to_value(temp_orig, equivalencies=eqv)
TypeError: unsupported operand type(s) for *: 'CompositeUnit' and 'NoneType'
Expected behavior
Should run error free and produce appropriate fit
Browser
Chrome
Jupyter
Selected Jupyter core packages...
IPython : 8.26.0
ipykernel : 6.29.5
ipywidgets : 8.1.3
jupyter_client : 8.6.2
jupyter_core : 5.7.2
jupyter_server : 2.14.1
jupyterlab : 4.2.3
nbclient : 0.7.4
nbconvert : 7.16.4
nbformat : 5.10.4
notebook : 7.2.1
qtconsole : not installed
traitlets : 5.14.3
Software versions
Versions:
macOS-14.2-arm64-arm-64bit
Python 3.11.9 (main, Apr 2 2024, 08:25:04) [Clang 15.0.0 (clang-1500.3.9.4)]
Numpy 2.0.0
astropy 6.1.1
matplotlib 3.8.4
scipy 1.14.0
scikit-image 0.24.0
asdf 3.2.0
stdatamodels 2.0.0
gwcs 0.21.0
regions 0.9
specutils 1.19.0
specreduce 1.4.1
photutils 1.13.0
astroquery 0.4.7
pyyaml 6.0.1
asteval 1.0.0
idna 3.7
traitlets 5.14.3
bqplot 0.12.43
bqplot-image-gl 1.5.0
glue-core 1.21.0
glue-jupyter 0.23.1
glue-astronomy 0.10.0
echo 0.8.0
ipyvue 1.11.1
ipyvuetify 1.9.4
ipysplitpanes 0.2.0
ipygoldenlayout 0.4.0
ipypopout 2.0.0
Jinja2 3.1.4
solara 1.41.0
vispy 0.14.3
sidecar 0.7.0
Jdaviz 0.1.dev5434+g1408806
RE optical IFU surveys, I've alsohad automatic spectral extraction errors for SAMI (e.g. file) and CALIFA (e.g. file) cubes. Was there any plan to support these?
are you on jdaviz main or the released version? https://github.com/spacetelescope/jdaviz/pull/3307 should fix this on main, but from the description there it wasn't expected to be a problem in 4.0 so we didn't issue a bugfix. cc @rosteen
@kecnry this is on current main - note I can now open MaNGA cubes with automatic spectral extraction, the errror above is when you try and run the model fitting. It's SAMI and CALIFA that have the same problem as the original MaNGA issue in https://github.com/spacetelescope/jdaviz/pull/3307
What is the cube data unit?
It is triggered here but somehow solid_angle_in_targ is None.
https://github.com/spacetelescope/jdaviz/blob/527d1f5e0e5ecff9585957ed153bd28963a51e51/jdaviz/utils.py#L512
@pllim cube BUNIT is '1E-17 erg/s/cm^2/Angstrom/spaxel' (see https://github.com/spacetelescope/jdaviz/pull/3307#issuecomment-2486512154)
RE solid_angle_in_targ - noticed this - I also noticed this conditional above: https://github.com/spacetelescope/jdaviz/blob/527d1f5e0e5ecff9585957ed153bd28963a51e51/jdaviz/utils.py#L506-L507
which should catch this case but couldn't work out why - perhaps a clue?
which should catch this case
L507 acknowledges that solid_angle_in_targ could be None but still tries to stuff it into a unit on L512, which does not make sense to me. Perhaps a bug of not handling this edge case properly?
I coincidentally noticed this bug sometime last week and currently have a PR that is currently in review for the exact lines mentioned in utils.py. I tested it on a few manga cubes including the one provided here and this PR should do the trick.
