error when running results_df.describe() in 03_motion.ipynb

[caseylam]

Thanks for making these tutorials! I am learning how to analyze Gaia data and use ADQL, and the notebooks have been very helpful.

Running the 03_motion.ipynb notebook through colab, when I try to run the line results_df.describe() it would return the error UnitConversionError: '' (dimensionless) and 'deg' (angle) are not convertible (full stack trace below).

This problem was fixed by adding .value to the skycoord_gd1 objects when adding them to results_df in the preceding lines:

results_df['phi1'] = skycoord_gd1.phi1.value
results_df['phi2'] = skycoord_gd1.phi2.value
results_df.shape

results_df['pm_phi1'] = skycoord_gd1.pm_phi1_cosphi2.value
results_df['pm_phi2'] = skycoord_gd1.pm_phi2.value
results_df.shape

It might be that astropy and/or pandas changed something about how they did their units? After making this change, everything seems to run fine.

Full stacktrace from the error described above:

AttributeError                            Traceback (most recent call last)
/usr/local/lib/python3.10/dist-packages/astropy/units/quantity.py in _to_own_unit(self, value, check_precision, unit)
   1630         try:
-> 1631             _value = value.to_value(unit)
   1632         except AttributeError:

AttributeError: 'numpy.ndarray' object has no attribute 'to_value'

During handling of the above exception, another exception occurred:

UnitConversionError                       Traceback (most recent call last)
27 frames
/usr/local/lib/python3.10/dist-packages/astropy/units/quantity.py in to_value(self, unit, equivalencies)
    958             try:
--> 959                 scale = self.unit._to(unit)
    960             except Exception:

/usr/local/lib/python3.10/dist-packages/astropy/units/core.py in _to(self, other)
   1128
-> 1129         raise UnitConversionError(f"'{self!r}' is not a scaled version of '{other!r}'")
   1130

UnitConversionError: 'Unit(dimensionless)' is not a scaled version of 'Unit("deg")'

During handling of the above exception, another exception occurred:

UnitConversionError                       Traceback (most recent call last)
<ipython-input-27-3e0a046dcd12> in <cell line: 1>()
----> 1 results_df.describe()

/usr/local/lib/python3.10/dist-packages/pandas/core/generic.py in describe(self, percentiles, include, exclude, datetime_is_numeric)
  10938         max            NaN      3.0
  10939         """
> 10940         return describe_ndframe(
  10941             obj=self,
  10942             include=include,

/usr/local/lib/python3.10/dist-packages/pandas/core/describe.py in describe_ndframe(obj, include, exclude, datetime_is_numeric, percentiles)
     99         )
    100
--> 101     result = describer.describe(percentiles=percentiles)
    102     return cast(NDFrameT, result)
    103

/usr/local/lib/python3.10/dist-packages/pandas/core/describe.py in describe(self, percentiles)
    174
    175     def describe(self, percentiles: Sequence[float] | np.ndarray) -> DataFrame:
--> 176         data = self._select_data()
    177
    178         ldesc: list[Series] = []

/usr/local/lib/python3.10/dist-packages/pandas/core/describe.py in _select_data(self)
    197             if self.datetime_is_numeric:
    198                 default_include.append("datetime")
--> 199             data = self.obj.select_dtypes(include=default_include)
    200             if len(data.columns) == 0:
    201                 data = self.obj

/usr/local/lib/python3.10/dist-packages/pandas/core/frame.py in select_dtypes(self, include, exclude)
   4751             return True
   4752
-> 4753         mgr = self._mgr._get_data_subset(predicate).copy(deep=None)
   4754         return type(self)(mgr).__finalize__(self)
   4755

/usr/local/lib/python3.10/dist-packages/pandas/core/internals/managers.py in copy(self, deep)
    668
    669         if deep:
--> 670             res._consolidate_inplace()
    671         return res
    672

/usr/local/lib/python3.10/dist-packages/pandas/core/internals/managers.py in _consolidate_inplace(self)
   1869         if not self.is_consolidated():
   1870             if self.refs is None:
-> 1871                 self.blocks = _consolidate(self.blocks)
   1872             else:
   1873                 self.blocks, self.refs = _consolidate_with_refs(self.blocks, self.refs)

/usr/local/lib/python3.10/dist-packages/pandas/core/internals/managers.py in _consolidate(blocks)
   2327     new_blocks: list[Block] = []
   2328     for (_can_consolidate, dtype), group_blocks in grouper:
-> 2329         merged_blocks, _ = _merge_blocks(
   2330             list(group_blocks), dtype=dtype, can_consolidate=_can_consolidate
   2331         )

/usr/local/lib/python3.10/dist-packages/pandas/core/internals/managers.py in _merge_blocks(blocks, dtype, can_consolidate)
   2379             # Sequence[Union[int, float, complex, str, bytes, generic]],
   2380             # Sequence[Sequence[Any]], SupportsArray]]
-> 2381             new_values = np.vstack([b.values for b in blocks])  # type: ignore[misc]
   2382         else:
   2383             bvals = [blk.values for blk in blocks]

/usr/local/lib/python3.10/dist-packages/numpy/core/overrides.py in vstack(*args, **kwargs)

/usr/local/lib/python3.10/dist-packages/astropy/units/quantity.py in __array_function__(self, function, types, args, kwargs)
   1821         # implementation.
   1822         if function in SUBCLASS_SAFE_FUNCTIONS:
-> 1823             return super().__array_function__(function, types, args, kwargs)
   1824
   1825         elif function in FUNCTION_HELPERS:

/usr/local/lib/python3.10/dist-packages/numpy/core/shape_base.py in vstack(tup)
    280     if not isinstance(arrs, list):
    281         arrs = [arrs]
--> 282     return _nx.concatenate(arrs, 0)
    283
    284

/usr/local/lib/python3.10/dist-packages/numpy/core/overrides.py in concatenate(*args, **kwargs)

/usr/local/lib/python3.10/dist-packages/astropy/units/quantity.py in __array_function__(self, function, types, args, kwargs)
   1826             function_helper = FUNCTION_HELPERS[function]
   1827             try:
-> 1828                 args, kwargs, unit, out = function_helper(*args, **kwargs)
   1829             except NotImplementedError:
   1830                 return self._not_implemented_or_raise(function, types)

/usr/local/lib/python3.10/dist-packages/astropy/units/quantity_helper/function_helpers.py in concatenate(arrays, axis, out, **kwargs)
    377     # TODO: make this smarter by creating an appropriately shaped
    378     # empty output array and just filling it.
--> 379     arrays, kwargs, unit, out = _iterable_helper(*arrays, out=out, axis=axis, **kwargs)
    380     return (arrays,), kwargs, unit, out
    381

/usr/local/lib/python3.10/dist-packages/astropy/units/quantity_helper/function_helpers.py in _iterable_helper(out, *args, **kwargs)
    369             raise NotImplementedError
    370
--> 371     arrays, unit = _quantities2arrays(*args)
    372     return arrays, kwargs, unit, out
    373

/usr/local/lib/python3.10/dist-packages/astropy/units/quantity_helper/function_helpers.py in _quantities2arrays(unit_from_first, *args)
    350     # as we want to allow arbitrary unit for 0, inf, and nan.
    351     try:
--> 352         arrays = tuple((q._to_own_unit(arg)) for arg in args)
    353     except TypeError:
    354         raise NotImplementedError

/usr/local/lib/python3.10/dist-packages/astropy/units/quantity_helper/function_helpers.py in <genexpr>(.0)
    350     # as we want to allow arbitrary unit for 0, inf, and nan.
    351     try:
--> 352         arrays = tuple((q._to_own_unit(arg)) for arg in args)
    353     except TypeError:
    354         raise NotImplementedError

/usr/local/lib/python3.10/dist-packages/astropy/units/quantity.py in _to_own_unit(self, value, check_precision, unit)
   1645             try:
   1646                 as_quantity = Quantity(value)
-> 1647                 _value = as_quantity.to_value(unit)
   1648             except UnitsError:
   1649                 # last chance: if this was not something with a unit

/usr/local/lib/python3.10/dist-packages/astropy/units/quantity.py in to_value(self, unit, equivalencies)
    960             except Exception:
    961                 # Short-cut failed; try default (maybe equivalencies help).
--> 962                 value = self._to_value(unit, equivalencies)
    963             else:
    964                 value = self.view(np.ndarray)

/usr/local/lib/python3.10/dist-packages/astropy/units/quantity.py in _to_value(self, unit, equivalencies)
    866         if not self.dtype.names or isinstance(self.unit, StructuredUnit):
    867             # Standard path, let unit to do work.
--> 868             return self.unit.to(
    869                 unit, self.view(np.ndarray), equivalencies=equivalencies
    870             )

/usr/local/lib/python3.10/dist-packages/astropy/units/core.py in to(self, other, value, equivalencies)
   1163             return UNITY
   1164         else:
-> 1165             return self._get_converter(Unit(other), equivalencies)(value)
   1166
   1167     def in_units(self, other, value=1.0, equivalencies=[]):

/usr/local/lib/python3.10/dist-packages/astropy/units/core.py in _get_converter(self, other, equivalencies)
   1092                             return lambda v: b(converter(v))
   1093
-> 1094             raise exc
   1095
   1096     def _to(self, other):

/usr/local/lib/python3.10/dist-packages/astropy/units/core.py in _get_converter(self, other, equivalencies)
   1075         # if that doesn't work, maybe we can do it with equivalencies?
   1076         try:
-> 1077             return self._apply_equivalencies(
   1078                 self, other, self._normalize_equivalencies(equivalencies)
   1079             )

/usr/local/lib/python3.10/dist-packages/astropy/units/core.py in _apply_equivalencies(self, unit, other, equivalencies)
   1052         other_str = get_err_str(other)
   1053
-> 1054         raise UnitConversionError(f"{unit_str} and {other_str} are not convertible")
   1055
   1056     def _get_converter(self, other, equivalencies=[]):

UnitConversionError: '' (dimensionless) and 'deg' (angle) are not convertible

[AllenDowney]

Thanks for letting me know about these issues. I will look into them as soon as I can, but in the meantime, you might want to switch to the Foundations of Astronomical Data Science, a curriculum based on the material in this repo and maintained by The Carpentries:

https://datacarpentry.org/astronomy-python/