Possible bug in to_table() method

[gnthibault]

Dear all,

I am still playing with astroplan for scheduling observing run. However, I recently came across a problem, that I cannot easily reproduce, so I decided to share a pickled version of my schedule object:

schedule.txt

Here is the code that allows to generate the error

import pickle
f = open('./schedule.txt','rb')
o = pickle.load(f) 
o.to_table()

The error message is the following:

---

UnitConversionError Traceback (most recent call last) ~/anaconda3/lib/python3.6/site-packages/astropy/units/quantity.py in float(self) 1200 try: -> 1201 return float(self.to_value(dimensionless_unscaled)) 1202 except (UnitsError, TypeError):

~/anaconda3/lib/python3.6/site-packages/astropy/units/quantity.py in to_value(self, unit, equivalencies) 877 if unit != self.unit: --> 878 value = self._to_value(unit, equivalencies) 879 return value if self.shape else value.item()

~/anaconda3/lib/python3.6/site-packages/astropy/units/quantity.py in _to_value(self, unit, equivalencies) 815 return self.unit.to(unit, self.view(np.ndarray), --> 816 equivalencies=equivalencies) 817

~/anaconda3/lib/python3.6/site-packages/astropy/units/core.py in to(self, other, value, equivalencies) 978 """ --> 979 return self._get_converter(other, equivalencies=equivalencies)(value) 980

~/anaconda3/lib/python3.6/site-packages/astropy/units/core.py in _get_converter(self, other, equivalencies) 912 --> 913 raise exc 914

~/anaconda3/lib/python3.6/site-packages/astropy/units/core.py in _get_converter(self, other, equivalencies) 898 return self._apply_equivalencies( --> 899 self, other, self._normalize_equivalencies(equivalencies)) 900 except UnitsError as exc:

~/anaconda3/lib/python3.6/site-packages/astropy/units/core.py in _apply_equivalencies(self, unit, other, equivalencies) 882 "{0} and {1} are not convertible".format( --> 883 unit_str, other_str)) 884

UnitConversionError: 'deg' (angle) and '' (dimensionless) are not convertible

During handling of the above exception, another exception occurred:

TypeError Traceback (most recent call last) in () ----> 1 o.to_table()

~/anaconda3/lib/python3.6/site-packages/astroplan/scheduling.py in to_table(self, show_transitions, show_unused) 302 return Table([target_names, start_times, end_times, durations, ra, dec, config], 303 names=('target', 'start time (UTC)', 'end time (UTC)', --> 304 'duration (minutes)', 'ra', 'dec', 'configuration')) 305 306 def new_slots(self, slot_index, start_time, end_time):

~/anaconda3/lib/python3.6/site-packages/astropy/table/table.py in init(self, data, masked, names, dtype, meta, copy, rows, copy_indices, **kwargs) 409 410 # Finally do the real initialization --> 411 init_func(data, names, dtype, n_cols, copy) 412 413 # Whatever happens above, the masked property should be set to a boolean

~/anaconda3/lib/python3.6/site-packages/astropy/table/table.py in _init_from_list(self, data, names, dtype, n_cols, copy) 675 elif isinstance(col, np.ndarray) or isiterable(col): 676 col = self.ColumnClass(name=(name or def_name), data=col, dtype=dtype, --> 677 copy=copy, copy_indices=self._init_indices) 678 else: 679 raise ValueError('Elements in list initialization must be '

~/anaconda3/lib/python3.6/site-packages/astropy/table/column.py in new(cls, data, name, dtype, shape, length, description, unit, format, meta, copy, copy_indices) 805 shape=shape, length=length, description=description, 806 unit=unit, format=format, meta=meta, --> 807 copy=copy, copy_indices=copy_indices) 808 return self 809

~/anaconda3/lib/python3.6/site-packages/astropy/table/column.py in new(cls, data, name, dtype, shape, length, description, unit, format, meta, copy, copy_indices) 228 if not six.PY2 and np.dtype(dtype).char == 'S': 229 data = cls._encode_str(data) --> 230 self_data = np.array(data, dtype=dtype, copy=copy) 231 232 self = self_data.view(cls)

~/anaconda3/lib/python3.6/site-packages/astropy/units/quantity.py in float(self) 1201 return float(self.to_value(dimensionless_unscaled)) 1202 except (UnitsError, TypeError): -> 1203 raise TypeError('only dimensionless scalar quantities can be ' 1204 'converted to Python scalars')

Do you have an idea of a workaround for this ? Thank you in advance for your help !

[bsipocz]

@gnthibault - Sharing pickes are usually not the safest way around, they are best when just passing between pipeline elements in the same environment. Have you tried to convert the schedule to a Table before picking, and save it to disk then?

[gnthibault]

I pickled the schedule only for sharing it, because I was not able to provide a simple script that exhibit the proble (MWE). Otherwise I never use Pickle inside my pipeline. It looks like this bug is target dependant.

[bsipocz]

OK, then I rephrase the question. I suspect you see the issue when you skip the pickling, and try to convert your schedule straight to a table? If yes, could you provide a small example to reproduce?

[jselsing]

I just wanted to note, that I encountered this as well. I tracked the issue to the automatic dtype inference by the Table initiation when there are no TransitionBlock. I fixed it by specifying all dtypes as strings.

    return Table([target_names, start_times, end_times, durations, ra, dec, config],
                 names=('target', 'start time (UTC)', 'end time (UTC)',
                        'duration (minutes)', 'ra', 'dec', 'configuration'),
                 dtype=[str, str, str, str, str, str, str])

"""

I realise this is maybe a bit hacky solution, so let's explore alternative fixes.

[jselsing]

Also, this suggestion does not seem to be fool-proof as now other things break.

[bsipocz]

@jselsing - could you provide a full example? That would help enormously to reproduce it and come up with a permanent fix for it.

Alternatively if you're interested contributing and have a fix please open a PR. (Casting everything as a string is not a preferred way, I suppose we need something along the line of adding a conditional before the return)

[jselsing]

Yea, I figured.

My astroplan usage is part of a larger piece of code, and not straightforward to produce a MWE. It only fails in small subset of calls - maybe one where there is no TransitionBlock. I will try and take look at it, Monday!

[bmorris3]

Closing as stale. Please reopen if necessary!