tiled icon indicating copy to clipboard operation
tiled copied to clipboard

Float datakey value gets converted to int when writing to postgres catalog

Open canismarko opened this issue 3 weeks ago • 1 comments

I have ophyd-async EpicsMotor devices that produce the following datakey in their configurations.

'sim_motor_2-user_offset':
    {'dtype': 'number',
     'shape': [],
     'dtype_numpy': '<f8',
     'source': 'ca://25idc:simMotor:m2.OFF',
     'units': 'degrees',
     'precision': 5,
     'limits': {'control': {'low': -1e+300, 'high': 1e+300},
      'display': {'low': -1e+300, 'high': 1e+300}}
    }

The 1e300 values comes from the IOC and is consistent across many motors. It seems a bit big for a limit, but it's a float so it should be fine. When writing this to a sqlite catalog, everything works as expected: the data key I get when reading the stream's metadata shows the limits as 1e300.

When I use this device with a postgres catalog, however, I get an exception in Tiled saying that an integer value is out of range. If I stick a print statement in before the data are sent to msgpack, I see that the 1e300 floats have been turned into integers and no longer fit into any c types.

'sim_motor_2-user_offset': {'dtype': 'number',
                            'dtype_numpy': '<f8',
                            'limits': {'control': {'high': 1000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000,
                                                   'low': -1000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000},
                                       'display': {'high': 1000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000,
                                                   'low': -1000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000}},
                            'precision': 5,
                            'shape': [],
                            'source': 'ca://25idc:simMotor:m2.OFF',
                            'units': 'degrees'},

Curiously, other float datakey limits do not get turned into ints. Like:

'sim_motor_2-velocity': {'dtype': 'number',
                         'dtype_numpy': '<f8',
                         'limits': {'control': {'high': 200.0,
                                                'low': 0.1},
                                    'display': {'high': 200.0,
                                                'low': 0.1}},
                         'precision': 5,
                         'shape': [],
                         'source': 'ca://25idc:simMotor:m2.VELO',
                         'units': 'degrees'}},

Maybe this only effects floats big enough to be serialized in scientific notation?

canismarko avatar Feb 06 '25 05:02 canismarko