databroker icon indicating copy to clipboard operation
databroker copied to clipboard

feat: serialize named tuples as dicts

Open maffettone opened this issue 10 months ago • 2 comments

Take advantage of dependent PR in Tiled to serialize NamedTuple like objects as dictionaries.

Description

Add function to be consumed as default serialization.

Motivation and Context

This default functionality was removed from orjson in version 3.10, and causes problems with Bluesky use (e.g., hklpy).

How Has This Been Tested?

maffettone avatar Feb 20 '25 13:02 maffettone

Tagging @ambarb for visibility.

maffettone avatar Feb 20 '25 13:02 maffettone

I appreciate the effort to isolate and target this fix. Unfortunately, as we look toward migrating the facility off of mongo_normalized during 2025, this fix would be short-lived.

At the moment, databroker registers a custom endpoint with the Tiled FastAPI Router, POST /documents/{path}, which you are customizing here, and blueksy_tiled_plugins.CatalogOfBlueskyRuns.post_document calls it. But in the future (and already at HEX) the bluesky.callbacks.tiled_writer.TiledWriter will be used, and it will send metadata from the start document at the vanilla POST /metadata endpoint.

Other options:

  • Make Tiled tolerate namedtuple and perhaps other by default. (I don't love this. I think orjson is right that there is no unambigous JSON serialization of namedtuple.)
  • Merge this fix, and add tolerant JSON serialization into TiledWriter as well. Tiled would need a way to expose this in tiled.client.container.Container.new().
  • In the Device, use a serializable type.

danielballan avatar Feb 21 '25 18:02 danielballan

Coming at this a little later than I'd like. On a deeper look into tiled.client.container.Container I think option 3 is the right approach. Anything else would have to change the signature of a new and a number of convenience methods that pass through to new. If we are willing to accept the standard that orjson provides as correct (named tuple is ambiguous and one should be explicit for serialization), we should leave that up to the client and device. In this case hklpy.

In other words, fewer changes would be required to hklpy than to a concerted effort of tiled, bluesky, and databroker simultaneously, so that seems like the correct route forward.

maffettone avatar Mar 31 '25 13:03 maffettone

I'm comfortable closing this and the Tiled PR if in agreement.

maffettone avatar Mar 31 '25 13:03 maffettone