databroker
databroker copied to clipboard
mongo_normalized fails to read a 0 length event stream
Expected Behavior
Current Behavior
MongoAdapter._build_event_stream(self, run_start_uid, stream_name, is_complete)
1466 # We need each of the sub-dicts to have a consistent length. If
1467 # Events are still being added, we need to choose a consistent
1468 # cutoff. If not, we need to know the length anyway. Note that this
1469 # is not the same thing as the number of Event documents in the
1470 # stream because seq_num may be repeated, nonunique.
1471 cursor = self._event_collection.aggregate(
1472 [
1473 {"$match": {"descriptor": {"$in": event_descriptor_uids}}},
(...)
1480 ]
1481 )
-> 1482 (result,) = cursor
1483 cutoff_seq_num = (
1484 1 + result["highest_seq_num"]
1485 ) # `1 +` because we use a half-open interval
1486 object_names = event_descriptors[0]["object_keys"]
ValueError: not enough values to unpack (expected 1, got 0)
Possible Solution
We need to deal with the possibility that the cursor that comes out of https://github.com/bluesky/databroker/blob/c41645019ac63fb2237beef9a69b628516c2fca5/databroker/mongo_normalized.py#L1471-L1482 may have 0 or 1 values in it (rather than always exactly 1 value).
There are a couple other such unpackings that may need to be fixed as well.
Steps to Reproduce (for bugs)
- try create the header object for a run that contains an empty stream (have a descriptor, but no events)
Context
had to change from a comprehension to a for loop so I could put a try/except in, but in general this is fatal to any process that wants to look at the meta data of runs that may have failed or been cancelled mid-collection between the descriptor going out and the first event going out.
The ability to pre-declare streams being proposed in bluesky will make it much easier to end up in this situation.
Your Environment
in the 2022-3.0-py39-tiled environment
$ conda list | grep databroker
databroker 2.0.0b8 pyhd8ed1ab_1 conda-forge
$ conda list | grep tiled
# packages in environment at /nsls2/conda/envs/2022-3.0-py39-tiled:
tiled 0.1.0a70 hd8ed1ab_0 conda-forge
tiled-array 0.1.0a70 hd8ed1ab_0 conda-forge
tiled-base 0.1.0a70 py39hf3d152e_0 conda-forge
tiled-client 0.1.0a70 hd8ed1ab_0 conda-forge
tiled-compression 0.1.0a70 hd8ed1ab_0 conda-forge
tiled-dataframe 0.1.0a70 hd8ed1ab_0 conda-forge
tiled-formats 0.1.0a70 hd8ed1ab_0 conda-forge
tiled-minimal-base 0.1.0a70 hd8ed1ab_0 conda-forge
tiled-server 0.1.0a70 py39hf3d152e_0 conda-forge
tiled-sparse 0.1.0a70 hd8ed1ab_0 conda-forge
tiled-xarray 0.1.0a70 hd8ed1ab_0 conda-forge