databroker icon indicating copy to clipboard operation
databroker copied to clipboard

mongo_normalized fails to read a 0 length event stream

Open tacaswell opened this issue 1 year ago • 0 comments

Expected Behavior

Current Behavior

MongoAdapter._build_event_stream(self, run_start_uid, stream_name, is_complete)
   1466 # We need each of the sub-dicts to have a consistent length. If
   1467 # Events are still being added, we need to choose a consistent
   1468 # cutoff. If not, we need to know the length anyway. Note that this
   1469 # is not the same thing as the number of Event documents in the
   1470 # stream because seq_num may be repeated, nonunique.
   1471 cursor = self._event_collection.aggregate(
   1472     [
   1473         {"$match": {"descriptor": {"$in": event_descriptor_uids}}},
   (...)
   1480     ]
   1481 )
-> 1482 (result,) = cursor
   1483 cutoff_seq_num = (
   1484     1 + result["highest_seq_num"]
   1485 )  # `1 +` because we use a half-open interval
   1486 object_names = event_descriptors[0]["object_keys"]

ValueError: not enough values to unpack (expected 1, got 0)

Possible Solution

We need to deal with the possibility that the cursor that comes out of https://github.com/bluesky/databroker/blob/c41645019ac63fb2237beef9a69b628516c2fca5/databroker/mongo_normalized.py#L1471-L1482 may have 0 or 1 values in it (rather than always exactly 1 value).

There are a couple other such unpackings that may need to be fixed as well.

Steps to Reproduce (for bugs)

  1. try create the header object for a run that contains an empty stream (have a descriptor, but no events)

Context

had to change from a comprehension to a for loop so I could put a try/except in, but in general this is fatal to any process that wants to look at the meta data of runs that may have failed or been cancelled mid-collection between the descriptor going out and the first event going out.

The ability to pre-declare streams being proposed in bluesky will make it much easier to end up in this situation.

Your Environment

in the 2022-3.0-py39-tiled environment

$ conda list | grep databroker
databroker                2.0.0b8            pyhd8ed1ab_1    conda-forge
$ conda list | grep tiled
# packages in environment at /nsls2/conda/envs/2022-3.0-py39-tiled:
tiled                     0.1.0a70             hd8ed1ab_0    conda-forge
tiled-array               0.1.0a70             hd8ed1ab_0    conda-forge
tiled-base                0.1.0a70         py39hf3d152e_0    conda-forge
tiled-client              0.1.0a70             hd8ed1ab_0    conda-forge
tiled-compression         0.1.0a70             hd8ed1ab_0    conda-forge
tiled-dataframe           0.1.0a70             hd8ed1ab_0    conda-forge
tiled-formats             0.1.0a70             hd8ed1ab_0    conda-forge
tiled-minimal-base        0.1.0a70             hd8ed1ab_0    conda-forge
tiled-server              0.1.0a70         py39hf3d152e_0    conda-forge
tiled-sparse              0.1.0a70             hd8ed1ab_0    conda-forge
tiled-xarray              0.1.0a70             hd8ed1ab_0    conda-forge

tacaswell avatar Aug 30 '22 18:08 tacaswell