Dan Allan

icon indicating a website link http://www.danallan.com icon indicating an email [email protected]

icon company Brookhaven National Laboratory - NSLS-II icon location Baltimore icon location Data Engineering Group Lead and Scientific Software Developer at a giant X-ray beam @NSLS2 @bluesky (the science bluesky, not bluesky-social)

Results 757 comments of Dan Allan

Send xarrays with tiled

This was superseded by work upstream in Tiled core.

Handle case where there is a 'time' data key.

[Copied from Slack] This is the place to start: https://github.com/bluesky/databroker/blob/a2a5d6600ecf75efdd1c70cb67a0ebe2a1584bbd/databroker/mongo_normalized.py#L87-L113

Update indexes to use tie-break ObjectId

The ObjectId is the document creation time, which _may_ be close to `start_doc.time` but may not. Consider the case where a MongoDB was populated by databroker-unpack from msgpack documents. Or...

Update indexes to use tie-break ObjectId

In terms of priority, I think it's clear that fast (and correct...) search is more critical than RunStart document insert time.

Update indexes to use tie-break ObjectId

It looks like this one will do it. https://github.com/bluesky/suitcase-mongo/blob/4895ad9784784bcbc346fbffdc7f1694f6739cc0/suitcase/mongo_normalized/__init__.py#L89-L91

Update indexes to use tie-break ObjectId

In practice, we are always sorting by time and looking up by time range or by `scan_id`. I think that one index covers both.

Update indexes to use tie-break ObjectId

Fair, sorry for the false trail. I dashed this off in too much of a hurry, and I hadn't looked at the index creation in awhile.

Record created_at and updated_at timestamps in experimental Document

https://www.mongodb.com/blog/post/building-with-patterns-the-document-versioning-pattern

An incremental step to an "embedded" implementation

Can you list some ideas here? My opening suggestion is `header` or `bluesky_header_v1`.

Filling external events fails because extract_dtype does not convert "number" and "integer"

Thanks for this clear description and for your patience with the delayed reply. We are in the final stages of a major internal refactor of databroker, moving from [`intake`](https://github.com/intake/intake) to...