go-ds-s3 icon indicating copy to clipboard operation
go-ds-s3 copied to clipboard

Published 20 hours ago • verified publisher icon ipfs

Fix query logic

Open Stebalien opened this issue 5 years ago • 3 comments

I've written a query test suite to test all query combination.

This PR currently fixes offset handling but we still need to handle:

  • Ordering
  • Filtering

Unfortunately, these are non-trivial because we need to handle them in the following order:

  1. Prefix
  2. Filter
  3. Order
  4. Offset
  5. Limit

To handle complex orders (i.e., anything other than "sort by key"), we can perform one query that handles 1-2 (prefix/filter) and then applies 3-5 via the naive query logic. However, getting this right is non-trivial.

Stebalien avatar Oct 01 '19 21:10 Stebalien

cc @MichaelMure @ianopolous, @hinshun, @tobowers

Unfortunately, I'm not sure when I'll have time to take this to completion but you should know about these issues/limitations. If any of you have the time/inclination to pick this up, I'd be happy to review a PR (and the query tests are now pretty thorough).

Stebalien avatar Oct 01 '19 21:10 Stebalien

@Stebalien what are the functionality impacts of not having this PR + the additional fixes you mentioned merged in?

obo20 avatar Apr 23 '20 15:04 obo20

  1. The S3 datastore probably still works for blocks, but may not be safe to use for other records.
  2. The test suite isn't running so there may be other bugs that we just haven't noticed.

Stebalien avatar Apr 23 '20 16:04 Stebalien