Add API allowing "last object before"

[arielshaqed]

What?

Let's add an API to return the last object with a given prefix! For instance, if I have objects with these keys:

/aa/000/x
/ab/000/y
/ab/001/abc
/ab/001/def
/ab/001/xyz
/ab/002/abc

then I would like statLastObject(repo, branch, prefix="/ab/001/", maybe_some_other_params) to stat /ab/001/xyz.

Why?

This frequently comes up in services that need to scan lakeFS. For instance: say the names of my objects contain a timestamp field. I would be able to find the key of the latest object by using a single API call.

How?

• PebbleDB iterators have SeekLT.
• DynamoDB Query let you set ScanIndexForward=false
• Badger iterators can scan backwards
• CosmosDB appears to support DESC ordering on its indexes, and we already get some indexing policy.

[talSofer]

Hi @arielshaqed can you please add a few use cases for this addition? How do you expect users to use it?

[talSofer]

@arielshaqed we will appreciate your feedback on this one, thanks!