noobaa-core
noobaa-core copied to clipboard
Fixed map_common_prefixes queries
Signed-off-by: Danny Zaken [email protected]
Explain the changes
- Changed pl/sql function
map_common_prefixes
to fetch all required entries in one call. - instead of querying for 1000 keys and then extracting the common prefixes from it (can be very few common prefixes if there are a lot of objects in that "dir"), it iterates over the objectmds until reaching the max keys or the end.
- To avoid unnecessary scanning of objects - if we find a common prefix, we set the marker to be the next possible value for a common prefix (by incrementing the last character by 1)
- These changes rely on the DB having collation of LC_COLLATE = 'C'. it will not work on DBs with utf8 collation
Issues: Fixed #xxx / Gap #xxx
- More cleanup can be done after this change. for example, it is probably unnecessary to redo the list_object queries in the object_server, since now it should return all of the required count https://github.com/noobaa/noobaa-core/blob/5b6bcb568bbb699e735523067b83a37e622e8db9/src/server/object_services/object_server.js#L1028-L1031
Testing Instructions:
- [ ] Doc added/updated
- [ ] Tests added