Using .orderBy in a Firestore query omits documents

Description

I have a recipe manager which relies heavily on the cache to show users their recipes.

I have a case I can reproduce locally (though I'm unsure how to reproduce it in a sample project) where doing a .getDocuments() on a collection returns 113 items, but doing a collection.order(by: "name", descending: false). getDocuments() returns only 34 items.

Every single document contains a name field.

Reproducing the issue

I'm not sure how to provide a reproducible example, but I can describe my setup.

On launch, clients query the cached firestore for the first 15 documents ordered by name. If they reach the bottom of the list I query the next 15 and so on, until there are no documents left.

I also attach a listener to the collection to monitor for remote changes, and items are inserted in the list as needed.

If I query my cache with no filters or sorts applied I see 113 documents, if I sort by name and attach no other modifiers to my query I get only 34 entries.

Firebase SDK Version

10.19.1

Xcode Version

15.2

Installation Method

Swift Package Manager

Firebase Product(s)

Firestore

Targeted Platforms

iOS

Relevant Log Output

No response

If using Swift Package Manager, the project's Package.resolved

{ "pins" : [ { "identity" : "abseil-cpp-binary", "kind" : "remoteSourceControl", "location" : "https://github.com/google/abseil-cpp-binary.git", "state" : { "revision" : "bfc0b6f81adc06ce5121eb23f628473638d67c5c", "version" : "1.2022062300.0" } }, { "identity" : "app-check", "kind" : "remoteSourceControl", "location" : "https://github.com/google/app-check.git", "state" : { "revision" : "3e464dad87dad2d29bb29a97836789bf0f8f67d2", "version" : "10.18.1" } }, { "identity" : "appauth-ios", "kind" : "remoteSourceControl", "location" : "https://github.com/openid/AppAuth-iOS.git", "state" : { "revision" : "71cde449f13d453227e687458144bde372d30fc7", "version" : "1.6.2" } }, { "identity" : "cocoalumberjack", "kind" : "remoteSourceControl", "location" : "https://github.com/CocoaLumberjack/CocoaLumberjack.git", "state" : { "revision" : "67ec5818a757aba4d7c534e21a905d878d128dbf", "version" : "3.8.1" } }, { "identity" : "firebase-ios-sdk", "kind" : "remoteSourceControl", "location" : "https://github.com/firebase/firebase-ios-sdk", "state" : { "revision" : "c60c958e707c50a9cf8bcb7cfd7d51c566d726c5", "version" : "10.19.1" } }, { "identity" : "googleappmeasurement", "kind" : "remoteSourceControl", "location" : "https://github.com/google/GoogleAppMeasurement.git", "state" : { "revision" : "6b332152355c372ace9966d8ee76ed191f97025e", "version" : "10.17.0" } }, { "identity" : "googledatatransport", "kind" : "remoteSourceControl", "location" : "https://github.com/google/GoogleDataTransport.git", "state" : { "revision" : "a732a4b47f59e4f725a2ea10f0c77e93a7131117", "version" : "9.3.0" } }, { "identity" : "googlesignin-ios", "kind" : "remoteSourceControl", "location" : "https://github.com/google/GoogleSignIn-iOS", "state" : { "revision" : "7932d33686c1dc4d7df7a919aae47361d1cdfda4", "version" : "7.0.0" } }, { "identity" : "googleutilities", "kind" : "remoteSourceControl", "location" : "https://github.com/google/GoogleUtilities.git", "state" : { "revision" : "bc27fad73504f3d4af235de451f02ee22586ebd3", "version" : "7.12.1" } }, { "identity" : "grpc-binary", "kind" : "remoteSourceControl", "location" : "https://github.com/google/grpc-binary.git", "state" : { "revision" : "a673bc2937fbe886dd1f99c401b01b6d977a9c98", "version" : "1.49.1" } }, { "identity" : "gtm-session-fetcher", "kind" : "remoteSourceControl", "location" : "https://github.com/google/gtm-session-fetcher.git", "state" : { "revision" : "115f75e43851774934d695449a4836123c3246e1", "version" : "3.2.0" } }, { "identity" : "gtmappauth", "kind" : "remoteSourceControl", "location" : "https://github.com/google/GTMAppAuth.git", "state" : { "revision" : "cee3c709307912d040bd1e06ca919875a92339c6", "version" : "2.0.0" } }, { "identity" : "gzipswift", "kind" : "remoteSourceControl", "location" : "https://github.com/1024jp/GzipSwift", "state" : { "revision" : "7a7f17761c76a932662ab77028a4329f67d645a4", "version" : "5.2.0" } }, { "identity" : "interop-ios-for-google-sdks", "kind" : "remoteSourceControl", "location" : "https://github.com/google/interop-ios-for-google-sdks.git", "state" : { "revision" : "2d12673670417654f08f5f90fdd62926dc3a2648", "version" : "100.0.0" } }, { "identity" : "leveldb", "kind" : "remoteSourceControl", "location" : "https://github.com/firebase/leveldb.git", "state" : { "revision" : "9d108e9112aa1d65ce508facf804674546116d9c", "version" : "1.22.3" } }, { "identity" : "nanopb", "kind" : "remoteSourceControl", "location" : "https://github.com/firebase/nanopb.git", "state" : { "revision" : "819d0a2173aff699fb8c364b6fb906f7cdb1a692", "version" : "2.30909.0" } }, { "identity" : "promises", "kind" : "remoteSourceControl", "location" : "https://github.com/google/promises.git", "state" : { "revision" : "e70e889c0196c76d22759eb50d6a0270ca9f1d9e", "version" : "2.3.1" } }, { "identity" : "purchases-ios", "kind" : "remoteSourceControl", "location" : "https://github.com/RevenueCat/purchases-ios", "state" : { "revision" : "3e41e9914d1e5a02fa1230a7b711dbaa4525c66a", "version" : "4.31.6" } }, { "identity" : "swift-log", "kind" : "remoteSourceControl", "location" : "https://github.com/apple/swift-log.git", "state" : { "revision" : "532d8b529501fb73a2455b179e0bbb6d49b652ed", "version" : "1.5.3" } }, { "identity" : "swift-protobuf", "kind" : "remoteSourceControl", "location" : "https://github.com/apple/swift-protobuf.git", "state" : { "revision" : "65e8f29b2d63c4e38e736b25c27b83e012159be8", "version" : "1.25.2" } }, { "identity" : "swiftclient", "kind" : "remoteSourceControl", "location" : "https://github.com/TelemetryDeck/SwiftClient", "state" : { "revision" : "98500be378267abaa072ed08c2caf7da6493489e", "version" : "1.5.0" } }, { "identity" : "swiftsoup", "kind" : "remoteSourceControl", "location" : "https://github.com/scinfu/SwiftSoup", "state" : { "revision" : "8b6cf29eead8841a1fa7822481cb3af4ddaadba6", "version" : "2.6.1" } }, { "identity" : "zip", "kind" : "remoteSourceControl", "location" : "https://github.com/marmelroy/Zip.git", "state" : { "revision" : "67fa55813b9e7b3b9acee9c0ae501def28746d76", "version" : "2.1.2" } } ], "version" : 2 }

If using CocoaPods, the project's Podfile.lock

No response

I couldn't figure out how to label this issue, so I've labeled it for a human to triage. Hang tight.

I was able to fix this by disabling automatic index creation and deleting all indexes. This is obvioulsy not ideal as I lose the benefit of having an offline index. Some of my users store 1000s of documents locally but I always read from the first document alphabetically by it's "name" field.

Hi @WillBishop ,

I am going to investigating this issue. In the mean time, if you can still reproduce this issue on your app, could you please help me confirm

1. this issue only happens when you add orderBy by name, doesn't happen if you add orderBy by other fields?
2. when you run the query, can you explicitly add orderBy cache?

Thank you for your help!

1. I can't confirm this at the moment as I have since disabled and deleted offline indexes
2. Not sure what you mean here? Here is the function when I was able to reproduce the issue by commenting out the order part

func getAllDocuments(from source: FirestoreSource = .cache) async throws -> [QueryDocumentSnapshot] {
	guard let documentCollection = self.documentCollectionReference() else { return [] }
	let baseQuery = documentCollection
		.order(by: "name", descending: false)
	return try await baseQuery.getDocuments(source: source).documents
}

Can you turn on your debug logs and paste what the SDk was doing when you turn on aut-index and run the query?

Hi @WillBishop ,

I try to investigate the behaviour by tracing the code however it is not obvious where does the problem come from. So a debug log would help to surface where the bug, for example is it from the process of creating target index or using index to find documents.

Also for you app, have you manually config the cache index before?

I hadn't configured the cache index before no, but since launching my app 2 years ago I've had offline cache enabled with no max index.

I'll try to get some logs

Hey @WillBishop. We need more information to resolve this issue but there hasn't been an update in 5 weekdays. I'm marking the issue as stale and if there are no new updates in the next 5 days I will close it automatically.

If you have more information that will help us get to the bottom of this, just add a comment!

Since there haven't been any recent updates here, I am going to close this issue.

@WillBishop if you're still experiencing this problem and want to continue the discussion just leave a comment here and we are happy to re-open this.