firebase-ios-sdk icon indicating copy to clipboard operation
firebase-ios-sdk copied to clipboard

Published 20 hours ago • verified publisher icon firebase

Using .orderBy in a Firestore query omits documents

Open WillBishop opened this issue 1 year ago • 15 comments

Description

I have a recipe manager which relies heavily on the cache to show users their recipes.

I have a case I can reproduce locally (though I'm unsure how to reproduce it in a sample project) where doing a .getDocuments() on a collection returns 46 items, but doing a collection.order(by: "name", descending: false). getDocuments() returns only 40 items.

Every single document contains a name field.

Reproducing the issue

I'm not sure how to provide a reproducible example, but I can describe my setup.

On launch, clients query the cached firestore for the first 15 documents ordered by name. If they reach the bottom of the list I query the next 15 and so on, until there are no documents left.

I also attach a listener to the collection to monitor for remote changes, and items are inserted in the list as needed.

If I query my cache with no filters or sorts applied I see 46 documents, if I sort by name and attach no other modifiers to my query I get only 40 entries.

You can see these expectations in the logs by filtering for "User Code"

Firebase SDK Version

10.28.1

Xcode Version

15.3

Installation Method

Swift Package Manager

Firebase Product(s)

Firestore

Targeted Platforms

iOS

Relevant Log Output

2024-07-02 18:39:35.130047+0930 Pestle[3064:660942] 10.28.1 - [FirebaseCore][I-COR000003] The default Firebase app has not yet been configured. Add `FirebaseApp.configure()` to your application initialization. This can be done in in the App Delegate's application(_:didFinishLaunchingWithOptions:)` (or the `@main` struct's initializer in SwiftUI). Read more: https://goo.gl/ctyzm8.
2024-07-02 18:39:35.131059+0930 Pestle[3064:660942] 10.28.1 - [FirebaseCore][I-COR000001] Configuring the default app.
2024-07-02 18:39:35.132657+0930 Pestle[3064:660550] <Warning> [AppCheckCore][I-GAC004001] App Check debug token: '6F04CB17-8ECD-46AE-909D-41EC4C66C0F6'.
2024-07-02 18:39:35.134451+0930 Pestle[3064:660942] 10.28.1 - [FirebaseMessaging][I-FCM002000] FIRMessaging library version 10.28.1
2024-07-02 18:39:35.134477+0930 Pestle[3064:660942] 10.28.1 - [GULReachability][I-REA902003] Monitoring the network status
2024-07-02 18:39:35.134491+0930 Pestle[3064:660942] 10.28.1 - [FirebaseInstallations][I-FIS002000] -[FIRInstallationsIDController createGetInstallationItemPromise], appName: __FIRAPP_DEFAULT
2024-07-02 18:39:35.134504+0930 Pestle[3064:660942] 10.28.1 - [FirebaseAppCheck][I-FAA005001] Firebase App Check debug token: '6F04CB17-8ECD-46AE-909D-41EC4C66C0F6'.
2024-07-02 18:39:35.134516+0930 Pestle[3064:660942] 10.28.1 - [FirebaseCore][I-COR000033] Data Collection flag is not set.
2024-07-02 18:39:35.134528+0930 Pestle[3064:660942] 10.28.1 - [GoogleUtilities/AppDelegateSwizzler][I-SWZ001011] App Delegate Proxy is disabled.
2024-07-02 18:39:35.137095+0930 Pestle[3064:660550] [Firebase/Crashlytics] Version 10.28.1
2024-07-02 18:39:35.140576+0930 Pestle[3064:660934] 10.28.1 - [FirebaseSessions][I-SES000000] Version 10.28.1. Expecting subscriptions from: [Crashlytics]
2024-07-02 18:39:35.140689+0930 Pestle[3064:660934] 10.28.1 - [FirebaseCrashlytics][I-CLS000000] [Crashlytics] Running on iPhone15,2, 18.0.0 (22A5297f)
2024-07-02 18:39:35.140755+0930 Pestle[3064:660934] 10.28.1 - [FirebaseCrashlytics][I-CLS000000] Registering Sessions SDK subscription for session data
2024-07-02 18:39:35.140816+0930 Pestle[3064:660934] 10.28.1 - [FirebaseSessions][I-SES000000] Registering Sessions SDK subscriber with name: Crashlytics, data collection enabled: true
2024-07-02 18:39:35.140843+0930 Pestle[3064:660934] 10.28.1 - [FirebaseCrashlytics][I-CLS000000] Session ID changed: c973ffa29d6c42e4bf120dc5b89f3e40
2024-07-02 18:39:35.141625+0930 Pestle[3064:660934] 10.28.1 - [FirebaseCrashlytics][I-CLS000000] [Crashlytics:Settings] Settings expired because build instance changed
2024-07-02 18:39:35.142957+0930 Pestle[3064:660942] 10.28.1 - [FirebaseCrashlytics][I-CLS000000] Root: /var/mobile/Containers/Data/Application/B12BC8BD-4835-418D-8521-9A77C3C89768/Library/Caches/com.crashlytics.data/com.willbishop.Simmer
[TelemetryDeck: INFO] Loading Telemetry cache from: file:///var/mobile/Containers/Data/Application/B12BC8BD-4835-418D-8521-9A77C3C89768/Library/Caches/telemetrysignalcache
2024-07-02 18:39:35.217802+0930 Pestle[3064:660550] [User Defaults] Couldn't read values in CFPrefsPlistSource<0x303849c20> (Domain: group.willbishop.Pestle, User: kCFPreferencesAnyUser, ByHost: Yes, Container: (null), Contents Need Refresh: Yes): Using kCFPreferencesAnyUser with a container is only allowed for System Containers, detaching from cfprefsd
2024-07-02 18:39:35.220777+0930 Pestle[3064:660934] 10.28.1 - [FirebaseCrashlytics][I-CLS000000] [Crashlytics:Crash:Reports:Event] Firebase Analytics SDK not detected. Crash-free statistics and breadcrumbs will not be reported
2024-07-02 18:39:35.220810+0930 Pestle[3064:660934] 10.28.1 - [FirebaseCrashlytics][I-CLS000000] Automatic data collection is enabled.
2024-07-02 18:39:35.222150+0930 Pestle[3064:660934] 10.28.1 - [FirebaseCrashlytics][I-CLS000000] Unsent reports will be uploaded at startup
2024-07-02 18:39:35.321988+0930 Pestle[3064:660941] 10.28.1 - [FirebaseFirestore][I-FST000001] Initializing. Current user: MYUSERID
2024-07-02 18:39:35.322075+0930 Pestle[3064:660941] 10.28.1 - [FirebaseFirestore][I-FST000001] Using /var/mobile/Containers/Data/Application/B12BC8BD-4835-418D-8521-9A77C3C89768/Library/Application Support/firestore/__FIRAPP_DEFAULT/cookingapp-57562/main for LevelDB storage
Applied Snapshot
2024-07-02 18:39:35.358430+0930 Pestle[3064:660955] 10.28.1 - [FirebaseFirestore][I-FST000001] Committing transaction: <LevelDbTransaction Start LevelDB: 0 changes (0 bytes):>
2024-07-02 18:39:35.358456+0930 Pestle[3064:660955] 10.28.1 - [FirebaseFirestore][I-FST000001] Committing transaction: <LevelDbTransaction Start MutationQueue: 0 changes (0 bytes):>
2024-07-02 18:39:35.358466+0930 Pestle[3064:660955] 10.28.1 - [FirebaseFirestore][I-FST000001] Committing transaction: <LevelDbTransaction Start IndexManager: 0 changes (0 bytes):>
2024-07-02 18:39:35.358476+0930 Pestle[3064:660955] 10.28.1 - [FirebaseFirestore][I-FST000001] Committing transaction: <LevelDbTransaction migrate overlays: 0 changes (0 bytes):>
2024-07-02 18:39:35.358484+0930 Pestle[3064:660955] 10.28.1 - [FirebaseFirestore][I-FST000001] Committing transaction: <LevelDbTransaction NextMutationBatchAfterBatchID: 0 changes (0 bytes):>
2024-07-02 18:39:35.358495+0930 Pestle[3064:660955] 10.28.1 - [FirebaseFirestore][I-FST000001] Committing transaction: <LevelDbTransaction Allocate target: 0 changes (0 bytes):>
2024-07-02 18:39:35.359561+0930 Pestle[3064:660955] 10.28.1 - [FirebaseFirestore][I-FST000001] Using full collection scan to execute query: Query(canonical_id=users/MYUSERID|f:|ob:__name__asc)
2024-07-02 18:39:35.359618+0930 Pestle[3064:660955] 10.28.1 - [FirebaseFirestore][I-FST000001] SDK will not create cache indexes for query: Query(canonical_id=users/MYUSERID|f:|ob:__name__asc), since it only creates cache indexes for collection contains more than or equal to 100 documents.
2024-07-02 18:39:35.359656+0930 Pestle[3064:660955] 10.28.1 - [FirebaseFirestore][I-FST000001] Committing transaction: <LevelDbTransaction ExecuteQuery: 0 changes (0 bytes):>
2024-07-02 18:39:35.359672+0930 Pestle[3064:660955] 10.28.1 - [FirebaseFirestore][I-FST000001] WatchStream (h\^\rm\^A
2024-07-02 18:39:35.359683+0930 Pestle[3064:660955] 10.28.1 - [FirebaseAuth][I-AUT000002] Token auto-refresh enabled.
2024-07-02 18:39:35.359695+0930 Pestle[3064:660955] 10.28.1 - [FirebaseAuth][I-AUT000004] Token auto-refresh scheduled in 40:11 for the new token.
2024-07-02 18:39:35.359706+0930 Pestle[3064:660955] 10.28.1 - [FirebaseFirestore][I-FST000001] Using full collection scan to execute query: Query(canonical_id=users/MYUSERID/privateRecipes|f:|ob:__name__asc|lt:f)
2024-07-02 18:39:35.359718+0930 Pestle[3064:660955] 10.28.1 - [FirebaseAuth][I-AUT000017] Has valid access token. Estimated expiration date: 2024-07-02 09:54:46 +0000, current date: 2024-07-02 09:09:35 +0000
2024-07-02 18:39:35.641671+0930 Pestle[3064:660955] [loading] Unable to create bundle at URL (file:///System/Library/CoreServices/SystemVersion.bundle): does not exist or not a directory (0)
2024-07-02 18:39:35.641701+0930 Pestle[3064:660955] [loading] Unable to create bundle at URL (file:///System/Library/CoreServices/SystemVersion.bundle): does not exist or not a directory (0)
2024-07-02 18:39:35.641717+0930 Pestle[3064:660955] [loading] Unable to create bundle at URL (file:///System/Library/CoreServices/SystemVersion.bundle): does not exist or not a directory (0)
2024-07-02 18:39:35.652411+0930 Pestle[3064:660936] 10.28.1 - [GULReachability][I-REA902004] Network status has changed. Code:3, status:Connected
2024-07-02 18:39:35.655051+0930 Pestle[3064:660932] [] nw_path_necp_check_for_updates Failed to copy updated result (22)
User Code: Executing getDocuments. Expecting 46
2024-07-02 18:39:35.831797+0930 Pestle[3064:660932] 10.28.1 - [FirebaseFirestore][I-FST000001] GetToken aborted due to token change.
2024-07-02 18:39:35.831877+0930 Pestle[3064:660932] 10.28.1 - [FirebaseAuth][I-AUT000017] Has valid access token. Estimated expiration date: 2024-07-02 09:54:46 +0000, current date: 2024-07-02 09:09:35 +0000
2024-07-02 18:39:35.831924+0930 Pestle[3064:660932] 10.28.1 - [FirebaseMessaging][I-FCM023012] Provisioning profile has specifically provisioned devices, most likely a Dev profile.
2024-07-02 18:39:35.832099+0930 Pestle[3064:660932] 10.28.1 - [FirebaseMessaging][I-FCM023013] APNS Environment in profile: development
2024-07-02 18:39:35.832532+0930 Pestle[3064:660932] 10.28.1 - [FirebaseInstallations][I-FIS002001] -[FIRInstallationsIDController installationWithValidAuthTokenForcingRefresh:0], appName: __FIRAPP_DEFAULT
2024-07-02 18:39:35.832685+0930 Pestle[3064:660932] 10.28.1 - [FirebaseFirestore][I-FST000001] Query: Query(canonical_id=users/MYUSERID/privateRecipes|f:|ob:__name__asc|lt:f), scans 773 local documents and returns 46 documents as results.
2024-07-02 18:39:35.832739+0930 Pestle[3064:660932] 10.28.1 - [FirebaseFirestore][I-FST000001] The SDK decides to create cache indexes for query: Query(canonical_id=users/MYUSERID/privateRecipes|f:|ob:__name__asc|lt:f), as using cache indexes may help improve performance.
2024-07-02 18:39:35.832785+0930 Pestle[3064:660932] 10.28.1 - [FirebaseFirestore][I-FST000001] Committing transaction: <LevelDbTransaction ExecuteQuery: 0 changes (0 bytes):>
2024-07-02 18:39:35.833048+0930 Pestle[3064:660932] 10.28.1 - [FirebaseFirestore][I-FST000001] Using index privateRecipes to execute target users/MYUSERID/privateRecipes|f:dateAdded>=time(1719738575,328838825)|ob:dateAddedasc__name__asc
2024-07-02 18:39:35.833184+0930 Pestle[3064:660932] 10.28.1 - [FirebaseCrashlytics][I-CLS000000] Settings downloaded successfully
2024-07-02 18:39:35.844348+0930 Pestle[3064:660971] 10.28.1 - [FirebaseSessions][I-SES000000] Data Collection is enabled for at least one Subscriber
2024-07-02 18:39:35.844388+0930 Pestle[3064:660971] 10.28.1 - [FirebaseSessions][I-SES000000] [Settings] Cache is not expired, no fetch will be made.
2024-07-02 18:39:35.844437+0930 Pestle[3064:660971] 10.28.1 - [FirebaseInstallations][I-FIS002001] -[FIRInstallationsIDController installationWithValidAuthTokenForcingRefresh:0], appName: __FIRAPP_DEFAULT
2024-07-02 18:39:35.844488+0930 Pestle[3064:660971] 10.28.1 - [FirebaseInstallations][I-FIS002000] -[FIRInstallationsIDController createGetInstallationItemPromise], appName: __FIRAPP_DEFAULT
2024-07-02 18:39:35.909993+0930 Pestle[3064:660933] 10.28.1 - [FirebaseSessions][I-SES000000] Successfully logged Session Start event to GoogleDataTransport
2024-07-02 18:39:35.910890+0930 Pestle[3064:660933] 10.28.1 - [FirebaseSessions][I-SES000000] Successfully logged Session Start event
2024-07-02 18:39:35.914246+0930 Pestle[3064:660933] 10.28.1 - [FirebaseFirestore][I-FST000001] Committing transaction: <LevelDbTransaction ExecuteQuery: 0 changes (0 bytes):>
2024-07-02 18:39:35.914380+0930 Pestle[3064:660933] 10.28.1 - [FirebaseFirestore][I-FST000001] Using full collection scan to execute query: Query(canonical_id=users/MYUSERID/privateRecipes|f:|ob:__name__asc|lt:f)
2024-07-02 18:39:36.008612+0930 Pestle[3064:660932] 10.28.1 - [FirebaseFirestore][I-FST000001] Query: Query(canonical_id=users/MYUSERID/privateRecipes|f:|ob:__name__asc|lt:f), scans 773 local documents and returns 46 documents as results.
2024-07-02 18:39:36.012231+0930 Pestle[3064:660932] 10.28.1 - [FirebaseFirestore][I-FST000001] The SDK decides to create cache indexes for query: Query(canonical_id=users/MYUSERID/privateRecipes|f:|ob:__name__asc|lt:f), as using cache indexes may help improve performance.
2024-07-02 18:39:36.012267+0930 Pestle[3064:660932] 10.28.1 - [FirebaseFirestore][I-FST000001] Committing transaction: <LevelDbTransaction ExecuteQuery: 0 changes (0 bytes):>
2024-07-02 18:39:36.014727+0930 Pestle[3064:660932] 10.28.1 - [FirebaseAuth][I-AUT000017] Has valid access token. Estimated expiration date: 2024-07-02 09:54:46 +0000, current date: 2024-07-02 09:09:36 +0000
2024-07-02 18:39:36.015886+0930 Pestle[3064:660932] 10.28.1 - [FirebaseFirestore][I-FST000001] Committing transaction: <LevelDbTransaction NextMutationBatchAfterBatchID: 0 changes (0 bytes):>
2024-07-02 18:39:36.016108+0930 Pestle[3064:660932] 10.28.1 - [FirebaseFirestore][I-FST000001] Credential Changed. Current user: MYUSERID
2024-07-02 18:39:36.016276+0930 Pestle[3064:660932] 10.28.1 - [FirebaseAuth][I-AUT000017] Has valid access token. Estimated expiration date: 2024-07-02 09:54:46 +0000, current date: 2024-07-02 09:09:36 +0000
2024-07-02 18:39:36.016314+0930 Pestle[3064:660932] 10.28.1 - [FirebaseFirestore][I-FST000001] Committing transaction: <LevelDbTransaction NextMutationBatchAfterBatchID: 0 changes (0 bytes):>
2024-07-02 18:39:36.016341+0930 Pestle[3064:660932] 10.28.1 - [FirebaseFirestore][I-FST000001] Using index privateRecipes to execute target users/MYUSERID/privateRecipes|f:|ob:nameasc__name__asc
2024-07-02 18:39:36.371359+0930 Pestle[3064:660932] 10.28.1 - [FirebaseFirestore][I-FST000001] Committing transaction: <LevelDbTransaction ExecuteQuery: 0 changes (0 bytes):>
2024-07-02 18:39:36.371933+0930 Pestle[3064:660932] 10.28.1 - [FirebaseFirestore][I-FST000001] Using index privateRecipes to execute target users/MYUSERID/privateRecipes|f:|ob:nameasc__name__asc
User Code: Recevied Optional(40) documents
2024-07-02 18:39:40.305127+0930 Pestle[3064:660941] 10.28.1 - [FirebaseFirestore][I-FST000001] Committing transaction: <LevelDbTransaction ExecuteQuery: 0 changes (0 bytes):>
2024-07-02 18:39:40.305371+0930 Pestle[3064:660941] 10.28.1 - [FirebaseFirestore][I-FST000001] Committing transaction: <LevelDbTransaction Allocate target: 3 changes (186 bytes):
  - Put [query_target: canonical_id=betaAnnouncements|f:endRelevant>time(1719911375,691611051)|ob:endRelevantasc__name__asc target_id=13980] (0 bytes)
  - Put [target: target_id=13980] (162 bytes)
  - Put [target_global:] (24 bytes)>
2024-07-02 18:39:40.306759+0930 Pestle[3064:660941] 10.28.1 - [FirebaseFirestore][I-FST000001] Using full collection scan to execute query: Query(canonical_id=betaAnnouncements|f:endRelevant>time(1719911375,691611051)|ob:endRelevantasc__name__asc)
2024-07-02 18:39:40.307517+0930 Pestle[3064:660941] 10.28.1 - [FirebaseFirestore][I-FST000001] SDK will not create cache indexes for query: Query(canonical_id=betaAnnouncements|f:endRelevant>time(1719911375,691611051)|ob:endRelevantasc__name__asc), since it only creates cache indexes for collection contains more than or equal to 100 documents.
2024-07-02 18:39:40.375912+0930 Pestle[3064:660941] 10.28.1 - [FirebaseFirestore][I-FST000001] Committing transaction: <LevelDbTransaction ExecuteQuery: 0 changes (0 bytes):>
2024-07-02 18:39:40.375999+0930 Pestle[3064:660941] 10.28.1 - [FirebaseFirestore][I-FST000001] Committing transaction: <LevelDbTransaction Locally write mutations: 7 changes (70324 bytes):
  - Put [collection_parent: collection_id=users] (0 bytes)
  - Put [document_mutation: user_id=MYUSERID path=users/MYUSERID batch_id=1] (0 bytes)
  - Put [document_overlays: user_id=MYUSERID path=users/MYUSERID batch_id=1] (42059 bytes)
  - Put [document_overlays_collection_group_index: user_id=MYUSERID collection_group=users batch_id=1 path=users/MYUSERID] (0 bytes)
  - Put [document_overlays_collection_index: user_id=MYUSERID path=users batch_id=1 document_id=MYUSERID] (0 bytes)
  - Put [document_overlays_largest_batch_id_index: user_id=MYUSERID batch_id=1 path=users/MYUSERID] (0 bytes)
  - Put [mutation: user_id=MYUSERID batch_id=1] (28265 bytes)>
2024-07-02 18:39:40.524922+0930 Pestle[3064:660941] 10.28.1 - [FirebaseFirestore][I-FST000001] Committing transaction: <LevelDbTransaction NotifyLocalViewChanges: 0 changes (0 bytes):>
2024-07-02 18:39:40.599815+0930 Pestle[3064:660941] 10.28.1 - [FirebaseFirestore][I-FST000001] Committing transaction: <LevelDbTransaction NextMutationBatchAfterBatchID: 0 changes (0 bytes):>
2024-07-02 18:39:40.758985+0930 Pestle[3064:660941] 10.28.1 - [FirebaseFirestore][I-FST000001] Committing transaction: <LevelDbTransaction NextMutationBatchAfterBatchID: 0 changes (0 bytes):>
2024-07-02 18:39:40.759689+0930 Pestle[3064:660941] 10.28.1 - [FirebaseFirestore][I-FST000001] Creating Firestore stub.
2024-07-02 18:39:40.759776+0930 Pestle[3064:660941] 10.28.1 - [FirebaseAuth][I-AUT000017] Has valid access token. Estimated expiration date: 2024-07-02 09:54:46 +0000, current date: 2024-07-02 09:09:39 +0000
2024-07-02 18:39:40.839330+0930 Pestle[3064:660937] 10.28.1 - [FirebaseFirestore][I-FST000001] LocalStore Ignoring outdated update for betaAnnouncements/mailinglist. Current version: Timestamp(seconds=1706179480, nanoseconds=788218000)  Remote version: Timestamp(seconds=1706179480, nanoseconds=788218000)
2024-07-02 18:39:40.839426+0930 Pestle[3064:660937] 10.28.1 - [FirebaseFirestore][I-FST000001] Committing transaction: <LevelDbTransaction Apply remote event: 5 changes (216 bytes):
  - Put [document_target: path=betaAnnouncements/mailinglist target_id=0] (3 bytes)
  - Put [document_target: path=betaAnnouncements/mailinglist target_id=13980] (0 bytes)
  - Put [target: target_id=13980] (189 bytes)
  - Put [target_document: target_id=13980 path=betaAnnouncements/mailinglist] (0 bytes)
  - Put [target_global:] (24 bytes)>
2024-07-02 18:39:40.839502+0930 Pestle[3064:660937] 10.28.1 - [FirebaseFirestore][I-FST000001] Committing transaction: <LevelDbTransaction NotifyLocalViewChanges: 0 changes (0 bytes):>
2024-07-02 18:39:40.839887+0930 Pestle[3064:660937] 10.28.1 - [FirebaseFirestore][I-FST000001] Committing transaction: <LevelDbTransaction Apply remote event: 1 changes (24 bytes):
  - Put [target_global:] (24 bytes)>
2024-07-02 18:39:40.839927+0930 Pestle[3064:660937] 10.28.1 - [FirebaseFirestore][I-FST000001] Committing transaction: <LevelDbTransaction NotifyLocalViewChanges: 0 changes (0 bytes):>
2024-07-02 18:39:41.033686+0930 Pestle[3064:660550] Metal API Validation Enabled
2024-07-02 18:39:41.147800+0930 Pestle[3064:660550] [Unknown process name] CGImageProviderGetContentHeadroom: Bad headroom value 0.000000 for SDR, returning 1.0
2024-07-02 18:39:41.153555+0930 Pestle[3064:660550] [Unknown process name] CGImageProviderGetContentHeadroom: Bad headroom value 0.000000 for SDR, returning 1.0
2024-07-02 18:39:41.160571+0930 Pestle[3064:660550] [Unknown process name] CGImageProviderGetContentHeadroom: Bad headroom value 0.000000 for SDR, returning 1.0
2024-07-02 18:39:41.187012+0930 Pestle[3064:660972] 10.28.1 - [FirebaseFirestore][I-FST000001] Committing transaction: <LevelDbTransaction Release target: 2 changes (227 bytes):
  - Put [target: target_id=13980] (203 bytes)
  - Put [target_global:] (24 bytes)>
2024-07-02 18:39:41.189991+0930 Pestle[3064:660550] [Unknown process name] CGImageProviderGetContentHeadroom: Bad headroom value 0.000000 for SDR, returning 1.0
2024-07-02 18:39:41.192967+0930 Pestle[3064:660550] [Unknown process name] CGImageProviderGetContentHeadroom: Bad headroom value 0.000000 for SDR, returning 1.0
2024-07-02 18:39:41.196554+0930 Pestle[3064:660550] [Unknown process name] CGImageProviderGetContentHeadroom: Bad headroom value 0.000000 for SDR, returning 1.0
User Code: Executing getDocuments. Expecting 46
2024-07-02 18:39:41.223270+0930 Pestle[3064:660937] 10.28.1 - [FirebaseFirestore][I-FST000001] Using index privateRecipes to execute target users/MYUSERID/privateRecipes|f:|ob:dateAddeddesc__name__desc
2024-07-02 18:39:41.306808+0930 Pestle[3064:660973] 10.28.1 - [FirebaseFirestore][I-FST000001] Committing transaction: <LevelDbTransaction ExecuteQuery: 0 changes (0 bytes):>
User Code: Recevied Optional(40) documents
2024-07-02 18:39:41.606172+0930 Pestle[3064:660972] 10.28.1 - [FirebaseFirestore][I-FST000001] Committing transaction: <LevelDbTransaction Set stream token: 1 changes (11 bytes):
  - Put [mutation_queue: user_id=MYUSERID] (11 bytes)>
2024-07-02 18:39:41.606184+0930 Pestle[3064:660972] 10.28.1 - [FirebaseFirestore][I-FST000001] (null)

If using Swift Package Manager, the project's Package.resolved

{ "originHash" : "59824019fe0baafd60e68f8074c49456c84efe83e448e2affee2e4f43135c13b", "pins" : [ { "identity" : "abseil-cpp-binary", "kind" : "remoteSourceControl", "location" : "https://github.com/google/abseil-cpp-binary.git", "state" : { "revision" : "748c7837511d0e6a507737353af268484e1745e2", "version" : "1.2024011601.1" } }, { "identity" : "app-check", "kind" : "remoteSourceControl", "location" : "https://github.com/google/app-check.git", "state" : { "revision" : "3b62f154d00019ae29a71e9738800bb6f18b236d", "version" : "10.19.2" } }, { "identity" : "firebase-ios-sdk", "kind" : "remoteSourceControl", "location" : "https://github.com/firebase/firebase-ios-sdk", "state" : { "revision" : "e57841b296d04370ea23580f908881b0ccab17b9", "version" : "10.28.1" } }, { "identity" : "googleappmeasurement", "kind" : "remoteSourceControl", "location" : "https://github.com/google/GoogleAppMeasurement.git", "state" : { "revision" : "fe727587518729046fc1465625b9afd80b5ab361", "version" : "10.28.0" } }, { "identity" : "googledatatransport", "kind" : "remoteSourceControl", "location" : "https://github.com/google/GoogleDataTransport.git", "state" : { "revision" : "a637d318ae7ae246b02d7305121275bc75ed5565", "version" : "9.4.0" } }, { "identity" : "googleutilities", "kind" : "remoteSourceControl", "location" : "https://github.com/google/GoogleUtilities.git", "state" : { "revision" : "57a1d307f42df690fdef2637f3e5b776da02aad6", "version" : "7.13.3" } }, { "identity" : "grpc-binary", "kind" : "remoteSourceControl", "location" : "https://github.com/google/grpc-binary.git", "state" : { "revision" : "e9fad491d0673bdda7063a0341fb6b47a30c5359", "version" : "1.62.2" } }, { "identity" : "gtm-session-fetcher", "kind" : "remoteSourceControl", "location" : "https://github.com/google/gtm-session-fetcher.git", "state" : { "revision" : "a2ab612cb980066ee56d90d60d8462992c07f24b", "version" : "3.5.0" } }, { "identity" : "interop-ios-for-google-sdks", "kind" : "remoteSourceControl", "location" : "https://github.com/google/interop-ios-for-google-sdks.git", "state" : { "revision" : "2d12673670417654f08f5f90fdd62926dc3a2648", "version" : "100.0.0" } }, { "identity" : "leveldb", "kind" : "remoteSourceControl", "location" : "https://github.com/firebase/leveldb.git", "state" : { "revision" : "a0bc79961d7be727d258d33d5a6b2f1023270ba1", "version" : "1.22.5" } }, { "identity" : "nanopb", "kind" : "remoteSourceControl", "location" : "https://github.com/firebase/nanopb.git", "state" : { "revision" : "b7e1104502eca3a213b46303391ca4d3bc8ddec1", "version" : "2.30910.0" } }, { "identity" : "promises", "kind" : "remoteSourceControl", "location" : "https://github.com/google/promises.git", "state" : { "revision" : "540318ecedd63d883069ae7f1ed811a2df00b6ac", "version" : "2.4.0" } }, { "identity" : "swift-log", "kind" : "remoteSourceControl", "location" : "https://github.com/apple/swift-log", "state" : { "revision" : "9cb486020ebf03bfa5b5df985387a14a98744537", "version" : "1.6.1" } }, { "identity" : "swift-protobuf", "kind" : "remoteSourceControl", "location" : "https://github.com/apple/swift-protobuf.git", "state" : { "revision" : "9f0c76544701845ad98716f3f6a774a892152bcb", "version" : "1.26.0" } }, ], "version" : 3 }

If using CocoaPods, the project's Podfile.lock

No response

WillBishop avatar Jul 02 '24 09:07 WillBishop

For some more context, I track the number of documents by counting the docs in the cache with no filters or orders.

Ordered

recipeCollection
    .order(by: "name")
    .getDocuments(source: .cache) { snapshot, error in
        // 40 documents in the snapshot
    }

Unordered

recipeCollection
    .getDocuments(source: .cache) { snapshot, error in
        // 46 documents in the snapshot
    }

WillBishop avatar Jul 02 '24 09:07 WillBishop

I quietened my code a bit to make sure I only execute a singly query, here are those logs:

2024-07-02 18:54:16.285121+0930 Pestle[3169:675912] 10.28.1 - [FirebaseCore][I-COR000003] The default Firebase app has not yet been configured. Add `FirebaseApp.configure()` to your application initialization. This can be done in in the App Delegate's application(_:didFinishLaunchingWithOptions:)` (or the `@main` struct's initializer in SwiftUI). Read more: https://goo.gl/ctyzm8.
2024-07-02 18:54:16.285155+0930 Pestle[3169:675912] 10.28.1 - [FirebaseCore][I-COR000001] Configuring the default app.
2024-07-02 18:54:16.351539+0930 Pestle[3169:675467] <Warning> [AppCheckCore][I-GAC004001] App Check debug token: 'DEBUGTOKEN'.
2024-07-02 18:54:16.365594+0930 Pestle[3169:675467] [Firebase/Crashlytics] Version 10.28.1
2024-07-02 18:54:16.367630+0930 Pestle[3169:675927] 10.28.1 - [FirebaseAppCheck][I-FAA005001] Firebase App Check debug token: 'DEBUGTOKEN'.
2024-07-02 18:54:16.367658+0930 Pestle[3169:675927] 10.28.1 - [FirebaseCore][I-COR000033] Data Collection flag is not set.
2024-07-02 18:54:16.367671+0930 Pestle[3169:675927] 10.28.1 - [GoogleUtilities/AppDelegateSwizzler][I-SWZ001011] App Delegate Proxy is disabled.
2024-07-02 18:54:16.369618+0930 Pestle[3169:675927] 10.28.1 - [FirebaseMessaging][I-FCM002000] FIRMessaging library version 10.28.1
2024-07-02 18:54:16.369864+0930 Pestle[3169:675927] 10.28.1 - [GULReachability][I-REA902003] Monitoring the network status
2024-07-02 18:54:16.369887+0930 Pestle[3169:675927] 10.28.1 - [FirebaseInstallations][I-FIS002000] -[FIRInstallationsIDController createGetInstallationItemPromise], appName: __FIRAPP_DEFAULT
2024-07-02 18:54:16.369904+0930 Pestle[3169:675927] 10.28.1 - [FirebaseSessions][I-SES000000] Version 10.28.1. Expecting subscriptions from: [Crashlytics]
2024-07-02 18:54:16.369920+0930 Pestle[3169:675927] 10.28.1 - [FirebaseCrashlytics][I-CLS000000] [Crashlytics] Running on iPhone15,2, 18.0.0 (22A5297f)
2024-07-02 18:54:16.369936+0930 Pestle[3169:675927] 10.28.1 - [FirebaseCrashlytics][I-CLS000000] Registering Sessions SDK subscription for session data
2024-07-02 18:54:16.370394+0930 Pestle[3169:675927] 10.28.1 - [FirebaseSessions][I-SES000000] Registering Sessions SDK subscriber with name: Crashlytics, data collection enabled: true
2024-07-02 18:54:16.370417+0930 Pestle[3169:675927] 10.28.1 - [FirebaseCrashlytics][I-CLS000000] Session ID changed: 301cada883404b96b31f90b27d55f022
2024-07-02 18:54:16.370433+0930 Pestle[3169:675927] 10.28.1 - [FirebaseCrashlytics][I-CLS000000] [Crashlytics:Settings] Settings expired because build instance changed
2024-07-02 18:54:16.370447+0930 Pestle[3169:675927] 10.28.1 - [FirebaseCrashlytics][I-CLS000000] Root: /var/mobile/Containers/Data/Application/A0035EDF-7ED5-41D8-8B3C-A59E944E5CA6/Library/Caches/com.crashlytics.data/com.willbishop.Simmer
2024-07-02 18:54:16.528536+0930 Pestle[3169:675926] 10.28.1 - [FirebaseCrashlytics][I-CLS000000] [Crashlytics:Crash:Reports:Event] Firebase Analytics SDK not detected. Crash-free statistics and breadcrumbs will not be reported
2024-07-02 18:54:16.528577+0930 Pestle[3169:675926] 10.28.1 - [FirebaseCrashlytics][I-CLS000000] Automatic data collection is enabled.
2024-07-02 18:54:16.528592+0930 Pestle[3169:675926] 10.28.1 - [FirebaseCrashlytics][I-CLS000000] Unsent reports will be uploaded at startup
2024-07-02 18:54:16.528614+0930 Pestle[3169:675926] 10.28.1 - [FirebaseFirestore][I-FST000001] Initializing. Current user: MYUSERID
2024-07-02 18:54:16.528628+0930 Pestle[3169:675926] 10.28.1 - [FirebaseFirestore][I-FST000001] Using /var/mobile/Containers/Data/Application/A0035EDF-7ED5-41D8-8B3C-A59E944E5CA6/Library/Application Support/firestore/__FIRAPP_DEFAULT/cookingapp-57562/main for LevelDB storage
2024-07-02 18:54:16.580176+0930 Pestle[3169:675923] 10.28.1 - [FirebaseFirestore][I-FST000001] Committing transaction: <LevelDbTransaction Start LevelDB: 0 changes (0 bytes):>
2024-07-02 18:54:16.582832+0930 Pestle[3169:675921] 10.28.1 - [FirebaseFirestore][I-FST000001] Committing transaction: <LevelDbTransaction Start MutationQueue: 0 changes (0 bytes):>
2024-07-02 18:54:16.583234+0930 Pestle[3169:675921] 10.28.1 - [FirebaseFirestore][I-FST000001] Committing transaction: <LevelDbTransaction Start IndexManager: 0 changes (0 bytes):>
2024-07-02 18:54:16.583514+0930 Pestle[3169:675921] 10.28.1 - [FirebaseFirestore][I-FST000001] Committing transaction: <LevelDbTransaction migrate overlays: 0 changes (0 bytes):>
2024-07-02 18:54:16.583684+0930 Pestle[3169:675914] 10.28.1 - [FirebaseFirestore][I-FST000001] Committing transaction: <LevelDbTransaction NextMutationBatchAfterBatchID: 0 changes (0 bytes):>
2024-07-02 18:54:16.585100+0930 Pestle[3169:675914] 10.28.1 - [FirebaseFirestore][I-FST000001] Committing transaction: <LevelDbTransaction Allocate target: 0 changes (0 bytes):>
2024-07-02 18:54:16.585589+0930 Pestle[3169:675914] 10.28.1 - [FirebaseFirestore][I-FST000001] Using full collection scan to execute query: Query(canonical_id=users/MYUSERID|f:|ob:__name__asc)
Applied Snapshot
2024-07-02 18:54:16.587029+0930 Pestle[3169:675914] 10.28.1 - [FirebaseFirestore][I-FST000001] SDK will not create cache indexes for query: Query(canonical_id=users/MYUSERID|f:|ob:__name__asc), since it only creates cache indexes for collection contains more than or equal to 100 documents.
2024-07-02 18:54:16.587051+0930 Pestle[3169:675914] 10.28.1 - [FirebaseFirestore][I-FST000001] Committing transaction: <LevelDbTransaction ExecuteQuery: 0 changes (0 bytes):>
2024-07-02 18:54:16.587293+0930 Pestle[3169:675926] 10.28.1 - [FirebaseFirestore][I-FST000001] (null)
2024-07-02 18:54:16.587326+0930 Pestle[3169:675926] 10.28.1 - [FirebaseAuth][I-AUT000002] Token auto-refresh enabled.
2024-07-02 18:54:16.587398+0930 Pestle[3169:675926] 10.28.1 - [FirebaseAuth][I-AUT000004] Token auto-refresh scheduled in 25:30 for the new token.
2024-07-02 18:54:16.587428+0930 Pestle[3169:675926] 10.28.1 - [FirebaseAuth][I-AUT000017] Has valid access token. Estimated expiration date: 2024-07-02 09:54:46 +0000, current date: 2024-07-02 09:24:16 +0000
2024-07-02 18:54:16.587496+0930 Pestle[3169:675926] 10.28.1 - [FirebaseFirestore][I-FST000001] Using full collection scan to execute query: Query(canonical_id=users/MYUSERID/privateRecipes|f:|ob:__name__asc|lt:f)
2024-07-02 18:54:16.892708+0930 Pestle[3169:675912] 10.28.1 - [GULReachability][I-REA902004] Network status has changed. Code:3, status:Connected
2024-07-02 18:54:16.937782+0930 Pestle[3169:675919] 10.28.1 - [FirebaseFirestore][I-FST000001] GetToken aborted due to token change.
2024-07-02 18:54:16.941951+0930 Pestle[3169:675919] 10.28.1 - [FirebaseAuth][I-AUT000017] Has valid access token. Estimated expiration date: 2024-07-02 09:54:46 +0000, current date: 2024-07-02 09:24:16 +0000
2024-07-02 18:54:16.942000+0930 Pestle[3169:675919] 10.28.1 - [FirebaseMessaging][I-FCM023012] Provisioning profile has specifically provisioned devices, most likely a Dev profile.
2024-07-02 18:54:16.943450+0930 Pestle[3169:675919] 10.28.1 - [FirebaseMessaging][I-FCM023013] APNS Environment in profile: development
2024-07-02 18:54:16.959529+0930 Pestle[3169:675919] 10.28.1 - [FirebaseInstallations][I-FIS002001] -[FIRInstallationsIDController installationWithValidAuthTokenForcingRefresh:0], appName: __FIRAPP_DEFAULT
2024-07-02 18:54:16.959658+0930 Pestle[3169:675919] 10.28.1 - [FirebaseSessions][I-SES000000] Data Collection is enabled for at least one Subscriber
2024-07-02 18:54:16.963401+0930 Pestle[3169:675921] 10.28.1 - [FirebaseSessions][I-SES000000] [Settings] Cache is not expired, no fetch will be made.
2024-07-02 18:54:16.963774+0930 Pestle[3169:675921] 10.28.1 - [FirebaseInstallations][I-FIS002001] -[FIRInstallationsIDController installationWithValidAuthTokenForcingRefresh:0], appName: __FIRAPP_DEFAULT
2024-07-02 18:54:16.995733+0930 Pestle[3169:675921] 10.28.1 - [FirebaseInstallations][I-FIS002000] -[FIRInstallationsIDController createGetInstallationItemPromise], appName: __FIRAPP_DEFAULT
2024-07-02 18:54:16.995805+0930 Pestle[3169:675921] 10.28.1 - [FirebaseFirestore][I-FST000001] Query: Query(canonical_id=users/MYUSERID/privateRecipes|f:|ob:__name__asc|lt:f), scans 773 local documents and returns 46 documents as results.
2024-07-02 18:54:16.995829+0930 Pestle[3169:675921] 10.28.1 - [FirebaseFirestore][I-FST000001] The SDK decides to create cache indexes for query: Query(canonical_id=users/MYUSERID/privateRecipes|f:|ob:__name__asc|lt:f), as using cache indexes may help improve performance.
2024-07-02 18:54:16.995980+0930 Pestle[3169:675921] 10.28.1 - [FirebaseFirestore][I-FST000001] Committing transaction: <LevelDbTransaction ExecuteQuery: 0 changes (0 bytes):>
2024-07-02 18:54:17.009213+0930 Pestle[3169:675967] 10.28.1 - [FirebaseFirestore][I-FST000001] Using index privateRecipes to execute target users/MYUSERID/privateRecipes|f:dateAdded>=time(1719739456,569427013)|ob:dateAddedasc__name__asc
2024-07-02 18:54:17.009597+0930 Pestle[3169:675967] 10.28.1 - [FirebaseSessions][I-SES000000] Successfully logged Session Start event to GoogleDataTransport
2024-07-02 18:54:17.009874+0930 Pestle[3169:675967] 10.28.1 - [FirebaseSessions][I-SES000000] Successfully logged Session Start event
2024-07-02 18:54:17.101235+0930 Pestle[3169:675919] 10.28.1 - [FirebaseFirestore][I-FST000001] Committing transaction: <LevelDbTransaction ExecuteQuery: 0 changes (0 bytes):>
2024-07-02 18:54:17.102116+0930 Pestle[3169:675919] 10.28.1 - [FirebaseFirestore][I-FST000001] Using full collection scan to execute query: Query(canonical_id=users/MYUSERID/privateRecipes|f:|ob:__name__asc|lt:f)
2024-07-02 18:54:18.224269+0930 Pestle[3169:675967] 10.28.1 - [FirebaseCrashlytics][I-CLS000000] Settings downloaded successfully
2024-07-02 18:54:21.411952+0930 Pestle[3169:675913] 10.28.1 - [FirebaseFirestore][I-FST000001] Query: Query(canonical_id=users/MYUSERID/privateRecipes|f:|ob:__name__asc|lt:f), scans 773 local documents and returns 46 documents as results.
2024-07-02 18:54:21.414109+0930 Pestle[3169:675913] 10.28.1 - [FirebaseFirestore][I-FST000001] The SDK decides to create cache indexes for query: Query(canonical_id=users/MYUSERID/privateRecipes|f:|ob:__name__asc|lt:f), as using cache indexes may help improve performance.
2024-07-02 18:54:21.414659+0930 Pestle[3169:675913] 10.28.1 - [FirebaseFirestore][I-FST000001] Committing transaction: <LevelDbTransaction ExecuteQuery: 0 changes (0 bytes):>
2024-07-02 18:54:21.414973+0930 Pestle[3169:675913] 10.28.1 - [FirebaseFirestore][I-FST000001] (null)
2024-07-02 18:54:21.416005+0930 Pestle[3169:675913] 10.28.1 - [FirebaseFirestore][I-FST000001] (null)
2024-07-02 18:54:21.416580+0930 Pestle[3169:675913] 10.28.1 - [FirebaseFirestore][I-FST000001] (null)
2024-07-02 18:54:21.416990+0930 Pestle[3169:675913] 10.28.1 - [FirebaseFirestore][I-FST000001] (null)
2024-07-02 18:54:21.417255+0930 Pestle[3169:675913] 10.28.1 - [FirebaseAuth][I-AUT000017] Has valid access token. Estimated expiration date: 2024-07-02 09:54:46 +0000, current date: 2024-07-02 09:24:19 +0000
2024-07-02 18:54:21.417346+0930 Pestle[3169:675913] 10.28.1 - [FirebaseFirestore][I-FST000001] Committing transaction: <LevelDbTransaction NextMutationBatchAfterBatchID: 0 changes (0 bytes):>
2024-07-02 18:54:21.418158+0930 Pestle[3169:675913] 10.28.1 - [FirebaseFirestore][I-FST000001] Credential Changed. Current user: MYUSERID
2024-07-02 18:54:21.418571+0930 Pestle[3169:675913] 10.28.1 - [FirebaseFirestore][I-FST000001] (null)
2024-07-02 18:54:21.418639+0930 Pestle[3169:675913] 10.28.1 - [FirebaseFirestore][I-FST000001] (null)
2024-07-02 18:54:21.419327+0930 Pestle[3169:675913] 10.28.1 - [FirebaseFirestore][I-FST000001] (null)
2024-07-02 18:54:21.583952+0930 Pestle[3169:675913] 10.28.1 - [FirebaseFirestore][I-FST000001] (null)
2024-07-02 18:54:21.584695+0930 Pestle[3169:675913] 10.28.1 - [FirebaseAuth][I-AUT000017] Has valid access token. Estimated expiration date: 2024-07-02 09:54:46 +0000, current date: 2024-07-02 09:24:19 +0000
2024-07-02 18:54:21.584836+0930 Pestle[3169:675913] 10.28.1 - [FirebaseFirestore][I-FST000001] Committing transaction: <LevelDbTransaction NextMutationBatchAfterBatchID: 0 changes (0 bytes):>
2024-07-02 18:54:22.247576+0930 Pestle[3169:675972] 10.28.1 - [FirebaseFirestore][I-FST000001] Using index privateRecipes to execute target users/MYUSERID/privateRecipes|f:|ob:nameasc__name__asc
2024-07-02 18:54:22.361015+0930 Pestle[3169:675921] 10.28.1 - [FirebaseFirestore][I-FST000001] Committing transaction: <LevelDbTransaction ExecuteQuery: 0 changes (0 bytes):>
2024-07-02 18:54:22.361564+0930 Pestle[3169:675921] 10.28.1 - [FirebaseFirestore][I-FST000001] Using index privateRecipes to execute target users/MYUSERID/privateRecipes|f:|ob:nameasc__name__asc
User Code: Recevied Optional(40) documents
2024-07-02 18:54:22.545203+0930 Pestle[3169:675919] 10.28.1 - [FirebaseFirestore][I-FST000001] Committing transaction: <LevelDbTransaction ExecuteQuery: 0 changes (0 bytes):>
2024-07-02 18:54:22.545227+0930 Pestle[3169:675919] 10.28.1 - [FirebaseFirestore][I-FST000001] Committing transaction: <LevelDbTransaction Allocate target: 3 changes (186 bytes):
  - Put [query_target: canonical_id=betaAnnouncements|f:endRelevant>time(1719912256,937561988)|ob:endRelevantasc__name__asc target_id=13996] (0 bytes)
  - Put [target: target_id=13996] (162 bytes)
  - Put [target_global:] (24 bytes)>
2024-07-02 18:54:22.545240+0930 Pestle[3169:675919] 10.28.1 - [FirebaseFirestore][I-FST000001] Using full collection scan to execute query: Query(canonical_id=betaAnnouncements|f:endRelevant>time(1719912256,937561988)|ob:endRelevantasc__name__asc)
2024-07-02 18:54:22.545251+0930 Pestle[3169:675919] 10.28.1 - [FirebaseFirestore][I-FST000001] SDK will not create cache indexes for query: Query(canonical_id=betaAnnouncements|f:endRelevant>time(1719912256,937561988)|ob:endRelevantasc__name__asc), since it only creates cache indexes for collection contains more than or equal to 100 documents.
2024-07-02 18:54:22.545260+0930 Pestle[3169:675919] 10.28.1 - [FirebaseFirestore][I-FST000001] Committing transaction: <LevelDbTransaction ExecuteQuery: 0 changes (0 bytes):>
2024-07-02 18:54:22.548402+0930 Pestle[3169:675920] 10.28.1 - [FirebaseFirestore][I-FST000001] Committing transaction: <LevelDbTransaction Locally write mutations: 7 changes (70324 bytes):
  - Put [collection_parent: collection_id=users] (0 bytes)
  - Put [document_mutation: user_id=MYUSERID path=users/MYUSERID batch_id=1] (0 bytes)
  - Put [document_overlays: user_id=MYUSERID path=users/MYUSERID batch_id=1] (42059 bytes)
  - Put [document_overlays_collection_group_index: user_id=MYUSERID collection_group=users batch_id=1 path=users/MYUSERID] (0 bytes)
  - Put [document_overlays_collection_index: user_id=MYUSERID path=users batch_id=1 document_id=MYUSERID] (0 bytes)
  - Put [document_overlays_largest_batch_id_index: user_id=MYUSERID batch_id=1 path=users/MYUSERID] (0 bytes)
  - Put [mutation: user_id=MYUSERID batch_id=1] (28265 bytes)>
2024-07-02 18:54:22.549066+0930 Pestle[3169:675920] 10.28.1 - [FirebaseFirestore][I-FST000001] Committing transaction: <LevelDbTransaction NotifyLocalViewChanges: 0 changes (0 bytes):>
2024-07-02 18:54:22.549700+0930 Pestle[3169:675919] 10.28.1 - [FirebaseFirestore][I-FST000001] Committing transaction: <LevelDbTransaction NextMutationBatchAfterBatchID: 0 changes (0 bytes):>
2024-07-02 18:54:22.549746+0930 Pestle[3169:675919] 10.28.1 - [FirebaseFirestore][I-FST000001] Committing transaction: <LevelDbTransaction NextMutationBatchAfterBatchID: 0 changes (0 bytes):>
2024-07-02 18:54:22.549780+0930 Pestle[3169:675919] 10.28.1 - [FirebaseFirestore][I-FST000001] (null)
2024-07-02 18:54:22.549814+0930 Pestle[3169:675919] 10.28.1 - [FirebaseFirestore][I-FST000001] Creating Firestore stub.
2024-07-02 18:54:22.549908+0930 Pestle[3169:675919] 10.28.1 - [FirebaseAuth][I-AUT000017] Has valid access token. Estimated expiration date: 2024-07-02 09:54:46 +0000, current date: 2024-07-02 09:24:22 +0000
2024-07-02 18:54:22.566817+0930 Pestle[3169:675919] 10.28.1 - [FirebaseFirestore][I-FST000001] (null)
2024-07-02 18:54:22.566952+0930 Pestle[3169:675919] 10.28.1 - [FirebaseFirestore][I-FST000001] (null)
2024-07-02 18:54:22.865056+0930 Pestle[3169:675927] 10.28.1 - [FirebaseFirestore][I-FST000001] (null)
2024-07-02 18:54:23.420373+0930 Pestle[3169:675972] 10.28.1 - [FirebaseFirestore][I-FST000001] (null)
2024-07-02 18:54:23.525642+0930 Pestle[3169:675972] 10.28.1 - [FirebaseFirestore][I-FST000001] (null)
2024-07-02 18:54:23.525884+0930 Pestle[3169:675972] 10.28.1 - [FirebaseFirestore][I-FST000001] (null)
2024-07-02 18:54:23.525964+0930 Pestle[3169:675972] 10.28.1 - [FirebaseFirestore][I-FST000001] (null)
2024-07-02 18:54:23.526146+0930 Pestle[3169:675972] 10.28.1 - [FirebaseFirestore][I-FST000001] Committing transaction: <LevelDbTransaction Set stream token: 1 changes (11 bytes):
  - Put [mutation_queue: user_id=MYUSERID] (11 bytes)>
2024-07-02 18:54:23.526227+0930 Pestle[3169:675972] 10.28.1 - [FirebaseFirestore][I-FST000001] (null)
2024-07-02 18:54:23.622348+0930 Pestle[3169:675919] 10.28.1 - [FirebaseFirestore][I-FST000001] (null)
2024-07-02 18:54:23.622406+0930 Pestle[3169:675919] 10.28.1 - [FirebaseFirestore][I-FST000001] (null)
2024-07-02 18:54:23.622429+0930 Pestle[3169:675919] 10.28.1 - [FirebaseFirestore][I-FST000001] (null)
2024-07-02 18:54:23.622450+0930 Pestle[3169:675919] 10.28.1 - [FirebaseFirestore][I-FST000001] (null)
2024-07-02 18:54:23.622474+0930 Pestle[3169:675919] 10.28.1 - [FirebaseFirestore][I-FST000001] (null)
2024-07-02 18:54:23.622493+0930 Pestle[3169:675919] 10.28.1 - [FirebaseFirestore][I-FST000001] (null)
2024-07-02 18:54:23.622513+0930 Pestle[3169:675919] 10.28.1 - [FirebaseFirestore][I-FST000001] (null)
2024-07-02 18:54:23.622532+0930 Pestle[3169:675919] 10.28.1 - [FirebaseFirestore][I-FST000001] (null)
2024-07-02 18:54:23.622608+0930 Pestle[3169:675919] 10.28.1 - [FirebaseFirestore][I-FST000001] (null)
2024-07-02 18:54:23.622640+0930 Pestle[3169:675919] 10.28.1 - [FirebaseFirestore][I-FST000001] (null)
2024-07-02 18:54:23.622689+0930 Pestle[3169:675919] 10.28.1 - [FirebaseFirestore][I-FST000001] LocalStore Ignoring outdated update for betaAnnouncements/mailinglist. Current version: Timestamp(seconds=1706179480, nanoseconds=788218000)  Remote version: Timestamp(seconds=1706179480, nanoseconds=788218000)
2024-07-02 18:54:23.622722+0930 Pestle[3169:675919] 10.28.1 - [FirebaseFirestore][I-FST000001] Committing transaction: <LevelDbTransaction Apply remote event: 5 changes (214 bytes):
  - Put [document_target: path=betaAnnouncements/mailinglist target_id=0] (3 bytes)
  - Put [document_target: path=betaAnnouncements/mailinglist target_id=13996] (0 bytes)
  - Put [target: target_id=13996] (188 bytes)
  - Put [target_document: target_id=13996 path=betaAnnouncements/mailinglist] (0 bytes)
  - Put [target_global:] (23 bytes)>
2024-07-02 18:54:23.622895+0930 Pestle[3169:675919] 10.28.1 - [FirebaseFirestore][I-FST000001] Committing transaction: <LevelDbTransaction NotifyLocalViewChanges: 0 changes (0 bytes):>
2024-07-02 18:54:23.628326+0930 Pestle[3169:675921] 10.28.1 - [FirebaseFirestore][I-FST000001] (null)
2024-07-02 18:54:23.628370+0930 Pestle[3169:675921] 10.28.1 - [FirebaseFirestore][I-FST000001] Committing transaction: <LevelDbTransaction Release target: 2 changes (224 bytes):
  - Put [target: target_id=13996] (201 bytes)
  - Put [target_global:] (23 bytes)>
2024-07-02 18:54:23.920254+0930 Pestle[3169:675921] 10.28.1 - [FirebaseFirestore][I-FST000001] (null)
2024-07-02 18:54:23.920382+0930 Pestle[3169:675921] 10.28.1 - [FirebaseFirestore][I-FST000001] (null)
2024-07-02 18:54:24.013282+0930 Pestle[3169:675919] 10.28.1 - [FirebaseFirestore][I-FST000001] (null)
2024-07-02 18:54:24.013460+0930 Pestle[3169:675919] 10.28.1 - [FirebaseFirestore][I-FST000001] (null)
2024-07-02 18:54:24.042494+0930 Pestle[3169:675919] 10.28.1 - [FirebaseFirestore][I-FST000001] Committing transaction: <LevelDbTransaction Acknowledge batch: 10 changes (61641 bytes):
  - Delete [document_mutation: user_id=MYUSERID path=users/MYUSERID batch_id=1]
  - Delete [document_overlays: user_id=MYUSERID path=users/MYUSERID batch_id=1]
  - Delete [document_overlays_collection_group_index: user_id=MYUSERID collection_group=users batch_id=1 path=users/MYUSERID]
  - Delete [document_overlays_collection_index: user_id=MYUSERID path=users batch_id=1 document_id=MYUSERID]
  - Delete [document_overlays_largest_batch_id_index: user_id=MYUSERID batch_id=1 path=users/MYUSERID]
  - Delete [mutation: user_id=MYUSERID batch_id=1]
  - Put [document_target: path=users/MYUSERID target_id=0] (3 bytes)
  - Put [mutation_queue: user_id=MYUSERID] (13 bytes)
  - Put [remote_document: path=users/MYUSERID] (61625 bytes)
  - Put [remote_document_read_time: path=users snapshot_version=Timestamp(seconds=1719912263, nanoseconds=903104000) document_id=MYUSERID] (0 bytes)>
2024-07-02 18:54:24.044487+0930 Pestle[3169:675919] 10.28.1 - [FirebaseFirestore][I-FST000001] Committing transaction: <LevelDbTransaction NotifyLocalViewChanges: 0 changes (0 bytes):>
2024-07-02 18:54:24.044963+0930 Pestle[3169:675919] 10.28.1 - [FirebaseFirestore][I-FST000001] Committing transaction: <LevelDbTransaction NextMutationBatchAfterBatchID: 0 changes (0 bytes):>
2024-07-02 18:54:24.100841+0930 Pestle[3169:675921] 10.28.1 - [FirebaseFirestore][I-FST000001] (null)
2024-07-02 18:54:24.110291+0930 Pestle[3169:675921] 10.28.1 - [FirebaseFirestore][I-FST000001] (null)
2024-07-02 18:54:24.110412+0930 Pestle[3169:675921] 10.28.1 - [FirebaseFirestore][I-FST000001] (null)
2024-07-02 18:54:24.110439+0930 Pestle[3169:675921] 10.28.1 - [FirebaseFirestore][I-FST000001] (null)
2024-07-02 18:54:24.124135+0930 Pestle[3169:675921] 10.28.1 - [FirebaseFirestore][I-FST000001] Committing transaction: <LevelDbTransaction Apply remote event: 4 changes (61787 bytes):
  - Put [remote_document: path=users/MYUSERID] (61623 bytes)
  - Put [remote_document_read_time: path=users snapshot_version=Timestamp(seconds=1719912263, nanoseconds=903104000) document_id=MYUSERID] (0 bytes)
  - Put [target: target_id=40] (140 bytes)
  - Put [target_global:] (24 bytes)>
2024-07-02 18:54:24.126057+0930 Pestle[3169:675921] 10.28.1 - [FirebaseFirestore][I-FST000001] Committing transaction: <LevelDbTransaction NotifyLocalViewChanges: 0 changes (0 bytes):>

WillBishop avatar Jul 02 '24 09:07 WillBishop

Using .orderBy("dateModified") (which is a FIRTimestamp), returns 40 documents also, even though it's never been used as an orderBy parameter, so I don't think an index for it would exist yet?

WillBishop avatar Jul 02 '24 09:07 WillBishop

Thank you for the logs. This will be difficult to fix without a reproduction.

Some things you might want to try are:

  1. Disable client side indexing.
  2. Clear the cache.

If either of these fix the problem, that will be helpful information as well.

tom-andersen avatar Jul 02 '24 13:07 tom-andersen

@WillBishop

In addition to above, it is not enough to disableIndexAutoCreation, you will need to deleteAllIndexes.

Again, we would love a reproduction. But feedback on above changes will also be helpful.

tom-andersen avatar Jul 02 '24 18:07 tom-andersen

Disabling client side indexing and deleting the indexes does result in the documents being visible again. Once indexing is renabled, documents are again lost.

When documents are lost, they consistently do not return in future queries. But when indexes are deleted and rebuilt, a different set of documents are lost.

I'd love to provide a reproduction, but given i'm not even sure how the bug is produced I'm sure I'd be able to do that.

An interesting thing I found is I copied all the documents to a new collection and read from that. In that case it was able to read all 47 documents, but as soon as I switched to the original collection it returned only 42 documents.

Using the exact same query if I change it to .getDocuments(source: .server), I still get 42 documents only.

WillBishop avatar Jul 05 '24 08:07 WillBishop

Hi @WillBishop

This is useful information. Can you share some of the document content from both the set of missing documents, and the set of documents that actually show up?

wu-hui avatar Jul 08 '24 16:07 wu-hui

Privately I can yes, what's the best way to share them with you?

WillBishop avatar Jul 11 '24 00:07 WillBishop

Privately I can yes, what's the best way to share them with you?

You can create a private repo and invite me as a collaborator.

wu-hui avatar Jul 11 '24 15:07 wu-hui

Done! You'll find it here: https://github.com/WillBishop/MissingDocs

WillBishop avatar Jul 13 '24 07:07 WillBishop

Thanks, will take a look and try to reproduce soon.

wu-hui avatar Jul 15 '24 15:07 wu-hui

Hey, any update on this?

WillBishop avatar Aug 22 '24 11:08 WillBishop

Hi, sorry for the delay. We are overwhelmed by the projects going on within firestore, and i could not find anytime to work on this. That being said, I will try to spend some time to work on this in the next couple weeks.

wu-hui avatar Aug 22 '24 13:08 wu-hui

I spent some time trying to reproduce this and failed. I did find something interesting though:

This line: 2024-07-02 18:39:36.008612+0930 Pestle[3064:660932] 10.28.1 - [FirebaseFirestore][I-FST000001] Query: Query(canonical_id=users/MYUSERID/privateRecipes|f:|ob:__name__asc|lt:f), scans 773 local documents and returns 46 documents as results.

Says it is doing a collection scan with no filter/orderBy (other than name which is document id), and there are a total of 773 documents in the SDK cache. Yet, only 46 were returned as result. How can this be? Isn't 773 the expected count?

This triggers the auto indexing process, and leads to your second query with orderBy("name") run with a newly created index, which is missing documents as you see.

So I think there might be at least two issues here, and they both seem to have something to do with the contents in the SDK cache being in a state they are not supposed to be.

In the short term, you can proceed with disabling auto indexing, which should make it possible for you to move forward.

At this point, I'd like to request a minimum reproduction app from you if possible. You can strip out all other logic and only leave this recipe collection to be populated and then queried. We'd be really grateful if you could take the effort to make our product better.

You can update the private repo with your reproducing App. In terms of data, you can create a new free firebase project, and share the googleservices.info file in the private repo as well.

Thanks!

wu-hui avatar Aug 23 '24 17:08 wu-hui

Hello, I'm working on a project and we are noticing similar behavior to this, but on Android. Commenting here because I've not been able to find existing reports that sound similar to this issue so this might help identify the problem.

From my discoveries, it's consistently reproducible when the cache is at or above it's "maximum size" (i.e. we've set our cache size limit to be 16 MB and the DB file is reported at or above that size) and the internet is disconnected. The behavior we'll see is, once said conditions are met, our app will write a new document to collection x and our snapshot listeners/UI reflect the new document, however if our app goes back to query collection x with an orderBy clause (i.e. latest created document in collection x), the document that was just created is not returned. It will actually return the document created before that document in the collection, almost as if the document that was just created while disconnected was never actually created.

It's important to note that we do not see this behavior when connected to internet.

A log statement that jumps out to me that I also see in this report seems suspect...

[FirebaseFirestore][I-FST000001] SDK will not create cache indexes for query: Query(canonical_id=betaAnnouncements|f:endRelevant>time(1719912256,937561988)|ob:endRelevantasc__name__asc), since it only creates cache indexes for collection contains more than or equal to 100 documents.

But I also see that regardless of internet connectivity, but I wonder if it plays a role in this issue.

griffenclark123 avatar Nov 21 '24 13:11 griffenclark123