Firebase query orderByKey().limitToFirst(n).startAfter() returns n-1 items iso n

[timalenustmf]

[READ] Step 1: Are you in the right place?

yes

[REQUIRED] Step 2: Describe your environment

• Operating System version: Ubuntu 22.04 lts
• Firebase SDK version: 11.11.0
• Firebase Product: realtime database (auth, database, storage, etc)
• Node.js version: 16.20.2
• NPM version: 8.19.4

[REQUIRED] Step 3: Describe the problem

Firebase query of type: admin.database.Database.ref('collection').orderByKey().limitToFirst(100).startAfter(lastKey) only returns 99 items iso 100 even if more keys are in the firebase realtime database.

On version 11.4.1 of firebase-admin this query behaves as expected and 100 items are returned.

Steps to reproduce:

What happened? How can we make the problem occur? This could be a description, log/console output, etc.

Relevant Code:

let query;
let lastDocument = null;
let lastBatchSize: number = 100;

while (lastBatchSize === 100) {
    query = database.ref('collection').orderByKey()
      .limitToFirst(100);

    if (lastDocument !== null) {
      query = query.startAfter(lastDocument.key);
    }

    const snapshot = await query.get();
    const keys = Object.keys(snapshot.val() || {});
    const docs = keys.map((key) => snapshot.child(key));

    if (docs.length === 0) break;
    console.log("docs length " + docs.length); //prints 99 on 11.11.0, prints 100 on 11.4.1

    if (docs.length) {
      lastDocument = docs[docs.length - 1];
    }
    lastBatchSize = docs.length;
  }

[google-oss-bot]

I couldn't figure out how to label this issue, so I've labeled it for a human to triage. Hang tight.

[willemjanvankranenburg]

We experience the same problem, but not on every firebase environment.

Every environment we have has the exact same configuration and same indexes, but if we run a script with the same query on one environment it returns exactly the limit, but on other environments it returns limit-1.

We use firebase-admin 11.8.0 and nodejs 16

[tannerstern]

I am experiencing this as well with firebase-admin 12.0.0 and Node v20.10.0, in a script that is crawling through a large dataset using pagination.