`getRange` will return the sharedStructuresKey, but this is not typed in Typescript

[vedantroy]

The following code:

    const pages = this.pages
      .getRange({ end: MAXIMUM_KEY })
      // Need to avoid un-intuitive behavior
      // .asArray.filter((entry) => entry.key !== STRUCTURES_KEY);

will return an all the entries in the database, of which one of the entries will have a key that is a symbol. (If you are using the sharedStructuresKey option).

This was causing behavior that I misdiagnosed in: https://github.com/kriszyp/lmdb-js/discussions/180 The Typescript definition types the key as a string, which is not correct (since it could be a symbol). It's also a bit unexpected, since I feel like the shared structures key is expected to not really be explicitly accessed?

[kriszyp]

I understand this concern. The shared structures key needs to be stored somewhere in the database. And I think the implicitly suggested alternative is to automatically filter it out. However, since lmdb-js is relatively low-level, I wanted to avoid such automatic behavior and instead make sharedStructuresKey very explicit and understood by user, which is why the API requires that you specify the key yourself, so you know exactly where it is in the database, and can decide if and when you ever might want to explicitly access that key or avoid that key (for example, you might want to include that key if you are copying all entries).

As far as the best way to avoid this entry, I'd suggest using a "low" key for the start. Booleans come before all numbers and strings, but after symbols, so using getRange({ start: false, end: MAXIMUM_KEY }), should skip past symbols if you used the suggested symbol as the shared structures key, but include all other keys. This would certainly have better performance than filtering (also FYI, you do not need to convert to an array before filtering, you can directly filter the RangeIterable that is returned from getRange and this maintains lazy access to the data without requiring eager loading of all the data in a range).

Again, I understand your concern, and perhaps it would be better or useful to have a "default" start key, that would skip past shared structures key. I could add this as an option (don't think I would make this the default without a major version change).

[vedantroy]

I understand this concern. The shared structures key needs to be stored somewhere in the database. And I think the implicitly suggested alternative is to automatically filter it out. However, since lmdb-js is relatively low-level, I wanted to avoid such automatic behavior and instead make sharedStructuresKey very explicit and understood by user, which is why the API requires that you specify the key yourself, so you know exactly where it is in the database, and can decide if and when you ever might want to explicitly access that key or avoid that key (for example, you might want to include that key if you are copying all entries).

As far as the best way to avoid this entry, I'd suggest using a "low" key for the start. Booleans come before all numbers and strings, but after symbols, so using getRange({ start: false, end: MAXIMUM_KEY }), should skip past symbols if you used the suggested symbol as the shared structures key, but include all other keys. This would certainly have better performance than filtering (also FYI, you do not need to convert to an array before filtering, you can directly filter the RangeIterable that is returned from getRange and this maintains lazy access to the data without requiring eager loading of all the data in a range).

Again, I understand your concern, and perhaps it would be better or useful to have a "default" start key, that would skip past shared structures key. I could add this as an option (don't think I would make this the default without a major version change).

Your reasoning makes a lot of sense! I guess this behavior is the natural / expected behavior of the system. Maybe just worth documenting this somewhere in the README next to the option. (e.g: "Note: structures keys aren't special, so if they might be returned in range queries, if the starting key is sufficiently low. I recommend using a starting key of false in order to avoid reading the structures key.")

Not sure how to fix the Typescript typings, you could make it a union of string | Symbol, but that is incorrect since the structures key could theoretically be anything.