Firestore: Typed Query API

[mihail-brinza]

Regarding the Firestore Client, I am curious to know what is the general opinion about having a Typed Firestore client, i.e., instead of writing the field name as string in the query, we use a lambda expression to select the field.

Below there is a comparison between the current implementation and a new version that I have implemented: Github repo.

// current implementation
Query adultsFromPortugalQuery = collection
    .OrderBy("Age")
    .WhereGreaterThanOrEqualTo("Age", 18)
    .WhereEqualTo("Location.home_country", "Portugal")
    .OrderByDescending("Age");

// typed client implementation
TypedQuery<User> adultsFromPortugalQuery = collection
    .OrderBy(user => user.Age)
    .WhereGreaterThanOrEqualTo(user => user.Age, 18)
    .WhereEqualTo(user => user.Location.Country, "Portugal")
    .OrderByDescending(user => user.Age);

This new implementation brings several advantages:

1. We can only compare a field with a value of the same type, e.g., .WhereGreaterThanOrEqualTo(user => user.Age, 18) only accepts an int, whereas the current implementation accepts a value of any type.
2. Automatic path creation, i.e., the path is calculated from the lambda expression, automatically considering custom names from [FirestoreProperty("home_country")]. This way, .WhereEqualTo(user => user.Location.Country, "Portugal") is translated to .WhereEqualTo("Location.home_country", "Portugal")

My implementation is a wrapper around the official client, however, ideally, the implementation should be in the official client as overloads to the current existing APIs, giving developers both typed and non-typed options.

From my understanding, the objective of the Firestore clients is to provide a similar experience across all languages, however, for people that come from EntityFramework, MongoDb, the typed approach feels more natural and would be a great improvement, but I would love to hear other opinions.

For more information about my typed implementation, please refer to Github repo.

[amanda-tarafa]

I have little experience in Firestore, but I guess one of my main concerns would be about latency. Have you benchmarked this against Google.Cloud.Firestore? I would expect some overhead from your implementation over Google.Cloud.Firestore but I guess usability will depend on how much that overhead is.

@jskeet will certainly be able to comment more once he's back and I'll flag this issue internally to some other folks who might have thoughts about this.

[mihail-brinza]

I've run some benchmarks locally against the Firestore Emulator. The source code of the benchmarks can be found here.

1. Lambda Field Translator Benchmark

Below are the results of translating a SimpleField u => u.FirstName and a NestedField u.Location.Country.

|      Method |     Mean |     Error |    StdDev | Allocated |
|------------ |---------:|----------:|----------:|----------:|
| SimpleField | 1.503 us | 0.0064 us | 0.0054 us |     696 B |
| NestedField | 2.936 us | 0.0116 us | 0.0103 us |   1,272 B |

2. Single Entity Benchmark

This benchmark consists of:

1. Creating a user.
2. Getting the the user by id
3. Querying the users by Country.

|         Method |     Mean |    Error |   StdDev | Allocated |
|--------------- |---------:|---------:|---------:|----------:|
|    TypedClient | 41.48 ms | 0.262 ms | 0.205 ms |     54 KB |
| OfficialClient | 41.79 ms | 0.644 ms | 0.571 ms |     51 KB |

As we can see, the mean time is virtually the same, only the allocated memory is higher because of the "lambda to field name" translation

3. Multiple Entities Benchmark

This benchmark consists of:

1. Generating a list with numberOfUsers Users.
2. Insert the users in batch.
3. Getting the the user by id.
4. Query the users by Age and Coutry.
5. Delete all users.

|         Method | numberOfUsers |     Mean |   Error |  StdDev | Allocated |
|--------------- |-------------- |---------:|--------:|--------:|----------:|
| OfficialClient |             1 | 122.7 ms | 0.67 ms | 0.59 ms |     83 KB |
|    TypedClient |             1 | 123.1 ms | 1.00 ms | 0.83 ms |     84 KB |
|    TypedClient |             5 | 125.6 ms | 1.32 ms | 1.17 ms |    167 KB |
| OfficialClient |             5 | 125.6 ms | 1.15 ms | 1.07 ms |    164 KB |
|    TypedClient |            10 | 128.5 ms | 0.71 ms | 0.59 ms |    271 KB |
| OfficialClient |            10 | 128.8 ms | 0.65 ms | 0.58 ms |    266 KB |
|    TypedClient |            50 | 152.0 ms | 1.01 ms | 0.99 ms |  1,089 KB |
| OfficialClient |            50 | 153.7 ms | 1.41 ms | 1.25 ms |  1,076 KB |
|    TypedClient |           100 | 184.2 ms | 2.09 ms | 1.74 ms |  2,148 KB |
| OfficialClient |           100 | 184.4 ms | 2.81 ms | 2.49 ms |  2,101 KB |
|    TypedClient |           200 | 243.7 ms | 2.66 ms | 2.22 ms |  4,158 KB |
| OfficialClient |           200 | 247.6 ms | 1.79 ms | 1.50 ms |  4,132 KB |
|    TypedClient |           400 | 370.4 ms | 5.99 ms | 5.00 ms |  8,247 KB |
| OfficialClient |           400 | 371.3 ms | 3.75 ms | 3.13 ms |  8,206 KB |

The differences between them are very small, the typed one allocates more memory because of all the objects that it must create to encapsulate the official Firestore Client, but other than that, the times are very similar.

[jskeet]

Hi @mihail-brinza, apologies for the delay getting back to you.

I've been talking with the Firestore team, and we think it would be best for you to maintain this as a separate library. That way we don't get ever-further from consistency between different languages in terms of the official client, and we don't incur an increased maintenance burden for the official client library.

If there are internals that need to be exposed in order to allow this separate layering, I'd be happy to at least consider feature requests like that - although the fact that you've already got a library suggests it's unnecessary, unless you've been using reflection etc to work around it.

[mihail-brinza]

Hi, thank you for your feedback @jskeet.

I understand that this approach would add inconsistencies between different languages. I am not using reflection at all, so I can easily maintain it.