Add TTL Policy to Firestore

[AlbertMoser]

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment. If the issue is assigned to the "modular-magician" user, it is either in the process of being autogenerated, or is planned to be autogenerated soon. If the issue is assigned to a user, that user is claiming responsibility for the issue. If the issue is assigned to "hashibot", a community member has claimed the issue already.

Description

There is a new feature, firestore ttl policy (https://cloud.google.com/firestore/docs/ttl). It is already in pre-ga, can we add this to the beta provider?

New or Affected Resource(s)

it should be something like google_firestore_ttl_policy

Potential Terraform Configuration

resource "google_firestore_ttl_policy" "default" {
  collection_group = "my_collection_group"
  timestamp_field = "expireAt"
}

References

Docu: https://cloud.google.com/firestore/docs/ttl API Docu: https://firebase.google.com/docs/firestore/reference/rest/v1/projects.databases.collectionGroups.fields

[melinath]

This would likely be a new resource called google_firestore_field. It looks like the TTL property is an object that needs to be explicitly set to an empty (but present) value in order to enable TTL on the field - which is a bit odd but which I believe should be doable.

[melinath]

hmm, actually, the Field resource doesn't seem to have a Create method - it may not be a real resource that we would want to support directly.

google_firestore_ttl_policy also doesn't map to a real API resource, as far as I can tell.

Ideally, we would add support for this feature to a resource that can be managed fully with CRUD operations - that's simplest to implement and maintain. Does such a a resource exist?

[AlbertMoser]

Thanks for looking into it. Concerning the last part:

Ideally, we would add support for this feature to a resource that can be managed fully with CRUD operations - that's simplest to implement and maintain. Does such a a resource exist? Is that something I should look into or was it more of a general question?

[melinath]

Is that something I should look into or was it more of a general question?

If you have the time and inclination to look into it, that would be great; otherwise it's a more general question I suppose! :-) No worries either way.

[AymericPlanche]

The TTL policy is not a pre-GA offering anymore. The feature is production ready.

[bschaatsbergen]

@melinath, I would like to pick this up but we should first agree on the implementation. As you've stated - it seems like this doesn't directly map to an API call, nor is there an existing Terraform resource to manage collections.. what are your thoughts on this?

I'm up for handwriting a new resource too as proposed by @AlbertMoser.

resource "google_firestore_ttl_policy" "default" {
  collection_group = "my_collection_group"
  timestamp_field = "expireAt"
}

[OscarVanL]

Thanks for looking at this, I just wanted to point out that this feature was also added to "Firestore in Datastore mode", which is often called Datastore in docs/packaging. See here: https://cloud.google.com/datastore/docs/ttl

As far as I am aware, the two products function very similarly as they are both Firestore, just operating in different modes. (Firestore is more geared to making client apps, Datastore is more geared to backends).

For example, here's the equivalent terraform for defining indexes in Firestore and Datastore. So this feature might need to be duplicated for datastore too.

[flovouin]

Does this need approval before implementation can start? I may have missed something but it's not clear to me whether the solution suggested by @AlbertMoser and @bschaatsbergen has been accepted, or if the shape of the resource is still being discussed.

Ideally, we would add support for this feature to a resource that can be managed fully with CRUD operations - that's simplest to implement and maintain. Does such a a resource exist?

By looking at the REST API documentation, there are GET and PATCH operations to get and set the configuration of a field within a collection. This configuration allows "enabling TTL on the field", but also disabling indexing, which is related to #11419 as @melinath pointed out. Not exactly CRUD-like, but a field in a collection can still be uniquely identified, read, and updated. I'm not entirely sure how creation would work. Deletion could disable the TTL configuration (if set) and revert the index config to the ancestor field for example. Although there's no clear concept of deleting a field, this would revert the field to the default configuration, and make it look like the configuration never existed.

From the documentation:

indexConfig If unset, field indexing will revert to the configuration defined by the ancestorField. ttlConfig Setting or unsetting this will enable or disable the TTL for documents that have this Field.

[pcostell]

I can take ownership of implementation on this. @flovouin your interpretation of the REST API is correct. The Fields API was (perhaps over-complicatedly) designed to represent that all indexing configurations implicitly exist (due to Firestore's auto-indexing semantics). I will propose that creation is a PATCH which sets the configuration of the field (and thus the indexing configuration no long "inherits from the ancestor field"). Deletion would then clear the override (reverting the field configuration back to it's inherited form) and delete the TTL config.

[remko]

Does this work with Datastore in Firestore mode (as mentioned earlier in the thread), or should a different request be filed?

[github-actions[bot]]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues. If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.