Flagsmith
Allow for large Segment rule values
Is your feature request related to a problem? Please describe.
Sometimes people want to provide a large (> 100) list of UUIDs or other identifiable datatype to target Identities. For various reasons (e.g. local evaluation mode with limited user context data in memory on the server), they might not be able to use traits.
Describe the solution you'd like
Allow the uploading of large lists of data that back segment rules like IN. This would be stored in a new type of data store such as Amazon S3 (or a new table in Postgres that accepts larger data values) and merged into memory of the flag engine at runtime.
We would need to design how the data is provided/ingested (via API, CSV upload etc) and what the performance considerations/limits of this approach might be.
Describe alternatives you've considered
Traits are the only way to achieve this sort of thing currently but sometimes disparate data makes them unworkable.
If we plan on having the Edge Proxy sync these segments, it might be worth considering if/how we could design this to support one/two-way sync of user traits. Currently, when using local flag evaluation, traits are limited in two ways:
- Traits stored by Flagsmith are not accessible in local flag evaluation.
- Applications using local flag evaluation do not propagate trait updates to Flagsmith.
The second limitation can be somewhat mitigated by making an explicit remote flag evaluation request to Flagsmith, but this is not very useful if server-side applications cannot normally access this data.
I mention this now in case we want applications to receive traits in addition to the list of identities that make up a cohort. And if we're doing that, it might be worth designing it in a way that could support two-way trait sync in the future.
It's not yet clear to me whether two-way trait sync is desirable because of all the complications that brings - at the very least it does not belong as part of the MVP. I can think of some reasons to implement it later:
- The Flagsmith frontend will show the actual trait and segment values for all identities. This makes using the frontend more desirable and adds more value to each user seat.
- Allows server-side SDKs to exclusively use local flag evaluation, which simplifies implementation and mitigates concerns about API call limits in SaaS. This eliminates an otherwise critical design decision that customers need to make early on about their Flagsmith implementation.
Discussing with @khvn26, we think that one way that we could achieve this by accepting a list of identifiers, upserting them in the background and adding a 'hidden trait' and segment to be able to segment those users.
This does, of course, hit the limitation mentioned by @rolodato above regarding the fact that traits are not sync'd from the server to the client. For now, I am suggesting that we restrict the functionality to remote evaluation and edge proxy only which should give us a little more flexibility on the implementation, rather than having to consider local evaluation mode. Probably, by implementing in the Edge Proxy, we will also determine a solution for local evaluation mode, but it is a little complicated to consider all options initially.
