Have a way to differenciate errors due to requests errors vs missing flags

[AdrianB-sovo]

trafficstars

When doing:

flags = flagsmith.get_environment_flags()
environment_flags.is_feature_enabled("foo")

• If the feature flag does not exist, the following error is raised: FlagsmithClientError("Feature does not exist: <my_feature>")
• If the environment failed to do an API request, the following error is raised: FlagsmithClientError("Feature does not exist: <my_feature>")

I would like to raise different alerts in those cases, because the fix to the problems are not the same (i.e., waiting for it to resolve itself vs adding the flag or removing the code using the flag).

Could we have a way to differentiate them?

Suggestions

• Flags could have an attribute to store the fact that the Flags were created following a FlagsmithAPIError, then, different errors could be raised in Flags.get_flag() using this attribute.

or

• The default_flag_handler() could have a parameter that would indicate the type of error.

[matthewelwell]

Hi @AdrianB-sovo, thanks for this. It's an interesting point and I certainly understand the request. As per your first suggestion, I think that adding a parameter to the Flags object to indicate if the request to the API was successful or not would be the best way to solve this.

Would you be up for submitting a PR for this? Otherwise we will add this to our backlog.

[AdrianB-sovo]

Hi @matthewelwell 🙂, I don't think I'll have time to work on a PR for this.

[matthewelwell]

@AdrianB-sovo no problem! I will include it in our backlog.

[tushar5526]

@matthewelwell I plan to work on this issue - is this still open or picked internally?

[matthewelwell]

@tushar5526 sure, it's still available to work on, let me know if you have any questions or need any guidance.

[tushar5526]

Hey @matthewelwell - I was trying to reproduce this, but it seems we raise a FlagsmithAPIerror at https://github.com/Flagsmith/flagsmith-python-client/blob/main/flagsmith/flagsmith.py#L339, hence the code will halt at flags = flagsmith.get_environment_flags().

It seems the above error is only reproducible in offline mode or local evaluation when the API call fails and we fallback to local evaluation or offline mode ?

[matthewelwell]

I'm sorry @tushar5526 , I'm struggling to understand what you're asking here. Can you illustrate with code / example scenarios perhaps?

My understanding of this issue is that if, in any evaluation mode, you do something like the following, then a FlagsmithClientError is raised in all failure scenarios. What we'd like to happen is that, if flag "foo" does not exist, then a new exception is raised (e.g. FlagDoesNotExist).

flags = flagsmith.get_environment_flags().is_feature_enabled("foo")

Are you saying that some part of this assumption is not correct?

[tushar5526]

Hey @matthewelwell

My understanding of this issue is that if, in any evaluation mode, you do something like the following, then a FlagsmithClientError is raised in all failure scenarios

Incase of network errors, it raises a FlagsmithAPIError instead of a FlagsmithClientError.

I have also attached screenshots below. I am using a basic Flagsmith cloud account.

Client Error if I try to get status of a non-existing flag

FlagsmithAPIError when I turn the wifi off

If you follow through the implementation of get_environment_flags you will see it raises a the FlagsmithAPIError for network errors, therefore I thought the behaviour described in the issue would only be reproducible in local or offline mode.

[matthewelwell]

Gotcha, ok - so technically this issue could probably be resolved as 'cannot reproduce'... but, I do think that adding a new exception (that inherits from FlagsmithClientError) which is raised when a flag is not found could be a good addition still.