Need an option to ignore specific HTTP errors for specific change files

[oridool]

I'm trying to use this project (well done BTW!!), but bumped into an issue because I don't have enough control on the execution behavior. On Opensearch, I'm trying to add an index policy. The problem is that some clusters already have this policy, while new ones don't have it. The API is: PUT _plugins/_ism/policies/<policy_name> However, this API is not like other ones in OpenSearch,: it fail with error 409 in case that such policy already exists (rather than just perform insert-or-update like in other entities) . See https://github.com/opensearch-project/index-management/issues/266

If I try to first delete the policy, then it will succeed on the clusters already have the policy, but the DELETE Api will fail with 404 on the cluster that doesn't have the policy.

DELETE _plugins/_ism/policies/<policy_name> --> 404 not exist

Right now, I have no solution for this.

Is there any option to just 'ignore' specific errors for specific scripts? For example, if my delete command returns 404 (not found), I want to consider it as a success and continue with the rest of the rest of the changes.

[xtermi2]

No, there is no feature at the moment to ignore errors in migration scripts. Do you have a multi cluster elasticsearch setup? Or how can it happen, that in a new cluster the index policy does not exist but elasticsearch-evolution has this script already marked as executed?

But I think this feature can be added easily e.g. with an 'annotation' in the migration scripts, maybe like this:

@IgnoreHttpStatusCode 409
PUT _plugins/_ism/policies/<policy_name>
...

What's your opinion @oridool

[oridool]

@xtermi2 , Thanks! That seems like a great solution. And I guess that more options can be added in the future, by adding more lines starting with '@'.

The reason for having different clusters is that I'm not starting my project from zero. Some clusters are already running in production, with existing changes (deployed by another automation tool). Other clusters, used for dev/test, are empty and need to run all the changes from scratch.

Another option for the syntax can be a single java-like annotation (even using parameters in the future):

@ExecuteOptions( enabled=${step17IsEnabled}, ignoreHttpStatusCodes={409,410}, retries=3)

Or a JSON:

{ "executeOptions" : [
  { "enabled'" : "${step17IsEnabled}" }, 
  { "ignoreHttpStatusCodes" : [409,410] },
  { "retries" : 3 }
 ]
}

Of course that what you suggested can also work great and maybe more simple to implement.

[oridool]

@xtermi2 , I gave it another thought, and I think that adding parameters into the scrips files will not be that good because it will change the checksum. There are cases where you'd want to change only the parameter values and you don't want it to affect the checksum. Therefore, I now think that the best approach for adding extra parameters for script execution would be to use an external configuration file, referencing all the migration script names, and providing execution options. Something like Liquibase changelog file is providing. WDYT?

[oridool]

Hi @xtermi2 , Please see my PR: https://github.com/senacor/elasticsearch-evolution/pull/144 It's not perfect, but I think it is a good start for the direction we talked about. I didn't complete the tests etc, but I did a basic check in my project and it seems to work well. WDYT?