deployadactyl icon indicating copy to clipboard operation
deployadactyl copied to clipboard

Canary Tests

Open marcosrmendezthd opened this issue 8 years ago • 6 comments

Please add support for specifying a number of paths (with query strings) to verify that a deployment actually works

marcosrmendezthd avatar Oct 24 '16 17:10 marcosrmendezthd

Hi @marcosrmendezthd, can you elaborate on "specifying a number of paths". Maybe provide us with an example of what you're looking to do?

trayo avatar Oct 24 '16 17:10 trayo

Hi @trayo . Sorry for the delay. The idea is to allow a hook into the deployment validation. Could add a validation script (taking the app domain, url, etc as params $1, $2) and/or a number of paths to verify.

Some validations are more complex than others, but think of the simple case where after deploying something, maybe to a cold route, maybe a blue/green route, one can verify that the service is up and running; /healtcheck, /synthetictransaction?param1=value1, ...

The script option will allow more complex interactions which can't be handled by simply calling a URL and expecting a successful response. Think about interacting with multiple micro services or canary transactions.

marcosrmendezthd avatar Oct 31 '16 17:10 marcosrmendezthd

We talked about a possible solution. Would it work for you if we were to add an event that is emitted before the push is finished around this line.

You could then write an event handler that listens for that event and runs any kind of script. If the the script fails the health check then you could send an error back and we could trigger a rollback.

Do you have an example of a complex validation?

trayo avatar Nov 02 '16 18:11 trayo

@trayo adding the event helps. I was hoping not to have to maintain a fork to implement this, but maybe we'll have to anyway. Have to include support in the yaml, maybe add hooks for blue/green, etc..

I'm hoping once our process standardizes a little bit, we can contribute this back. We'd want to empower devs to simply add the validation configuration to their yaml. This will vary from team to team... app to app.

A complex validation would be something like getting an oauth token and using it to access a resource.

marcosrmendezthd avatar Nov 03 '16 03:11 marcosrmendezthd

I see what you mean now. By

add the validation configuration to their yaml

do you mean the Cloud Foundry manifest? We tried on purpose to not modify the manifest in order to prevent confusion to the developers when they are trying to find documentation on fields that are Deployadactyl specific.

Other than that, the way our developers solve the health check problem is by using a CI tool, like jenkins or concourse, to deploy their application to a non-production space. Then the pipeline will run the health check scripts and if those pass, push their app to production.

trayo avatar Nov 03 '16 17:11 trayo

I think he intends that the config.yaml for Deploydactyl be updated to allow for the specification of the hooks. IT would probably need to be something that can be set at an env. level generically and then allow for app specific overrides.

jonathannewell avatar Jan 12 '17 17:01 jonathannewell