cloud_controller_ng icon indicating copy to clipboard operation
cloud_controller_ng copied to clipboard

Can't rollback to revision N if rollback to revision N was previously executed and cancelled before deployment is completed

Open heyjcollins opened this issue 4 years ago • 3 comments

Issue

Once a rollback to a previous revision has been cancelled before the rollback has completed, all attempts to re-run that same rollback fail because the backend seems to think that the code and config to be rolled back to is the same as the currently deployed revision.

Context

The CLI Team is implementing the revisions and rollback CLI commands. In the output of cf revisions APP_NAME, a (deployed) decorator is displayed adjacent to the revision number(s) that's currently deployed as per /v3/apps/:guid/revisions/deployed. Click HERE to view screenshot of the UX.

We found a specific sequence where the backend is preventing rollback to a previous revision because it is the same as the deployed revision, but that is clearly not the case.

Once it's in this state, it isn't possible to rollback to the target revision.

Steps to Reproduce

Env/Setup:

cf api
API endpoint:   https://api.arsicault.cli.fun
API version:    3.88.0
cf version
cf version 7.1.0+e3e7571bc.2020-09-26

Repro: Using the latest edge version of the v7 CLI

  1. open two terminals and target the same env/org/space
  2. In term 1:
    • cf push app_name 3X
    • cf rollback app_name --revision 1 -f
  3. In term 2:
    • before the term 1 rollback has completed
    • cf cancel-deployment app_name
  4. In term 1:
    • cf rollback app_name --revision 1 -f

Expected result

I'd expect the rollback to succeed without error since the previous rollback to revision 1 was cancelled and the currently deployed revision is revision 3 - which doesn't share the same droplet as revision 1. A new revision cloudfoundry/capi-release#4 was created during the cancelled rollback, but the system still appears to think the currently deployed revision is revision cloudfoundry/capi-release#3 cf curl /v3/apps/$(cf app app_name --guid)/revisions/deployed However, it's responding as if it believes the rollback was completed successfully and that I'm trying to rollback from revision #4 which does have the same code and configs as revision cloudfoundry/capi-release#1

Current result

the rollback fails with the following error:

cf rollback dora3 --revision 1 -f
This command is in EXPERIMENTAL stage and may change without notice

Rolling back to revision 1 for app dora3 in org o / space s as admin...
Creating deployment for app dora3...

Unable to rollback. The code and configuration you are rolling back to is the same as the deployed revision.
FAILED

verbose_rollback_failure_output.txt

Screen Shot 2020-10-09 at 5 25 13 PM

heyjcollins avatar Oct 09 '20 22:10 heyjcollins

We have created an issue in Pivotal Tracker to manage this:

https://www.pivotaltracker.com/story/show/175213605

The labels on this github issue will be updated when the story is started.

cf-gitbot avatar Oct 09 '20 22:10 cf-gitbot

I wrote a script to reproduce this behavior:

#! /usr/bin/env bash

app_name=app_name

cf -v
cf d $app_name -f

for _ in {1..3}; do
  cf push $app_name -i 5
done

cf rollback $app_name --version 1 -f &
sleep 3
cf cancel-deployment $app_name
cf rollback $app_name --version 1 -f
wait

Here are the resulting revisions:

cf revisions app_name
This command is in EXPERIMENTAL stage and may change without notice

Getting revisions for app app_name in org org / space space as admin...

revision      description                                        deployable   revision guid                          created at
4             New droplet deployed. Rolled back to revision 1.   true         e6920c41-2e7a-4c2b-b608-73435579e718   2020-11-18T23:34:05Z
3(deployed)   New droplet deployed.                              true         79b9ced5-7c35-4e78-b503-efeedb022d76   2020-11-18T23:33:47Z
2             New droplet deployed.                              true         317e80d0-482d-4846-901b-b414a302436c   2020-11-18T23:32:57Z
1             Initial revision.                                  true         cee5b006-3f68-421a-a5ee-8091df7cc91a   2020-11-18T23:32:08Z

I believe what's happening here is the first rollback creates a new revision (4). Canceling the deployment doesn't create a new revision, but keeps the app's instances on the previous revision (3).

It looks like we intentionally block deploying an older revision if the latest revision matches the revision you are deploying: https://www.pivotaltracker.com/story/show/164410170. Not sure why deploying a revision isn't idempotent.

Aside: It's a little weird that the error message here mentions "rollbacks". I don't think we talk about rollbacks anywhere else on the API or in the documentation. Isn't it also possible that the revision you are deploying is "ahead" of the revision you are currently on?

Gerg avatar Nov 18 '20 23:11 Gerg

We have created an issue in Pivotal Tracker to manage this:

https://www.pivotaltracker.com/story/show/175791061

The labels on this github issue will be updated when the story is started.

cf-gitbot avatar Nov 19 '20 00:11 cf-gitbot