google-cloud-python icon indicating copy to clipboard operation
google-cloud-python copied to clipboard

Update service fails when removing items from traffic field

Open steve148 opened this issue 1 year ago • 5 comments

Environment details

  • macOS Ventura v13.5
  • Python 3.11.4
  • pip version 23.2.1
  • google-cloud-run version 0.9.1

Steps to reproduce

  1. Run the code sample below with some variables filled in.

Expected result

Service is updated with tags being removed from revisions which don't have an open Pull Request anymore.

Actual result

Script fails complaining revision receiving 100% of traffic exists with different configuration, although the snippet doesn't change that item in the traffic field.

Background

Was looking at this tutorial in the docs and wanted to test out the clean up part. I believe the code to be quite similar, but I wanted to use the client library for cloud run.

https://cloud.google.com/run/docs/tutorials/configure-deployment-previews

Only thing I could find relating to the error seems to be this terraform issue, but I don't know how much it applies here.

https://github.com/hashicorp/terraform-provider-google/issues/13410

Code example

from github import Auth, Github
from github.PullRequest import PullRequest
from google.cloud.run_v2 import Service, ServicesClient

service_name = "projects/pogon-155405/locations/us-central1/services/my-cloud-run-service"
repository_name = "perpetua1/my-repo"


def main():
    service = get_service(name=service_name)

    open_pull_requests = get_open_pull_requests(repository_name=repository_name)

    revision_tags = [traffic.tag for traffic in service.traffic if traffic.tag]

    tags_to_stop: list[str] = []
    for revision_tag in revision_tags:
        if revision_tag != "main" and revision_tag not in [
            f"pr-{pull_request.number}" for pull_request in open_pull_requests
        ]:
            tags_to_stop.append(revision_tag)

    for tag in tags_to_stop:
        for traffic in service.traffic:
            if traffic.tag and traffic.tag == tag:
                service.traffic.remove(traffic)

    update_service(service)


def get_service(name: str) -> Service:
    client = ServicesClient()
    return client.get_service(name=service_name)


def get_open_pull_requests(repository_name: str) -> list[PullRequest]:
    token = "SUPER_SECRET"
    auth = Auth.Token(token)
    g = Github(auth=auth)

    repo = g.get_repo(repository_name)

    pulls: list[PullRequest] = []
    for pull in repo.get_pulls(state="open"):
        pulls.append(pull)
    return pulls


def update_service(service: Service) -> None:
    client = ServicesClient()
    operation = client.update_service(service=service)

    operation.result()


if __name__ == "__main__":
    main()

Stack trace

Traceback (most recent call last):
  File "/Users/lennoxstevenson/Library/Caches/pypoetry/virtualenvs/reporting-aggregation-timeseries-service-vqnb887V-py3.11/lib/python3.11/site-packages/google/api_core/grpc_helpers.py", line 72, in error_remapped_callable
    return callable_(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/lennoxstevenson/Library/Caches/pypoetry/virtualenvs/reporting-aggregation-timeseries-service-vqnb887V-py3.11/lib/python3.11/site-packages/grpc/_channel.py", line 1161, in __call__
    return _end_unary_response_blocking(state, call, False, None)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/lennoxstevenson/Library/Caches/pypoetry/virtualenvs/reporting-aggregation-timeseries-service-vqnb887V-py3.11/lib/python3.11/site-packages/grpc/_channel.py", line 1004, in _end_unary_response_blocking
    raise _InactiveRpcError(state)  # pytype: disable=not-instantiable
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
grpc._channel._InactiveRpcError: <_InactiveRpcError of RPC that terminated with:
        status = StatusCode.ALREADY_EXISTS
        details = "Revision named 'reporting-aggregation-timeseries-service-00048-hot' with different configuration already exists."
        debug_error_string = "UNKNOWN:Error received from peer ipv4:142.250.80.74:443 {grpc_message:"Revision named \'reporting-aggregation-timeseries-service-00048-hot\' with different configuration already exists.", grpc_status:6, created_time:"2023-08-28T15:17:24.506657-04:00"}"
>

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/Users/lennoxstevenson/dev/reporting-aggregation-timeseries-service/scripts/stop-closed-preview-environments.py", line 58, in <module>
    main()
  File "/Users/lennoxstevenson/dev/reporting-aggregation-timeseries-service/scripts/stop-closed-preview-environments.py", line 29, in main
    update_service(service)
  File "/Users/lennoxstevenson/dev/reporting-aggregation-timeseries-service/scripts/stop-closed-preview-environments.py", line 52, in update_service
    operation = client.update_service(service=service)
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/lennoxstevenson/Library/Caches/pypoetry/virtualenvs/reporting-aggregation-timeseries-service-vqnb887V-py3.11/lib/python3.11/site-packages/google/cloud/run_v2/services/services/client.py", line 1079, in update_service
    response = rpc(
               ^^^^
  File "/Users/lennoxstevenson/Library/Caches/pypoetry/virtualenvs/reporting-aggregation-timeseries-service-vqnb887V-py3.11/lib/python3.11/site-packages/google/api_core/gapic_v1/method.py", line 113, in __call__
    return wrapped_func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/lennoxstevenson/Library/Caches/pypoetry/virtualenvs/reporting-aggregation-timeseries-service-vqnb887V-py3.11/lib/python3.11/site-packages/google/api_core/timeout.py", line 120, in func_with_timeout
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/Users/lennoxstevenson/Library/Caches/pypoetry/virtualenvs/reporting-aggregation-timeseries-service-vqnb887V-py3.11/lib/python3.11/site-packages/google/api_core/grpc_helpers.py", line 74, in error_remapped_callable
    raise exceptions.from_grpc_error(exc) from exc
google.api_core.exceptions.AlreadyExists: 409 Revision named 'reporting-aggregation-timeseries-service-00048-hot' with different configuration already exists.

steve148 avatar Aug 28 '23 19:08 steve148

@parthea you were mentioned by our rep from SADA as the person to mention. Are you the right person to ping on this issue or is there someone else?

steve148 avatar Sep 05 '23 17:09 steve148

Hi @steve148,

Please could you attempt to create the problem using gcloud to help rule out if this is a client library issue or API specific issue? https://cloud.google.com/sdk/gcloud/reference/run/services/update

If the issue also appears with gcloud then this is likely an API specific issue. We'll redirect you to the API team to file an issue in the public issue tracker: https://issuetracker.google.com/issues/new?component=608076&template=0

parthea avatar Sep 12 '23 15:09 parthea

Hey! I can confirm that the gcloud equivalent command worked for what I wanted to do. That would lead me to think it's either a) me using the python client wrong here or b) something is up with the client library.

gcloud run services update-traffic MY_SERVICE --platform=managed --region=MY_REGION --remove-tags="pr-1,pr-2"

steve148 avatar Sep 19 '23 14:09 steve148

I'm going to transfer this issue to google-cloud-python as we're planning to move the code for google-cloud-run there in the next 1-2 weeks

parthea avatar Oct 21 '23 20:10 parthea

We'll look into this, but have no updates yet.

vchudnov-g avatar Dec 08 '23 20:12 vchudnov-g