clusterfuzz icon indicating copy to clipboard operation
clusterfuzz copied to clipboard

Cannot set up new ClusterFuzz instance in Europe

Open ngg opened this issue 1 year ago • 0 comments

I've tried to modified the regions to be in Europe by adding the location/zone parameters to create_config: python butler.py create_config --oauth-client-secrets-path=$CLIENT_SECRETS_PATH --firebase-api-key=$FIREBASE_API_KEY --project-id=$CLOUD_PROJECT_ID $CONFIG_DIR --gce-zone europe-west3-c --appengine-location europe-west3

However, this results in several resources created in the EU and several in the US, e.g. most buckets are created in the US multi-region.

It causes us different types of problems:

  1. I'd like to have everything in the EU for compliance reasons (preferably in a specifyable region like europe-west3).
  2. Some operations do not work when the resources are split between EU and US, e.g. the Backup cron job fails with the following error:
Traceback (most recent call last):
  File "/layers/google.python.runtime/python/lib/python3.7/threading.py", line 890, in _bootstrap
    self._bootstrap_inner()
  File "/layers/google.python.runtime/python/lib/python3.7/threading.py", line 926, in _bootstrap_inner
    self.run()
  File "/layers/google.python.runtime/python/lib/python3.7/threading.py", line 870, in run
    self._target(*self._args, **self._kwargs)
  File "/layers/google.python.runtime/python/lib/python3.7/concurrent/futures/thread.py", line 80, in _worker
    work_item.run()
  File "/layers/google.python.runtime/python/lib/python3.7/concurrent/futures/thread.py", line 57, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/layers/google.python.pip/pip/lib/python3.7/site-packages/gunicorn/workers/gthread.py", line 271, in handle
    keepalive = self.handle_request(req, conn)
  File "/layers/google.python.pip/pip/lib/python3.7/site-packages/gunicorn/workers/gthread.py", line 323, in handle_request
    respiter = self.wsgi(environ, resp.start_response)
  File "/layers/google.python.pip/pip/lib/python3.7/site-packages/flask/app.py", line 2464, in __call__
    return self.wsgi_app(environ, start_response)
  File "/srv/server.py", line 95, in middleware
    return wsgi_app(environ, start_response)
  File "/layers/google.python.pip/pip/lib/python3.7/site-packages/flask/app.py", line 2447, in wsgi_app
    response = self.full_dispatch_request()
  File "/layers/google.python.pip/pip/lib/python3.7/site-packages/flask/app.py", line 1950, in full_dispatch_request
    rv = self.dispatch_request()
  File "/layers/google.python.pip/pip/lib/python3.7/site-packages/flask/app.py", line 1936, in dispatch_request
    return self.view_functions[rule.endpoint](**req.view_args)
  File "/layers/google.python.pip/pip/lib/python3.7/site-packages/flask/views.py", line 89, in view
    return self.dispatch_request(*args, **kwargs)
  File "/srv/handlers/base_handler.py", line 277, in dispatch_request
    return super(Handler, self).dispatch_request(*args, **kwargs)
  File "/layers/google.python.pip/pip/lib/python3.7/site-packages/flask/views.py", line 163, in dispatch_request
    return meth(*args, **kwargs)
  File "/srv/libs/handler.py", line 99, in wrapper
    result = func(self)
  File "/srv/handlers/cron/backup.py", line 80, in get
    logs.log_error(message, error=str(e))
LogError: Datastore export failed.
Traceback (most recent call last):
  File "/srv/handlers/cron/backup.py", line 72, in get
    response = request.execute()
  File "/layers/google.python.pip/pip/lib/python3.7/site-packages/googleapiclient/_helpers.py", line 134, in positional_wrapper
    return wrapped(*args, **kwargs)
  File "/layers/google.python.pip/pip/lib/python3.7/site-packages/googleapiclient/http.py", line 907, in execute
    raise HttpError(resp, content, uri=self.uri)
googleapiclient.errors.HttpError: <HttpError 400 when requesting https://datastore.googleapis.com/v1/projects/*****:export?alt=json returned "Bucket backup.*****.appspot.com is in location US. This project can only operate on buckets spanning location eu or europe-west3.">

ngg avatar Jan 13 '23 07:01 ngg