Zappa
Zappa copied to clipboard
zappa
Running zappa within codebuild: Name or service not known error
Context
I'm executing a zappa build from with aws codebuild. It runs fine when run locally.
Expected Behavior
It should run in codebuild just like it runs successfully locally
Actual Behavior
It gives error "Name or service not known" when run in codebuild.
Steps to Reproduce
My buildspec is as follows:
`version: 0.2
phases: install: runtime-versions: python: 3.8 commands: finally: - echo install step completed pre_build: commands: - python -m venv venv - . venv/bin/activate - pip install -r requirements.txt build: commands: - zappa update production finally: - echo build step completed post_build: commands: - echo starting post build finally: - echo post build completed`
AWS environment variables set in code build are: AWS_DEFAULT_REGION AWS_ACCESS_KEY_ID AWS_SECRET_ACCESS_KEY
The output response from codebuild execution is this:
` [Container] 2022/02/18 17:47:07 Waiting for agent ping [Container] 2022/02/18 17:47:10 Waiting for DOWNLOAD_SOURCE [Container] 2022/02/18 17:47:13 Phase is DOWNLOAD_SOURCE [Container] 2022/02/18 17:47:13 CODEBUILD_SRC_DIR=/codebuild/output/src150018930/src [Container] 2022/02/18 17:47:13 YAML location is /codebuild/readonly/buildspec.yml [Container] 2022/02/18 17:47:13 Processing environment variables [Container] 2022/02/18 17:47:13 Selecting 'python' runtime version '3.8' based on manual selections... [Container] 2022/02/18 17:47:13 Running command echo "Installing Python version 3.8 ..." Installing Python version 3.8 ...
[Container] 2022/02/18 17:47:13 Running command pyenv global $PYTHON_38_VERSION
[Container] 2022/02/18 17:47:15 Moving to directory /codebuild/output/src150018930/src [Container] 2022/02/18 17:47:16 Configuring ssm agent with target id: codebuild:7494fd75-98a0-457f-bb9e-783517fb0049 [Container] 2022/02/18 17:47:16 Successfully updated ssm agent configuration [Container] 2022/02/18 17:47:16 Registering with agent [Container] 2022/02/18 17:47:16 Phases found in YAML: 4 [Container] 2022/02/18 17:47:16 INSTALL: 1 commands [Container] 2022/02/18 17:47:16 PRE_BUILD: 3 commands [Container] 2022/02/18 17:47:16 BUILD: 1 commands [Container] 2022/02/18 17:47:16 POST_BUILD: 1 commands [Container] 2022/02/18 17:47:16 Phase complete: DOWNLOAD_SOURCE State: SUCCEEDED [Container] 2022/02/18 17:47:16 Phase context status code: Message: [Container] 2022/02/18 17:47:16 Entering phase INSTALL [Container] 2022/02/18 17:47:16 Running command
[Container] 2022/02/18 17:47:16 Running command echo install step completed install step completed
[Container] 2022/02/18 17:47:16 Phase complete: INSTALL State: SUCCEEDED [Container] 2022/02/18 17:47:16 Phase context status code: Message: [Container] 2022/02/18 17:47:16 Entering phase PRE_BUILD [Container] 2022/02/18 17:47:16 Running command python -m venv venv
[Container] 2022/02/18 17:47:18 Running command . venv/bin/activate
[Container] 2022/02/18 17:47:18 Running command pip install -r requirements.txt Collecting wheel Downloading wheel-0.37.1-py2.py3-none-any.whl (35 kB) Collecting asgiref==3.2.10 Downloading asgiref-3.2.10-py3-none-any.whl (19 kB) ....... ....... ....... Collecting click>=5.1 Downloading click-8.0.3-py3-none-any.whl (97 kB) Collecting placebo>=0.8.1 Downloading placebo-0.10.0.tar.gz (15 kB) Collecting pep517 Downloading pep517-0.12.0-py2.py3-none-any.whl (19 kB) Collecting tomli>=1.1.0; python_version >= "3.6" Downloading tomli-2.0.1-py3-none-any.whl (12 kB) Using legacy 'setup.py install' for Unipath, since package 'wheel' is not installed. Using legacy 'setup.py install' for future, since package 'wheel' is not installed. Using legacy 'setup.py install' for kappa, since package 'wheel' is not installed. Using legacy 'setup.py install' for wsgi-request-logger, since package 'wheel' is not installed. Using legacy 'setup.py install' for durationpy, since package 'wheel' is not installed. Using legacy 'setup.py install' for placebo, since package 'wheel' is not installed. Installing collected packages: wheel, asgiref, pycodestyle, toml, autopep8, pytz, sqlparse, Django, jsonfield, certifi, urllib3, idna, charset-normalizer, requests, stripe, dj-stripe, jmespath, six, python-dateutil, botocore, s3transfer, boto3, future, django-ses, Unipath, dj-database-url, python-decouple, gunicorn, whitenoise, djangorestframework, zipp, importlib-metadata, markdown, PyMySQL, django-filter, django-cors-headers, Werkzeug, click, PyYAML, cfn-flip, troposphere, hjson, text-unidecode, python-slugify, placebo, kappa, wsgi-request-logger, durationpy, tomli, pep517, pip-tools, argcomplete, tqdm, zappa Running setup.py install for future: started Running setup.py install for future: finished with status 'done' Running setup.py install for Unipath: started Running setup.py install for Unipath: finished with status 'done' Running setup.py install for placebo: started Running setup.py install for placebo: finished with status 'done' Running setup.py install for kappa: started Running setup.py install for kappa: finished with status 'done' Running setup.py install for wsgi-request-logger: started Running setup.py install for wsgi-request-logger: finished with status 'done' Running setup.py install for durationpy: started Running setup.py install for durationpy: finished with status 'done' ERROR: After October 2020 you may experience errors when installing or updating packages. This is because pip will change the way that it resolves dependency conflicts.
We recommend you use --use-feature=2020-resolver to test your packages with the new resolver before it becomes the default.
pip-tools 6.5.1 requires pip>=21.2, but you'll have pip 20.2.3 which is incompatible. Successfully installed Django-3.1.1 PyMySQL-1.0.2 PyYAML-6.0 Unipath-1.1 Werkzeug-2.0.3 argcomplete-2.0.0 asgiref-3.2.10 autopep8-1.6.0 boto3-1.21.2 botocore-1.24.2 certifi-2021.10.8 cfn-flip-1.3.0 charset-normalizer-2.0.12 click-8.0.3 dj-database-url-0.5.0 dj-stripe-2.5.1 django-cors-headers-3.11.0 django-filter-21.1 django-ses-1.0.3 djangorestframework-3.13.1 durationpy-0.5 future-0.18.2 gunicorn-20.1.0 hjson-3.0.2 idna-3.3 importlib-metadata-4.11.1 jmespath-0.10.0 jsonfield-3.1.0 kappa-0.6.0 markdown-3.3.6 pep517-0.12.0 pip-tools-6.5.1 placebo-0.10.0 pycodestyle-2.8.0 python-dateutil-2.8.2 python-decouple-3.6 python-slugify-6.0.1 pytz-2021.3 requests-2.27.1 s3transfer-0.5.1 six-1.16.0 sqlparse-0.4.2 stripe-2.66.0 text-unidecode-1.3 toml-0.10.2 tomli-2.0.1 tqdm-4.62.3 troposphere-3.2.2 urllib3-1.26.8 wheel-0.37.1 whitenoise-6.0.0 wsgi-request-logger-0.4.6 zappa-0.54.1 zipp-3.7.0 WARNING: You are using pip version 20.2.3; however, version 22.0.3 is available. You should consider upgrading via the '/codebuild/output/src150018930/src/venv/bin/python -m pip install --upgrade pip' command.
[Container] 2022/02/18 17:47:32 Phase complete: PRE_BUILD State: SUCCEEDED [Container] 2022/02/18 17:47:32 Phase context status code: Message: [Container] 2022/02/18 17:47:32 Entering phase BUILD [Container] 2022/02/18 17:47:32 Running command zappa update production (pip 20.2.3 (/codebuild/output/src150018930/src/venv/lib/python3.8/site-packages), Requirement.parse('pip>=21.2'), {'pip-tools'}) Calling update for stage production.. /codebuild/output/src150018930/src/venv/lib/python3.8/site-packages/setuptools/distutils_patch.py:25: UserWarning: Distutils was imported before Setuptools. This usage is discouraged and may exhibit undesirable behaviors or errors. Please use Setuptools' objects directly or at least import Setuptools first. warnings.warn( Warning! AWS Lambda may not be available in this AWS Region! Warning! AWS API Gateway may not be available in this AWS Region! Downloading and installing dependencies.. Packaging project as zip.
Warning: Application zip package is likely to be too large for AWS Lambda. Try setting "slim_handler" to true in your Zappa settings file.
Traceback (most recent call last): File "/codebuild/output/src150018930/src/venv/lib/python3.8/site-packages/urllib3/connection.py", line 174, in _new_conn conn = connection.create_connection( File "/codebuild/output/src150018930/src/venv/lib/python3.8/site-packages/urllib3/util/connection.py", line 72, in create_connection for res in socket.getaddrinfo(host, port, family, socket.SOCK_STREAM): File "/root/.pyenv/versions/3.8.7/lib/python3.8/socket.py", line 918, in getaddrinfo for res in _socket.getaddrinfo(host, port, family, type, proto, flags): socket.gaierror: [Errno -2] Name or service not known
During handling of the above exception, another exception occurred:
Traceback (most recent call last): File "/codebuild/output/src150018930/src/venv/lib/python3.8/site-packages/botocore/httpsession.py", line 403, in send urllib_response = conn.urlopen( File "/codebuild/output/src150018930/src/venv/lib/python3.8/site-packages/urllib3/connectionpool.py", line 785, in urlopen retries = retries.increment( File "/codebuild/output/src150018930/src/venv/lib/python3.8/site-packages/urllib3/util/retry.py", line 525, in increment raise six.reraise(type(error), error, _stacktrace) File "/codebuild/output/src150018930/src/venv/lib/python3.8/site-packages/urllib3/packages/six.py", line 770, in reraise raise value File "/codebuild/output/src150018930/src/venv/lib/python3.8/site-packages/urllib3/connectionpool.py", line 703, in urlopen httplib_response = self._make_request( File "/codebuild/output/src150018930/src/venv/lib/python3.8/site-packages/urllib3/connectionpool.py", line 386, in _make_request self._validate_conn(conn) File "/codebuild/output/src150018930/src/venv/lib/python3.8/site-packages/urllib3/connectionpool.py", line 1040, in _validate_conn conn.connect() File "/codebuild/output/src150018930/src/venv/lib/python3.8/site-packages/urllib3/connection.py", line 358, in connect conn = self._new_conn() File "/codebuild/output/src150018930/src/venv/lib/python3.8/site-packages/urllib3/connection.py", line 186, in _new_conn raise NewConnectionError( urllib3.exceptions.NewConnectionError: <botocore.awsrequest.AWSHTTPSConnection object at 0x7f03fbc44d30>: Failed to establish a new connection: [Errno -2] Name or service not known
During handling of the above exception, another exception occurred:
Traceback (most recent call last): File "/codebuild/output/src150018930/src/venv/lib/python3.8/site-packages/zappa/cli.py", line 3415, in handle sys.exit(cli.handle()) File "/codebuild/output/src150018930/src/venv/lib/python3.8/site-packages/zappa/cli.py", line 588, in handle self.dispatch_command(self.command, stage) File "/codebuild/output/src150018930/src/venv/lib/python3.8/site-packages/zappa/cli.py", line 641, in dispatch_command self.update( File "/codebuild/output/src150018930/src/venv/lib/python3.8/site-packages/zappa/cli.py", line 1093, in update success = self.zappa.upload_to_s3( File "/codebuild/output/src150018930/src/venv/lib/python3.8/site-packages/zappa/core.py", line 1051, in upload_to_s3 self.s3_client.head_bucket(Bucket=bucket_name) File "/codebuild/output/src150018930/src/venv/lib/python3.8/site-packages/botocore/client.py", line 391, in _api_call return self._make_api_call(operation_name, kwargs) File "/codebuild/output/src150018930/src/venv/lib/python3.8/site-packages/botocore/client.py", line 705, in _make_api_call http, parsed_response = self._make_request( File "/codebuild/output/src150018930/src/venv/lib/python3.8/site-packages/botocore/client.py", line 725, in _make_request return self._endpoint.make_request(operation_model, request_dict) File "/codebuild/output/src150018930/src/venv/lib/python3.8/site-packages/botocore/endpoint.py", line 106, in make_request return self._send_request(request_dict, operation_model) File "/codebuild/output/src150018930/src/venv/lib/python3.8/site-packages/botocore/endpoint.py", line 182, in _send_request while self._needs_retry(attempts, operation_model, request_dict, File "/codebuild/output/src150018930/src/venv/lib/python3.8/site-packages/botocore/endpoint.py", line 301, in _needs_retry responses = self._event_emitter.emit( File "/codebuild/output/src150018930/src/venv/lib/python3.8/site-packages/botocore/hooks.py", line 357, in emit return self._emitter.emit(aliased_event_name, **kwargs) File "/codebuild/output/src150018930/src/venv/lib/python3.8/site-packages/botocore/hooks.py", line 228, in emit return self._emit(event_name, kwargs) File "/codebuild/output/src150018930/src/venv/lib/python3.8/site-packages/botocore/hooks.py", line 211, in _emit response = handler(**kwargs) File "/codebuild/output/src150018930/src/venv/lib/python3.8/site-packages/botocore/retryhandler.py", line 192, in call if self._checker(**checker_kwargs): File "/codebuild/output/src150018930/src/venv/lib/python3.8/site-packages/botocore/retryhandler.py", line 265, in call should_retry = self._should_retry(attempt_number, response, File "/codebuild/output/src150018930/src/venv/lib/python3.8/site-packages/botocore/retryhandler.py", line 292, in _should_retry return self._checker(attempt_number, response, caught_exception) File "/codebuild/output/src150018930/src/venv/lib/python3.8/site-packages/botocore/retryhandler.py", line 331, in call checker_response = checker(attempt_number, response, File "/codebuild/output/src150018930/src/venv/lib/python3.8/site-packages/botocore/retryhandler.py", line 231, in call return self._check_caught_exception( File "/codebuild/output/src150018930/src/venv/lib/python3.8/site-packages/botocore/retryhandler.py", line 374, in _check_caught_exception raise caught_exception File "/codebuild/output/src150018930/src/venv/lib/python3.8/site-packages/botocore/endpoint.py", line 248, in _do_get_response http_response = self._send(request) File "/codebuild/output/src150018930/src/venv/lib/python3.8/site-packages/botocore/endpoint.py", line 317, in _send return self.http_session.send(request) File "/codebuild/output/src150018930/src/venv/lib/python3.8/site-packages/botocore/httpsession.py", line 432, in send raise EndpointConnectionError(endpoint_url=request.url, error=e) botocore.exceptions.EndpointConnectionError: Could not connect to the endpoint URL: "https://zappa-xxx.s3.use-east-2.amazonaws.com/"
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/codebuild/output/src150018930/src/venv/bin/zappa", line 8, in
[Container] 2022/02/18 17:48:07 Command did not exit successfully zappa update production exit status 1 [Container] 2022/02/18 17:48:07 Running command echo build step completed build step completed
[Container] 2022/02/18 17:48:07 Phase complete: BUILD State: FAILED [Container] 2022/02/18 17:48:07 Phase context status code: COMMAND_EXECUTION_ERROR Message: Error while executing command: zappa update production. Reason: exit status 1 [Container] 2022/02/18 17:48:07 Entering phase POST_BUILD [Container] 2022/02/18 17:48:07 Running command echo starting post build starting post build
[Container] 2022/02/18 17:48:07 Running command echo post build completed post build completed
[Container] 2022/02/18 17:48:07 Phase complete: POST_BUILD State: SUCCEEDED [Container] 2022/02/18 17:48:07 Phase context status code: Message: `
Your Environment
- Zappa version used: 0.54.1
- Operating System and Python version: AWS Ubuntu with python 3.8
- The output of
pip freeze:
argcomplete==1.12.3 asgiref==3.2.10 autopep8==1.6.0 boto3==1.19.2 botocore==1.22.2 certifi==2021.10.8 cfn-flip==1.3.0 charset-normalizer==2.0.7 click==8.0.3 colorama==0.4.4 dj-database-url==0.5.0 dj-stripe==2.5.1 Django==3.1.1 django-cors-headers==3.4.0 django-filter==2.2.0 django-ses==1.0.3 djangorestframework==3.11.2 durationpy==0.5 future==0.18.2 gunicorn==20.1.0 hjson==3.0.2 idna==3.3 jmespath==0.10.0 jsonfield==3.1.0 kappa==0.6.0 Markdown==3.3.4 pep517==0.12.0 pip-tools==6.4.0 placebo==0.10.0 pycodestyle==2.8.0 PyMySQL==1.0.2 python-dateutil==2.8.2 python-decouple==3.5 python-slugify==5.0.2 pytz==2021.3 PyYAML==6.0 requests==2.26.0 s3transfer==0.5.0 six==1.16.0 sqlparse==0.4.2 stripe==2.61.0 text-unidecode==1.3 toml==0.10.2 tomli==1.2.2 tqdm==4.62.3 troposphere==3.1.1 Unipath==1.1 urllib3==1.26.7 Werkzeug==2.0.2 whitenoise==5.3.0 wsgi-request-logger==0.4.6 zappa==0.54.1
- Your
zappa_settings.json:
{ "production": { "aws_region": "us-east-2", "django_settings": "core.settings", "domain": "api.xxx.io", "profile_name": "default", "project_name": "xxx-ios-api", "runtime": "python3.9", "s3_bucket": "zappa-xxx", "aws_environment_variables" : { "DB_USER":"admin", "DB_PASSWORD":"xxx", "DB_HOST":"xxx-app.xxx.us-east-2.rds.amazonaws.com", "DB_PORT":"3306", "STAGE":"Production", "DEPLOYMENT_BUCKET":"xxx-django-deployment" } } }
Your error appears to be:
botocore.exceptions.EndpointConnectionError: Could not connect to the endpoint URL: "https://zappa-xxx.s3.use-east-2.amazonaws.com/"
Which is likely due to a permission error in your configuration of codebuild. make sure it has permissions to the bucket.
I started getting this error as well out of the blue, checked my config file inside .aws and region setting seems to be ok, as well as credentials, been stuck on this error for a while now.
Getting a connection error "out of the blue" is strange, and suggests that your configuration changed some where.
I'm closing this due to stale activity. For connection issues, you need to debug your configuration and assure that your functions can reach the desired service.