wpt.fyi icon indicating copy to clipboard operation
wpt.fyi copied to clipboard

Resource issues occur sporadically for staging redeployments

Open DanielRyanSmith opened this issue 2 years ago • 1 comments

When deploying a staging environment through the check suite, errors like this can occur.

ERROR: (gcloud.app.deploy) Error Response: [9] An internal error occurred while processing task /app-engine-flex/flex_await_healthy/flex_await_healthy>2022-07-20T20:57:43.045Z20061.fm.0: 
No sufficient free disk space left for your App Engine Flexible application.
Please increase your VM instances disk size in the resource settings in the
app.yaml file for your deployment and retry.

These errors appeared commonly when running redeployments with new commits on the scoring change PR.

@jcscottiii investigated the error on this PR and after deleting the staging app engine version entirely and rerunning the deployment from scratch, this issue disappeared. It seems that this resource issue can occur when a new instance is created in an existing version of the application. Some more investigation is needed to diagnose the source of this problem.

DanielRyanSmith avatar Jul 20 '22 22:07 DanielRyanSmith

Same as https://github.com/web-platform-tests/wpt.fyi/issues/2867. I have a previous attempt to reduce processor staging deployment size - https://github.com/web-platform-tests/wpt.fyi/pull/2869. App engine only has resources for 3 processor staging deployment at the moment.

KyleJu avatar Aug 08 '22 02:08 KyleJu