aws-serverless-nextcloud icon indicating copy to clipboard operation
aws-serverless-nextcloud copied to clipboard

[BUG] ELB throws 500 bad GW

Open emctl opened this issue 3 years ago • 6 comments

Tested this with defaults apart from image (I used 23.0.1) and while everything provisions correct, NC never passed the health check and is in a 500 loop. I unfortunately don't have time to investigate any further today but can follow up as needed over the weekend. I just wanted to flag this to see if anyone else is in the same boat?

I have not tried this with the default image from the template as I want to use the latest available

emctl avatar Feb 18 '22 08:02 emctl

It takes loads amount of time to spin up the container. This is an issue with the official docker image and how they initialize nextcloud. At the first run and on every upgrade they use rsync to copy over the application files to a persistent file storage (EFS in this case). Initially EFS has no IOPS provisioned and is therefore quite slow.

I will increase the Health check grace period on the ECS service configuration. Tested with 2500, that should be enough for now. See https://github.com/aws-samples/aws-serverless-nextcloud/blob/main/ecs-nextcloud.yml#L623

We are trying to get the nextcloud docker image in line with best practices, such as keeping it stateless. There is no reason to require a persistent volume for it. See https://github.com/nextcloud/docker/pull/1548

f7o avatar Feb 20 '22 22:02 f7o

I will further push for improvement of the official docker image or provide an alternative docker image which follows container best practices.

f7o avatar Feb 21 '22 07:02 f7o

@f7o - FWIW, I tried this again with multiple different NC images waiting 30+ mins between redeploys with the same problem. Happy to spin up again for the needs of debugging if needed. This was the exact same issue, the image was running, all metrics looked fine (including EFS).

emctl avatar Feb 24 '22 07:02 emctl

Can you share logs from ECS tasks? Especially the initial task running when EFS and RDS is deployed the first time?

Which AWS Region did you deploy into? I assume the cloudformation stack was created successfully, it is just the application never coming up, including health checks?

If, for what ever reason, the first container being started does not finish the initial installation routine and is being stopped and replaced with a new one, the whole installation is broken. One has to prune the EFS and remove all tables from the RDS database to get a clean installation.

f7o avatar Feb 24 '22 11:02 f7o

Can you share logs from ECS tasks? Especially the initial task running when EFS and RDS is deployed the first time?

Which AWS Region did you deploy into? I assume the cloudformation stack was created successfully, it is just the application never coming up, including health checks?

If, for what ever reason, the first container being started does not finish the initial installation routine and is being stopped and replaced with a new one, the whole installation is broken. One has to prune the EFS and remove all tables from the RDS database to get a clean installation. @f7o Sorry for the delay -

  • Default region from quickstart (i.e. eu-west-1)
  • Stack (and nested stacks) all provision successfully

Defaults apart from usernames/pw's for the db and nc itself.

looks like it doesnt like a non default db name - image

I deleted the stack and retried to default db names -

This seems to work so I geuss this is due to changing db names - should that be removed as an input?

emctl avatar Feb 25 '22 19:02 emctl

okay, seems like you are right and non-default db name does not quite work. i will investigate further

f7o avatar Mar 01 '22 10:03 f7o