Document that high db pool size requires increased shared memory for postgres
Setting a high db pool size causes various sql query errors, most notably logins failing entirely. To fix it, shared memory for postgres needs to be increased with the following line in docker-compose.yml. This should be documented, but not sure where it fits.
postgres:
image: postgres:15-alpine
shm_size: 1g
...
Original Lemmy issue
Logins on lemmy.ml are consistently failing with known working user/password. Server logs show the following error:
lemmy_1 | 2023-07-26T12:52:38.922771Z WARN lemmy_server::root_span_builder: incorrect_login: could not resize shared memory segment "/PostgreSQL.1513506408" to 220224 bytes: No space left on device
lemmy_1 | 0: lemmy_api::local_user::login::perform
lemmy_1 | with self=Login { username_or_email: Sensitive, password: Sensitive, totp_2fa_token: None }
lemmy_1 | at crates/api/src/local_user/login.rs:16
lemmy_1 | 1: lemmy_server::root_span_builder::HTTP request
lemmy_1 | with http.method=POST http.scheme="http" http.host=lemmy.ml http.target=/api/v3/user/login otel.kind="server" request_id=1853cb5f-c9c7-4794-bf02-4bbef2ed5c41
lemmy_1 | at src/root_span_builder.rs:16
lemmy_1 | LemmyError { message: Some("incorrect_login"), inner: could not resize shared memory segment "/PostgreSQL.1513506408" to 220224 bytes: No space left on device, context: SpanTrace [{ target: "lemmy_api::local_user::login", name: "perform", fields: "self=Login { username_or_email: Sensitive, password: Sensitive, totp_2fa_token: None }", file: "crates/api/src/local_user/login.rs", line: 16 }, { target: "lemmy_server::root_span_builder", name: "HTTP request", fields: "http.method=POST http.scheme=\"http\" http.host=lemmy.ml http.target=/api/v3/user/login otel.kind=\"server\" request_id=1853cb5f-c9c7-4794-bf02-4bbef2ed5c41", file: "src/root_span_builder.rs", line: 16 }] }
A lemmy restart fixes the problem temporarily. There is plenty of free disk space on the server.
The problem might be related to the fact that I set the db pool size to 95 because of LemmyNet/lemmy#3661. I lowered it to the previous value of 32 now, lets see if the problem happens again.
Version: 0.18.2
could not resize shared memory segment "/PostgreSQL.1513506408" is a problem with shared memory (/dev/shm) not disk space. Is postgres constrained in the amount of shared memory it can use, or is there only a small amount on the server?
df -h shows this (same if executed inside the postgres docker container):
tmpfs 16G 0 16G 0% /dev/shm
Also it looks like lowering db pool size really did fix the issue.
It's a bit old but have a look at the docker settings: https://stackoverflow.com/questions/56751565/pq-could-not-resize-shared-memory-segment-no-space-left-on-device
Thanks, that works! I added the following to docker-compose.yml as suggested in the stackoverflow link.
postgres:
image: postgres:15-alpine
shm_size: 1g
...
Gonna move this to the docs repo.