libsql icon indicating copy to clipboard operation
libsql copied to clipboard

Bottomless replication requires synchronous call to S3

Open athoscouto opened this issue 1 year ago • 1 comments

The issue applies both to namespace loading and metastore loading during startup. This causes cold start to take over 10s in some cases since it requires multiple roundtrips to S3.

athoscouto avatar Apr 07 '24 13:04 athoscouto

Agreed, this is a major issue for us too at the moment. Our load tests suffer quite a bit because of this, as spawning multiple tenant DBs in a short span causes some of the db creation requests to wait for as long as 30s and this is the case even when a single db cold start time with bottomless is quite manageable for us at 2 secs since we have a local cluster minio where our bottomless backups go to. Currently our way around this is to disable sqld bottomless backups and asynchronously trigger a bottomless backup using cli after namespace creation, which is quite hacky and we would like to move away from it.

sveltespot avatar Apr 08 '24 04:04 sveltespot