Memory/resource leak in postgresql process (probably by prepared statements)

[enomado]

Hi guys. I'm doing some performance-driven development and just and notice that my computer go to swap. All memory was consumed by postgresql processes.

It was just simple api called by wrk tool selecting table with something like

store := md.NewCategoryStore(db)
q := md.NewCategoryQuery()
cats, err := store.FindAll(q)

db.Exec("ROLLBACK;") doesnt help db.Begin() ... tx.Rollback help but i didnt understand how.

I'm used to feel that nothing can leak in postgresql session and started thinking what it could be. Than i've noticed power using of prepared statements and gotcha, db.Exec("DEALLOCATE ALL") prevent leaking.

[nadiamoe]

Hello,

I've been investigating a bit about this, and in order to check if kallax is responsible for this, I would need you to check if the leak is produced with squirrel's statement cacher disabled.

You can do it like this:

store := md.NewCategoryStore(db)
store = store.DisableCacher() // !! DisableCacher returns a store with the cacher disabled
q := md.NewCategoryQuery()
cats, err := store.FindAll(q)

[enomado]

Oh I see. https://github.com/src-d/go-kallax/commit/75a61190be346416056d88c21e932eea39f62ae7 My version suddenly don't have this method.

I can't re-generate models for the new version for now. From the beginning it was done by ~/go/bin/kallax but now it prints: could not import ./models (can't find import: "models")

I lose my passion about go to investigate more time to figure out how to fix it. Probably best thing I can do is to share my code? https://gist.github.com/enomado/eeb101b7c04f2af822386be7803ff2e2 It was originally a playground of few orms... It was nothing more than simple query flood without a commit. Its a rare case anyway.

[denchenko]

If you are doing store := md.NewCategoryStore(db) on each api call, then it is expected behaviour, because squirrel stmt cacher does not find any prepared statements in local cache and executes PREPARE ... on database over and over. Thus, all these prepared statements live entire lifetime of db connection.