ewallet icon indicating copy to clipboard operation
ewallet copied to clipboard

Add a caching mechanism for the most common DB calls

Open T-Dnzt opened this issue 5 years ago • 1 comments

Things like:

  • Building the account hierarchy
  • Get a token or a wallet
  • ...

T-Dnzt avatar Jul 24 '18 06:07 T-Dnzt

On a basic GCS n1-standard-2 instance with the eWallet and postgres residing on the same instance. Requests to /transaction.create start to fail when reaching above 2,500 transactions/minute.

All errors are this:

Erlang error: {:timeout, {:gen_server, :call, [LocalLedgerDB.Repo.Pool, {:checkout, #Reference<0.2304877775.2979004418.24482>, true}, 5000]}}

With stacktrace:

(stdlib) gen_server.erl:214: :gen_server.call/3
(poolboy) src/poolboy.erl:55: :poolboy.checkout/3
(db_connection) lib/db_connection/poolboy.ex:41: DBConnection.Poolboy.checkout/2
(db_connection) lib/db_connection.ex:928: DBConnection.checkout/2
(db_connection) lib/db_connection.ex:750: DBConnection.run/3
(db_connection) lib/db_connection.ex:644: DBConnection.execute/4
(ecto) lib/ecto/adapters/postgres/connection.ex:105: Ecto.Adapters.Postgres.Connection.execute/4
(ecto) lib/ecto/adapters/sql.ex:243: Ecto.Adapters.SQL.sql_call/6

With these as their top 5 queries (reported by AppSignal):

-- Mean: 542ms
-- Throughput: 20,116
-- Impact: 40.6%
SELECT t0."uuid", t0."id", t0."metadata", t0."encrypted_metadata", t0."inserted_at", t0."updated_at" 
FROM "token" AS t0 
WHERE (t0."id" = $1)
-- Mean: 457ms
-- Throughput: 20,022
-- Impact: 35.25%
SELECT w0."uuid", w0."address", w0."metadata", w0."encrypted_metadata", w0."inserted_at", w0."updated_at"
FROM "wallet" AS w0 
WHERE (w0."address" = $1)
-- Mean: 175ms
-- Throughput: 29,621
-- Impact: 20%
begin
-- Mean: 71ms
-- Throughput: 9,688
-- Impact: 2.63%
SELECT w0."uuid", w0."address", w0."metadata", w0."encrypted_metadata", w0."inserted_at", w0."updated_at"
FROM "wallet" AS w0
WHERE (w0."address" = ANY($1)) 
FOR UPDATE
-- Mean: 8ms
-- Throughput: 9,683
-- Impact: 0.29%
SELECT sum(e0."amount") 
FROM "entry" AS e0 
WHERE (((e0."wallet_address" = $1) AND (e0."type" = $2)) AND (e0."token_id" = $3))

The rest takes between 462 µs and 5 ms (on average) to respond.

unnawut avatar Jan 21 '19 10:01 unnawut