arcadedb icon indicating copy to clipboard operation
arcadedb copied to clipboard

Is it possible to run a whole database in-memory?

Open shasaur opened this issue 1 year ago • 3 comments

I'd very much love to be able to run a very low latency ArcadeDB instance. In my application, I do a lot of I/O a second, and the optimisation for cypher queries in general is not great.

For a big use case, my DB right now can be about:

  • 150MB
  • 100,360 nodes
  • 124,959 relationships

Even for this, queries can take a really long time of up to 5 seconds. And that's after severe query structure optimisation.

shasaur avatar Jan 09 '24 16:01 shasaur

You can play with the settings to keep the whole database in RAM. These are the main settings:

  • arcadedb.maxPageRAM (Maximum amount of pages (in MB) to keep in RAM). The default is 4GB so it should be enough for your use case
  • arcadedb.initialPageCacheSize (Initial number of entries for page cache). Default is 65,535. Since each page by default is 65K, it's 4GB of RAM. Again, should be good for your use case

So I think it's not in the settings but in the way you use ArcadeDB that can make a lot of difference. First 2 questions:

  • are you using ArcadeDB embedded or as a server?
  • if as a server, are you using HTTP driver or Postgres/anything else?

lvca avatar Jan 09 '24 16:01 lvca

It's as a server. My application is a mix of golang and C#, and embedded is only for Java right?

I'm using the Postgres plug-in to enable the graph / gremlin / cypher stuff.

shasaur avatar Jan 09 '24 17:01 shasaur

In this case >90% of the time is spent in the communication between client and server and that can be fixed only with using ArcadeDB as embedded. Unfortunately, there is no Go and C# binding on top of the JVM (yet).

But 5 seconds is way too much for a query, especially for a database this size. Could you post your query here?

lvca avatar Jan 14 '24 23:01 lvca

Closed this issue because of no response from the user for a while. In case this issue is still valid, please comment on this issue.

lvca avatar Mar 12 '24 20:03 lvca