add a static cache for nodes based on properties or other criteria

[jexp]

for frequenly used nodes and relationships, create an in-memory cache (index) that puts them into a (optionally sorted) map by property

• a named cache lives in a static field

• map property to node-id or node-proxy

• need to be multi-thread-save

• manage TTL/refresh (caffeine) ?

• build cache from cypher statement

• add to cache dynamically

• make sure the cache can be configured to be sorted by value, so that it allows range-queries for the property and also returns the nodes / relationship in sort-order

apoc.cache.add(key, node | relationship) apoc.cache.add.query("match (n) return n.foo as value, n as node")

apoc.cache.nodes.for(key,value) -> stream of nodes apoc.cache.rels.for(key, value) -> stream of relationships apoc.cache.nodes.between(key, from, to) -> return stream values in sort order

[crichey]

I'm not convinced that using memory to do this is the right thing. You are going to run into heap issues and threading challenges. I'm thinking of doing this by using a special node in the DB to cache node ids

[jexp]

Definitely not, caching nodes or id's is no problem. A node-proxy is just a thin wrapper around the long-id. so it's at most 16+8 bytes.

[crichey]

OK. We are going to fork the repo and work on this. Will send a pull request when initial work in place.

[jexp]

Also add a cache for certain node or relationship properties -> node/rel-id -> property value. Potentially as optimized column store where repeated values are only stored once or with fewer bits

As Cypher now uses caffeine, we should also use that, with TTL and dynamic cache rebuilds (if needed)

[crichey]

Can you expand on cyperh's use of caffeine? I must have missed that

Sent from my iPhone

On Jan 9, 2017, at 19:18, Michael Hunger [email protected] wrote:

Also add a cache for certain node or relationship properties -> node/rel-id -> property value. Potentially as optimized column store where repeated values are only stored once or with fewer bits

As Cypher now uses caffeine, we should also use that, with TTL and dynamic cache rebuilds (if needed)

— You are receiving this because you commented. Reply to this email directly, view it on GitHub, or mute the thread.

[jexp]

@crichey it uses it for its internal query -> query-plan cache. Just means we don't have to add an extra library dependency

[jexp]

@crichey any updates on your work, otherwise I'd love to look at this in the next weeks.

[crichey]

We are doing some internal testing today and will review tomorrow. I will provide update then.

From: Michael Hunger [mailto:[email protected]] Sent: Wednesday, January 18, 2017 11:44 AM To: neo4j-contrib/neo4j-apoc-procedures [email protected] Cc: Clark D. Richey, Jr. [email protected]; Mention [email protected] Subject: Re: [neo4j-contrib/neo4j-apoc-procedures] add a static cache for nodes based on properties or other criteria (#226)

@crichey https://github.com/crichey any updates on your work, otherwise I'd love to look at this in the next weeks.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/neo4j-contrib/neo4j-apoc-procedures/issues/226#issuecomment-273529507 , or mute the thread https://github.com/notifications/unsubscribe-auth/AANEYHaYxdPxUpje2K0rjd7ulE7s3GBmks5rTkFTgaJpZM4LHN5_ . https://github.com/notifications/beacon/AANEYEOb849TapoCW0K5zo3uwd4T7rtAks5rTkFTgaJpZM4LHN5_.gif

---

This email has been checked for viruses by Avast antivirus software. https://www.avast.com/antivirus

[albertodelazzari]

Is this still in progress or not? Otherwise we can work on it