Training on large corpora is extremely slow, anyway to parallelize the pattern detector?

[jhashemi]

[nabilblk]

+1 , the Training is extremely slow

[kbastani]

Can you provide memory configurations? Please copy and paste your properties from neo4j.properties in the neo4j /conf directory.

Recommended memory settings are below:

neostore.nodestore.db.mapped_memory=512M neostore.relationshipstore.db.mapped_memory=2048M neostore.propertystore.db.mapped_memory=1024M neostore.propertystore.db.strings.mapped_memory=500M neostore.propertystore.db.arrays.mapped_memory=500M

This configuration assumes you have at least 8GB of available system memory.

[jhashemi]

Definitely helped training, but now classification takes upwards of 3+minutes per entity. This is using a HA cluster

[kbastani]

Glad to hear it helped training. I'm going to need more information about your dataset in order to get you fixed up. You can reach me on Skype at kenny.bastani or e-mail [email protected].

[letronje]

Using the recommended memory settings above certain improves the training speed(most requests are sub-second). Classification requests take anywhere between 15 to 30 seconds. Any way to speed them up ? Also, if multiple classify requests are sent in parallel, it throws a 500.