spotlight-docker
spotlight-docker copied to clipboard
dbpedia-spotlight
RAM requirements?
Running:
docker run -i -p 2222:80 dbpedia/spotlight-english spotlight.sh
Resulted in the process being killed, could this be due to running out of memory? I can see in spotlight.sh you require 20GB for the java heapsize so could this cause the below behaviour? What is the minimum RAM requirement to run dbpedia spotlight from docker?
Logs:
$ docker run -i -p 2222:80 dbpedia/spotlight-english spotlight.sh
Unable to find image 'dbpedia/spotlight-english:latest' locally
latest: Pulling from dbpedia/spotlight-english
88286f41530e: Pull complete
720349d0916a: Pull complete
9431a0557160: Pull complete
89c0205cd042: Pull complete
8be3620205bc: Pull complete
f24c3713766d: Pull complete
Digest: sha256:ca9577202bf7d490bde06288339df98ffcabb12da7c3b4841ac3385924d1c67d
Status: Downloaded newer image for dbpedia/spotlight-english:latest
[main] INFO org.dbpedia.spotlight.db.memory.MemoryStore$ - Loading MemoryQuantizedCountStore...
[main] INFO org.dbpedia.spotlight.db.memory.MemoryStore$ - Done (43 ms)
[main] INFO org.dbpedia.spotlight.db.memory.MemoryStore$ - Loading MemoryTokenTypeStore...
[main] INFO org.dbpedia.spotlight.db.memory.MemoryTokenTypeStore - Creating reverse-lookup for Tokens.
[main] INFO org.dbpedia.spotlight.db.memory.MemoryStore$ - Done (1300 ms)
[main] INFO org.dbpedia.spotlight.db.memory.MemoryStore$ - Loading MemorySurfaceFormStore...
[main] INFO org.dbpedia.spotlight.db.memory.MemorySurfaceFormStore - Summing total SF counts.
[main] INFO org.dbpedia.spotlight.db.memory.MemorySurfaceFormStore - Creating reverse-lookup for surface forms, adding normalized surface forms.
[main] INFO org.dbpedia.spotlight.db.memory.MemoryStore$ - Done (23969 ms)
[main] INFO org.dbpedia.spotlight.db.memory.MemoryStore$ - Loading MemoryResourceStore...
[main] INFO org.dbpedia.spotlight.db.memory.MemoryResourceStore - Creating reverse-lookup for DBpedia resources.
[main] INFO org.dbpedia.spotlight.db.memory.MemoryResourceStore - Counting total support...
[main] INFO org.dbpedia.spotlight.db.memory.MemoryResourceStore - Done.
[main] INFO org.dbpedia.spotlight.db.memory.MemoryStore$ - Done (42725 ms)
[main] INFO org.dbpedia.spotlight.db.memory.MemoryStore$ - Loading MemoryCandidateMapStore...
Killed
Hi @swartchris8 ,
Your question reminds me that we must make a benchmark test to adjust our memory parameters at Docker.
Anyway... It seems that you do not have enough RAM to run it. You must have at least 8GB RAM free to run English Spotlight. If you have it, please tell us what version Docker version are you using and your S.O.
All the best,
Hey @sandroacoelho , Thanks for your quick reply I am on a Mac, docker version 17.09.0-ce-mac35 (19611) and I only have about 5GB of RAM after on an 8 GB RAM machine with docker running. Cheers,
Hi @sandroacoelho
I am having a similar issue on a Win 10 machine using docker version 17.09.0-ce-win33 (13620) - see logs below. Note: I am running this on 32GB of RAM.
Thanks!
logs: docker run -i -p 2222:80 dbpedia/spotlight-english:latest spotlight.sh [main] INFO org.dbpedia.spotlight.db.memory.MemoryStore$ - Loading MemoryQuantizedCountStore... [main] INFO org.dbpedia.spotlight.db.memory.MemoryStore$ - Done (43 ms) [main] INFO org.dbpedia.spotlight.db.memory.MemoryStore$ - Loading MemoryTokenTypeStore... [main] INFO org.dbpedia.spotlight.db.memory.MemoryTokenTypeStore - Creating reverse-lookup for Tokens. [main] INFO org.dbpedia.spotlight.db.memory.MemoryStore$ - Done (1342 ms) [main] INFO org.dbpedia.spotlight.db.memory.MemoryStore$ - Loading MemorySurfaceFormStore... [main] INFO org.dbpedia.spotlight.db.memory.MemorySurfaceFormStore - Summing total SF counts. [main] INFO org.dbpedia.spotlight.db.memory.MemorySurfaceFormStore - Creating reverse-lookup for surface forms, adding normalized surface forms. [main] INFO org.dbpedia.spotlight.db.memory.MemoryStore$ - Done (23352 ms) [main] INFO org.dbpedia.spotlight.db.memory.MemoryStore$ - Loading MemoryResourceStore... [main] INFO org.dbpedia.spotlight.db.memory.MemoryResourceStore - Creating reverse-lookup for DBpedia resources. [main] INFO org.dbpedia.spotlight.db.memory.MemoryResourceStore - Counting total support... [main] INFO org.dbpedia.spotlight.db.memory.MemoryResourceStore - Done. [main] INFO org.dbpedia.spotlight.db.memory.MemoryStore$ - Done (29628 ms) [main] INFO org.dbpedia.spotlight.db.memory.MemoryStore$ - Loading MemoryCandidateMapStore... Killed
Hi @owensacco,
Under linux we know that it works... I will test it under Windows :(
:/home/spotlight# docker run -i -p 2222:80 dbpedia/spotlight-english spotlight.sh
Unable to find image 'dbpedia/spotlight-english:latest' locally
latest: Pulling from dbpedia/spotlight-english
88286f41530e: Already exists
720349d0916a: Already exists
9431a0557160: Already exists
4d91280b983d: Pull complete
2852514d6a0b: Pull complete
51a808189d57: Pull complete
Digest: sha256:0798c91bafbd5d8e5d5487481a1b322a2173c613c112d34bc103e672ad478cd1
Status: Downloaded newer image for dbpedia/spotlight-english:latest
[main] INFO org.dbpedia.spotlight.db.memory.MemoryStore$ - Loading MemoryQuantizedCountStore...
[main] INFO org.dbpedia.spotlight.db.memory.MemoryStore$ - Done (33 ms)
[main] INFO org.dbpedia.spotlight.db.memory.MemoryStore$ - Loading MemoryTokenTypeStore...
[main] INFO org.dbpedia.spotlight.db.memory.MemoryTokenTypeStore - Creating reverse-lookup for Tokens.
[main] INFO org.dbpedia.spotlight.db.memory.MemoryStore$ - Done (428 ms)
[main] INFO org.dbpedia.spotlight.db.memory.MemoryStore$ - Loading MemorySurfaceFormStore...
[main] INFO org.dbpedia.spotlight.db.memory.MemorySurfaceFormStore - Summing total SF counts.
[main] INFO org.dbpedia.spotlight.db.memory.MemorySurfaceFormStore - Creating reverse-lookup for surface forms, adding normalized surface forms.
[main] INFO org.dbpedia.spotlight.db.memory.MemoryStore$ - Done (11007 ms)
[main] INFO org.dbpedia.spotlight.db.memory.MemoryStore$ - Loading MemoryResourceStore...
[main] INFO org.dbpedia.spotlight.db.memory.MemoryResourceStore - Creating reverse-lookup for DBpedia resources.
[main] INFO org.dbpedia.spotlight.db.memory.MemoryResourceStore - Counting total support...
[main] INFO org.dbpedia.spotlight.db.memory.MemoryResourceStore - Done.
[main] INFO org.dbpedia.spotlight.db.memory.MemoryStore$ - Done (2735 ms)
[main] INFO org.dbpedia.spotlight.db.memory.MemoryStore$ - Loading MemoryCandidateMapStore...
[main] INFO org.dbpedia.spotlight.db.memory.MemoryStore$ - Done (1430 ms)
[main] INFO org.dbpedia.spotlight.db.memory.MemoryStore$ - Loading MemoryContextStore...
[main] INFO org.dbpedia.spotlight.db.memory.MemoryStore$ - Done (28681 ms)
[main] INFO org.dbpedia.spotlight.db.memory.MemoryStore$ - Loading FSADictionary...
[main] INFO org.dbpedia.spotlight.db.memory.MemoryStore$ - Done (373 ms)
[main] INFO org.dbpedia.spotlight.web.rest.Server - Initiated 1 disambiguators.
[main] INFO org.dbpedia.spotlight.web.rest.Server - Initiated 2 spotters.
Oct 27, 2017 12:34:21 AM com.sun.grizzly.Controller logVersion
INFO: GRIZZLY0001: Starting Grizzly Framework 1.9.48 - 10/27/17 12:34 AM
Server started in / listening on http://0.0.0.0:80/rest
[main] INFO org.dbpedia.spotlight.web.rest.Server - Core threads: 5
[main] INFO org.dbpedia.spotlight.web.rest.Server - Max threads: 5
[main] INFO org.dbpedia.spotlight.web.rest.Server - Max POST size: 2097152
Oct 27, 2017 12:34:21 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.dbpedia.spotlight.web.rest.resources
Oct 27, 2017 12:34:25 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.dbpedia.spotlight.web.rest.resources.Feedback
class org.dbpedia.spotlight.web.rest.resources.Candidates
class org.dbpedia.spotlight.web.rest.resources.Annotate
class org.dbpedia.spotlight.web.rest.resources.Disambiguate
class org.dbpedia.spotlight.web.rest.resources.Spot
Oct 27, 2017 12:34:25 AM com.sun.jersey.api.core.ScanningResourceConfig init
INFO: No provider classes found.
Oct 27, 2017 12:34:25 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM
FYI: In my case, I increased the Java heap size (Xms/Xmx) in my docker container, as described in the Wiki->Troubleshooting (https://github.com/dbpedia-spotlight/dbpedia-spotlight/wiki/Troubleshooting), and it works now.
docker run -i -e JAVA_OPTS='-Xmx16g' -p 2222:80 dbpedia/spotlight-english spotlight.sh
