arcadeanalytics icon indicating copy to clipboard operation
arcadeanalytics copied to clipboard

Published 20 hours ago • verified publisher icon ArcadeData

Can't connect to Gremlin Neptune DB

Open ericmillsio opened this issue 6 years ago • 43 comments

Hey! Awesome that you brought this open source. I'm just getting started, docker installation was a breeze.

I'm trying to connect my AWS Neptune DB as a datasource. I'm running a local docker instance while VPN'd in from my computer, which connects me to my AWS VPC. I shouldn't need any database/user/password/ssh or anything. Right?

Type: Gremlin Gremlin Implementation: Neptune Server: my-db.xxxxxxxxxxx.us-east-1.neptune.amazonaws.com Port: 8182

I found this article you guys have, but many variations of this did not work.

I've been using graphexp and connecting works fine for that.

Any suggestions? Thanks for the help! Super excited to start using this.

ericmillsio avatar Feb 18 '19 23:02 ericmillsio

hi @crearc , you should be able to "ping" your neptune clusterfrom the command line while in VPN:

ping my-db.xxxxxxxxxxx.us-east-1.neptune.amazonaws.com

if the ping command doesn't work, try with the cluster IP:

ping <neptune IP>

if the IP works, configure the datasource with the ip instead of the address.

BTW, you can watch the logs of the container and report the error message :

docker logs -f   docker_arcadeanalytics-app_1

robfrank avatar Feb 19 '19 07:02 robfrank

As far as the logs go, I'm getting:

c.a.p.gremlin.GremlinDataProvider        : testing connection to :: 'my-db.xxxxxxxxxxxx.us-east-1.neptune.amazonaws.com'
o.z.p.spring.web.advice.AdviceTrait      : Internal Server Error
java.lang.RuntimeException: java.lang.RuntimeException: java.lang.RuntimeException: java.util.concurrent.TimeoutException: Timed out while waiting for an available host - check the client configuration and connectivity to the server if this message persists
at com.arcadeanalytics.provider.gremlin.GremlinDataProvider.testConnection(GremlinDataProvider.kt:298)
at com.arcadeanalytics.web.rest.DataSourceResource.testDataSource(DataSourceResource.java:123)

ping is not effective for Neptune instances. By default AWS does not allow ping access. I've attempted to add a 'Custom ICMP Rule' to allow for ping which usually works for EC2 instances, but Neptune totally ignores this rule and it does not apply (similar to other database services on AWS).

For RDS, but same rules apply:

"RDS Instances are not configured to accept and respond to an ICMP packet for pings. The only way you can establish connectivity to your RDS instance is through a standard SQL client application." This means, that adding ICMP rule into particular RDS security group, doesn't make your RDS instance reachable over ICMP.

However, nmap works fine (cluster IP works too): nmap -Pn -p 8182 my-db.xxxxxxxxxxx.us-east-1.neptune.amazonaws.com.

Starting Nmap 7.70 ( https://nmap.org ) at 2019-02-19 10:56 CST
Nmap scan report for my-db.xxxxxxxxxxx.us-east-1.neptune.amazonaws.com (XXX.XX.XX.XX)
Host is up (0.041s latency).
rDNS record for XXX.XX.XX.XX: ip-XXX-XX-XX-XX.ec2.internal

PORT     STATE SERVICE
8182/tcp open  vmware-fdm

Nmap done: 1 IP address (1 host up) scanned in 0.15 seconds

Note the -PN is required because I was getting Note: Host seems down. If it is really up, but blocking our ping probes, try -Pn.

Curling the status also works fine (cluster IP works too):

curl -G http://my-db.xxxxxxxxxxx.us-east-1.neptune.amazonaws.com:8182/status
{"status":"healthy"}

It's most definitely available. As I said I've been able to connect graphexp fine and was able to interact with the DB.

ericmillsio avatar Feb 19 '19 17:02 ericmillsio

Full log output:

arcadeanalytics-app_1            | 2019-02-19 16:26:17.683  INFO 1 --- [ XNIO-2 task-30] c.a.p.gremlin.GremlinDataProvider        : testing connection to :: 'my-db.xxxxxxxxxx.us-east-1.neptune.amazonaws.com'
arcadeanalytics-app_1            | 2019-02-19 16:26:18.146 ERROR 1 --- [ XNIO-2 task-30] o.z.p.spring.web.advice.AdviceTrait      : Internal Server Error
arcadeanalytics-app_1            |
arcadeanalytics-app_1            | java.lang.RuntimeException: java.lang.RuntimeException: java.lang.RuntimeException: java.util.concurrent.TimeoutException: Timed out while waiting for an available host - check the client configuration and connectivity to the server if this message persists
arcadeanalytics-app_1            | 	at com.arcadeanalytics.provider.gremlin.GremlinDataProvider.testConnection(GremlinDataProvider.kt:298)
arcadeanalytics-app_1            | 	at com.arcadeanalytics.web.rest.DataSourceResource.testDataSource(DataSourceResource.java:123)
arcadeanalytics-app_1            | 	at com.arcadeanalytics.web.rest.DataSourceResource$$FastClassBySpringCGLIB$$89a9850e.invoke(<generated>)
arcadeanalytics-app_1            | 	at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204)
arcadeanalytics-app_1            | 	at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:738)
arcadeanalytics-app_1            | 	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157)
arcadeanalytics-app_1            | 	at com.ryantenney.metrics.spring.TimedMethodInterceptor.invoke(TimedMethodInterceptor.java:48)
arcadeanalytics-app_1            | 	at com.ryantenney.metrics.spring.TimedMethodInterceptor.invoke(TimedMethodInterceptor.java:34)
arcadeanalytics-app_1            | 	at com.ryantenney.metrics.spring.AbstractMetricMethodInterceptor.invoke(AbstractMetricMethodInterceptor.java:59)
arcadeanalytics-app_1            | 	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
arcadeanalytics-app_1            | 	at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:673)
arcadeanalytics-app_1            | 	at com.arcadeanalytics.web.rest.DataSourceResource$$EnhancerBySpringCGLIB$$6affad1b.testDataSource(<generated>)
arcadeanalytics-app_1            | 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
arcadeanalytics-app_1            | 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
arcadeanalytics-app_1            | 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
arcadeanalytics-app_1            | 	at java.lang.reflect.Method.invoke(Method.java:498)
arcadeanalytics-app_1            | 	at org.springframework.web.method.support.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:205)
arcadeanalytics-app_1            | 	at org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:133)
arcadeanalytics-app_1            | 	at org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:97)
arcadeanalytics-app_1            | 	at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandlerMethod(RequestMappingHandlerAdapter.java:827)
arcadeanalytics-app_1            | 	at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:738)
arcadeanalytics-app_1            | 	at org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:85)
arcadeanalytics-app_1            | 	at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:967)
arcadeanalytics-app_1            | 	at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:901)
arcadeanalytics-app_1            | 	at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:970)
arcadeanalytics-app_1            | 	at org.springframework.web.servlet.FrameworkServlet.doPost(FrameworkServlet.java:872)
arcadeanalytics-app_1            | 	at javax.servlet.http.HttpServlet.service(HttpServlet.java:707)
arcadeanalytics-app_1            | 	at org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:846)
arcadeanalytics-app_1            | 	at javax.servlet.http.HttpServlet.service(HttpServlet.java:790)
arcadeanalytics-app_1            | 	at io.undertow.servlet.handlers.ServletHandler.handleRequest(ServletHandler.java:74)
arcadeanalytics-app_1            | 	at io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:129)
arcadeanalytics-app_1            | 	at com.codahale.metrics.servlet.AbstractInstrumentedFilter.doFilter(AbstractInstrumentedFilter.java:111)
arcadeanalytics-app_1            | 	at io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:61)
arcadeanalytics-app_1            | 	at io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:131)
arcadeanalytics-app_1            | 	at org.springframework.boot.web.filter.ApplicationContextHeaderFilter.doFilterInternal(ApplicationContextHeaderFilter.java:55)
arcadeanalytics-app_1            | 	at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
arcadeanalytics-app_1            | 	at io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:61)
arcadeanalytics-app_1            | 	at io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:131)
arcadeanalytics-app_1            | 	at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:101)
arcadeanalytics-app_1            | 	at io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:61)
arcadeanalytics-app_1            | 	at io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:131)
arcadeanalytics-app_1            | 	at org.springframework.boot.actuate.trace.WebRequestTraceFilter.doFilterInternal(WebRequestTraceFilter.java:111)
arcadeanalytics-app_1            | 	at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
arcadeanalytics-app_1            | 	at io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:61)
arcadeanalytics-app_1            | 	at io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:131)
arcadeanalytics-app_1            | 	at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:317)
arcadeanalytics-app_1            | 	at org.springframework.security.web.access.intercept.FilterSecurityInterceptor.invoke(FilterSecurityInterceptor.java:127)
arcadeanalytics-app_1            | 	at org.springframework.security.web.access.intercept.FilterSecurityInterceptor.doFilter(FilterSecurityInterceptor.java:91)
arcadeanalytics-app_1            | 	at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
arcadeanalytics-app_1            | 	at org.springframework.security.web.access.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:114)
arcadeanalytics-app_1            | 	at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
arcadeanalytics-app_1            | 	at org.springframework.security.web.session.SessionManagementFilter.doFilter(SessionManagementFilter.java:137)
arcadeanalytics-app_1            | 	at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
arcadeanalytics-app_1            | 	at org.springframework.security.web.authentication.AnonymousAuthenticationFilter.doFilter(AnonymousAuthenticationFilter.java:111)
arcadeanalytics-app_1            | 	at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
arcadeanalytics-app_1            | 	at org.springframework.security.web.servletapi.SecurityContextHolderAwareRequestFilter.doFilter(SecurityContextHolderAwareRequestFilter.java:170)
arcadeanalytics-app_1            | 	at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
arcadeanalytics-app_1            | 	at org.springframework.security.web.savedrequest.RequestCacheAwareFilter.doFilter(RequestCacheAwareFilter.java:63)
arcadeanalytics-app_1            | 	at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
arcadeanalytics-app_1            | 	at com.arcadeanalytics.security.jwt.JWTFilter.doFilter(JWTFilter.java:36)
arcadeanalytics-app_1            | 	at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
arcadeanalytics-app_1            | 	at org.springframework.web.filter.CorsFilter.doFilterInternal(CorsFilter.java:96)
arcadeanalytics-app_1            | 	at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
arcadeanalytics-app_1            | 	at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
arcadeanalytics-app_1            | 	at org.springframework.security.web.authentication.logout.LogoutFilter.doFilter(LogoutFilter.java:116)
arcadeanalytics-app_1            | 	at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
arcadeanalytics-app_1            | 	at org.springframework.security.web.header.HeaderWriterFilter.doFilterInternal(HeaderWriterFilter.java:66)
arcadeanalytics-app_1            | 	at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
arcadeanalytics-app_1            | 	at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
arcadeanalytics-app_1            | 	at org.springframework.security.web.context.SecurityContextPersistenceFilter.doFilter(SecurityContextPersistenceFilter.java:105)
arcadeanalytics-app_1            | 	at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
arcadeanalytics-app_1            | 	at org.springframework.security.web.context.request.async.WebAsyncManagerIntegrationFilter.doFilterInternal(WebAsyncManagerIntegrationFilter.java:56)
arcadeanalytics-app_1            | 	at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
arcadeanalytics-app_1            | 	at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
arcadeanalytics-app_1            | 	at org.springframework.security.web.FilterChainProxy.doFilterInternal(FilterChainProxy.java:214)
arcadeanalytics-app_1            | 	at org.springframework.security.web.FilterChainProxy.doFilter(FilterChainProxy.java:177)
arcadeanalytics-app_1            | 	at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:347)
arcadeanalytics-app_1            | 	at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:263)
arcadeanalytics-app_1            | 	at io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:61)
arcadeanalytics-app_1            | 	at io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:131)
arcadeanalytics-app_1            | 	at org.springframework.web.filter.RequestContextFilter.doFilterInternal(RequestContextFilter.java:99)
arcadeanalytics-app_1            | 	at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
arcadeanalytics-app_1            | 	at io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:61)
arcadeanalytics-app_1            | 	at io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:131)
arcadeanalytics-app_1            | 	at org.springframework.web.filter.HttpPutFormContentFilter.doFilterInternal(HttpPutFormContentFilter.java:109)
arcadeanalytics-app_1            | 	at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
arcadeanalytics-app_1            | 	at io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:61)
arcadeanalytics-app_1            | 	at io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:131)
arcadeanalytics-app_1            | 	at org.springframework.web.filter.HiddenHttpMethodFilter.doFilterInternal(HiddenHttpMethodFilter.java:81)
arcadeanalytics-app_1            | 	at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
arcadeanalytics-app_1            | 	at io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:61)
arcadeanalytics-app_1            | 	at io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:131)
arcadeanalytics-app_1            | 	at org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:197)
arcadeanalytics-app_1            | 	at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
arcadeanalytics-app_1            | 	at io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:61)
arcadeanalytics-app_1            | 	at io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:131)
arcadeanalytics-app_1            | 	at io.undertow.servlet.handlers.FilterHandler.handleRequest(FilterHandler.java:84)
arcadeanalytics-app_1            | 	at io.undertow.servlet.handlers.security.ServletSecurityRoleHandler.handleRequest(ServletSecurityRoleHandler.java:62)
arcadeanalytics-app_1            | 	at io.undertow.servlet.handlers.ServletChain$1.handleRequest(ServletChain.java:64)
arcadeanalytics-app_1            | 	at io.undertow.servlet.handlers.ServletDispatchingHandler.handleRequest(ServletDispatchingHandler.java:36)
arcadeanalytics-app_1            | 	at io.undertow.servlet.handlers.security.SSLInformationAssociationHandler.handleRequest(SSLInformationAssociationHandler.java:132)
arcadeanalytics-app_1            | 	at io.undertow.servlet.handlers.security.ServletAuthenticationCallHandler.handleRequest(ServletAuthenticationCallHandler.java:57)
arcadeanalytics-app_1            | 	at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)
arcadeanalytics-app_1            | 	at io.undertow.security.handlers.AbstractConfidentialityHandler.handleRequest(AbstractConfidentialityHandler.java:46)
arcadeanalytics-app_1            | 	at io.undertow.servlet.handlers.security.ServletConfidentialityConstraintHandler.handleRequest(ServletConfidentialityConstraintHandler.java:64)
arcadeanalytics-app_1            | 	at io.undertow.security.handlers.AuthenticationMechanismsHandler.handleRequest(AuthenticationMechanismsHandler.java:60)
arcadeanalytics-app_1            | 	at io.undertow.servlet.handlers.security.CachedAuthenticatedSessionHandler.handleRequest(CachedAuthenticatedSessionHandler.java:77)
arcadeanalytics-app_1            | 	at io.undertow.security.handlers.AbstractSecurityContextAssociationHandler.handleRequest(AbstractSecurityContextAssociationHandler.java:43)
arcadeanalytics-app_1            | 	at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)
arcadeanalytics-app_1            | 	at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)
arcadeanalytics-app_1            | 	at io.undertow.servlet.handlers.ServletInitialHandler.handleFirstRequest(ServletInitialHandler.java:292)
arcadeanalytics-app_1            | 	at io.undertow.servlet.handlers.ServletInitialHandler.access$100(ServletInitialHandler.java:81)
arcadeanalytics-app_1            | 	at io.undertow.servlet.handlers.ServletInitialHandler$2.call(ServletInitialHandler.java:138)
arcadeanalytics-app_1            | 	at io.undertow.servlet.handlers.ServletInitialHandler$2.call(ServletInitialHandler.java:135)
arcadeanalytics-app_1            | 	at io.undertow.servlet.core.ServletRequestContextThreadSetupAction$1.call(ServletRequestContextThreadSetupAction.java:48)
arcadeanalytics-app_1            | 	at io.undertow.servlet.core.ContextClassLoaderSetupAction$1.call(ContextClassLoaderSetupAction.java:43)
arcadeanalytics-app_1            | 	at io.undertow.servlet.handlers.ServletInitialHandler.dispatchRequest(ServletInitialHandler.java:272)
arcadeanalytics-app_1            | 	at io.undertow.servlet.handlers.ServletInitialHandler.access$000(ServletInitialHandler.java:81)
arcadeanalytics-app_1            | 	at io.undertow.servlet.handlers.ServletInitialHandler$1.handleRequest(ServletInitialHandler.java:104)
arcadeanalytics-app_1            | 	at io.undertow.server.Connectors.executeRootHandler(Connectors.java:336)
arcadeanalytics-app_1            | 	at io.undertow.server.HttpServerExchange$1.run(HttpServerExchange.java:830)
arcadeanalytics-app_1            | 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
arcadeanalytics-app_1            | 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
arcadeanalytics-app_1            | 	at java.lang.Thread.run(Thread.java:748)
arcadeanalytics-app_1            | Caused by: java.lang.RuntimeException: java.lang.RuntimeException: java.util.concurrent.TimeoutException: Timed out while waiting for an available host - check the client configuration and connectivity to the server if this message persists
arcadeanalytics-app_1            | 	at org.apache.tinkerpop.gremlin.driver.Client.submit(Client.java:225)
arcadeanalytics-app_1            | 	at org.apache.tinkerpop.gremlin.driver.Client.submit(Client.java:209)
arcadeanalytics-app_1            | 	at com.arcadeanalytics.provider.gremlin.GremlinDataProvider.testConnection(GremlinDataProvider.kt:288)
arcadeanalytics-app_1            | 	... 123 common frames omitted
arcadeanalytics-app_1            | Caused by: java.lang.RuntimeException: java.util.concurrent.TimeoutException: Timed out while waiting for an available host - check the client configuration and connectivity to the server if this message persists
arcadeanalytics-app_1            | 	at org.apache.tinkerpop.gremlin.driver.Client.submitAsync(Client.java:321)
arcadeanalytics-app_1            | 	at org.apache.tinkerpop.gremlin.driver.Client.submitAsync(Client.java:253)
arcadeanalytics-app_1            | 	at org.apache.tinkerpop.gremlin.driver.Client.submit(Client.java:223)
arcadeanalytics-app_1            | 	... 125 common frames omitted
arcadeanalytics-app_1            | Caused by: java.util.concurrent.TimeoutException: Timed out while waiting for an available host - check the client configuration and connectivity to the server if this message persists
arcadeanalytics-app_1            | 	at org.apache.tinkerpop.gremlin.driver.Client$ClusteredClient.chooseConnection(Client.java:510)
arcadeanalytics-app_1            | 	at org.apache.tinkerpop.gremlin.driver.Client.submitAsync(Client.java:316)
arcadeanalytics-app_1            | 	... 127 common frames omitted
arcadeanalytics-app_1            |

ericmillsio avatar Feb 19 '19 17:02 ericmillsio

We tested only using an EC2 in the same VPC instance as SSH gateway: https://docs.aws.amazon.com/neptune/latest/userguide/security-vpc.html

Which kind of configuration do you provide to graphexp ? I mean, accesKey and so on?

robfrank avatar Feb 19 '19 18:02 robfrank

Ahh okay. I'm running locally while connected to my VPN (that gives access to the same VPC where Neptune lives). I just figured out that nmap -Pn -p 8182 my-db.xxxxxxxxxxx.us-east-1.neptune.amazonaws.com does not work while docker exec -it docker_arcadeanalytics-app_1 sh into the container. It works fine on my Mac host machine. I'm going to dig further into this, might just be a docker config <=> VPN issue.

Below is the answer to your questions anyways:

I don't need to provide any authentication at the moment with graphexp. As long as I'm within the VPC I can access it without any access keys. You'll notice the requests in graphexp are pretty simple. I did not enable 'IAM DB authentication' yet although I may in the future.

The only thing used when running the gremlin console is the Amazon Root CA Certificate. (See Root CA Certificates) But I don't think this is used with graphexp

ericmillsio avatar Feb 19 '19 20:02 ericmillsio

Oh yes!!! Arcade is running locally but inside a container in another subnet!

I will be back with a solution tomorrow.

robfrank avatar Feb 19 '19 21:02 robfrank

More playing around, let me know if this should be a different issue:

So I decided to role this into an actually EC2 instance because I was going to need to eventually anyway. Was able to get the server up and running, and it seems being exec sh'd into the container that Neptune is reachable now. I filled out the info, but got a new error. Seems like 'gateway' and 'sshUser' are coming in as 'null' instead of actual null which is causing an error to be thrown. Note that I left 'Use SSH Tunnel' unchecked.

DataSourceUtil.java:26

Optional.ofNullable(ds.getSshUser()).orElse("")

Seems like it's just the transfer from frontend ends up quoting null inputs in the API. I feel like it might work on this cloud instance if I get passed this...

Docker container log:

2019-02-20 17:11:17.761  INFO 1 --- [  XNIO-2 task-2] c.a.web.rest.DataSourceResource          : REST request to test DataSource : DataSourceDTO{id=null, name='Graph', description='null', type='GREMLIN_NEPTUNE', indexing='null', server='my-db.xxxxxxxx.neptune.amazonaws.com', port=8182, database='na', username='', password='', remote='false', gateway='null', sshPort=null, sshUser='null'}
2019-02-20 17:11:17.762  INFO 1 --- [  XNIO-2 task-2] c.a.web.rest.DataSourceResource          : DataSource : DataSource{id=null, name='Identity Graph', description='null', type='GREMLIN_NEPTUNE', indexing='null', server='my-db.xxxxxxxx.neptune.amazonaws.com', port=8182, database='na', username='', password='', remote='false', gateway='null', sshPort=null, sshUser='null'}
2019-02-20 17:11:17.764 ERROR 1 --- [  XNIO-2 task-2] o.z.p.spring.web.advice.AdviceTrait      : Internal Server Error
2019-02-20T16:57:46.176551482Z        java.lang.IllegalArgumentException: Parameter specified as non-null is null: method com.arcadeanalytics.provider.DataSourceInfo.\u003cinit\u003e, parameter description
2019-02-20T16:57:46.176555732Z     at com.arcadeanalytics.provider.DataSourceInfo.\u003cinit\u003e(DataSourceInfo.kt)
2019-02-20T16:57:46.176559675Z     at com.arcadeanalytics.service.util.DataSourceUtil.toDataSourceInfo(DataSourceUtil.java:26)
2019-02-20T16:57:46.176563594Z     at com.arcadeanalytics.web.rest.DataSourceResource.testDataSource(DataSourceResource.java:120)
2019-02-20T16:57:46.176567429Z     at com.arcadeanalytics.web.rest.DataSourceResource$$FastClassBySpringCGLIB$$89a9850e.invoke(\u003cgenerated\u003e)
2019-02-20T16:57:46.176571408Z     at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204)
2019-02-20T16:57:46.176575124Z     at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:738)
2019-02-20T16:57:46.176578969Z     at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157)
2019-02-20T16:57:46.176582757Z     at com.ryantenney.metrics.spring.TimedMethodInterceptor.invoke(TimedMethodInterceptor.java:48)
2019-02-20T16:57:46.176586639Z     at com.ryantenney.metrics.spring.TimedMethodInterceptor.invoke(TimedMethodInterceptor.java:34)
2019-02-20T16:57:46.176590369Z     at com.ryantenney.metrics.spring.AbstractMetricMethodInterceptor.invoke(AbstractMetricMethodInterceptor.java:59)
2019-02-20T16:57:46.176639709Z     at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
2019-02-20T16:57:46.176645974Z     at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:673)
2019-02-20T16:57:46.176650058Z     at com.arcadeanalytics.web.rest.DataSourceResource$$EnhancerBySpringCGLIB$$7adc8e07.testDataSource(\u003cgenerated\u003e)
2019-02-20T16:57:46.176654134Z     at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
2019-02-20T16:57:46.176657972Z     at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
2019-02-20T16:57:46.176661803Z     at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
2019-02-20T16:57:46.176665628Z     at java.lang.reflect.Method.invoke(Method.java:498)
2019-02-20T16:57:46.176670223Z     at org.springframework.web.method.support.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:205)
2019-02-20T16:57:46.176680918Z     at org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:133)
2019-02-20T16:57:46.176685148Z     at org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:97)
2019-02-20T16:57:46.176689031Z     at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandlerMethod(RequestMappingHandlerAdapter.java:827)
2019-02-20T16:57:46.176692957Z     at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:738)
2019-02-20T16:57:46.176696867Z     at org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:85)
2019-02-20T16:57:46.176700727Z     at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:967)
2019-02-20T16:57:46.176704572Z     at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:901)
2019-02-20T16:57:46.176708366Z     at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:970)
2019-02-20T16:57:46.176712084Z     at org.springframework.web.servlet.FrameworkServlet.doPost(FrameworkServlet.java:872)
2019-02-20T16:57:46.176715794Z     at javax.servlet.http.HttpServlet.service(HttpServlet.java:707)
2019-02-20T16:57:46.17671959Z      at org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:846)
2019-02-20T16:57:46.176723396Z     at javax.servlet.http.HttpServlet.service(HttpServlet.java:790)
2019-02-20T16:57:46.176727094Z     at io.undertow.servlet.handlers.ServletHandler.handleRequest(ServletHandler.java:74)
2019-02-20T16:57:46.176730868Z     at io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:129)
2019-02-20T16:57:46.176734623Z     at com.codahale.metrics.servlet.AbstractInstrumentedFilter.doFilter(AbstractInstrumentedFilter.java:111)
2019-02-20T16:57:46.176738351Z     at io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:61)
2019-02-20T16:57:46.176742043Z     at io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:131)
2019-02-20T16:57:46.176745783Z     at org.springframework.boot.web.filter.ApplicationContextHeaderFilter.doFilterInternal(ApplicationContextHeaderFilter.java:55)
2019-02-20T16:57:46.176749553Z     at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
2019-02-20T16:57:46.180774911Z     at io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:61)
2019-02-20T16:57:46.180805507Z     at io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:131)
2019-02-20T16:57:46.180811481Z     at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:101)
2019-02-20T16:57:46.180815529Z     at io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:61)
2019-02-20T16:57:46.180819301Z     at io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:131)
2019-02-20T16:57:46.180831363Z     at org.springframework.boot.actuate.trace.WebRequestTraceFilter.doFilterInternal(WebRequestTraceFilter.java:111)
2019-02-20T16:57:46.180835499Z     at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
2019-02-20T16:57:46.180839346Z     at io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:61)
2019-02-20T16:57:46.18084316Z      at io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:131)
2019-02-20T16:57:46.18084713Z      at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:317)
2019-02-20T16:57:46.180850924Z     at org.springframework.security.web.access.intercept.FilterSecurityInterceptor.invoke(FilterSecurityInterceptor.java:127)
2019-02-20T16:57:46.180854828Z     at org.springframework.security.web.access.intercept.FilterSecurityInterceptor.doFilter(FilterSecurityInterceptor.java:91)
2019-02-20T16:57:46.180858642Z     at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
2019-02-20T16:57:46.180862411Z     at org.springframework.security.web.access.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:114)
2019-02-20T16:57:46.180866219Z     at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
2019-02-20T16:57:46.180870104Z     at org.springframework.security.web.session.SessionManagementFilter.doFilter(SessionManagementFilter.java:137)
2019-02-20T16:57:46.180873876Z     at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
2019-02-20T16:57:46.180877806Z     at org.springframework.security.web.authentication.AnonymousAuthenticationFilter.doFilter(AnonymousAuthenticationFilter.java:111)
2019-02-20T16:57:46.18088163Z      at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
2019-02-20T16:57:46.180885453Z     at org.springframework.security.web.servletapi.SecurityContextHolderAwareRequestFilter.doFilter(SecurityContextHolderAwareRequestFilter.java:170)
2019-02-20T16:57:46.180890449Z     at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
2019-02-20T16:57:46.180894443Z     at org.springframework.security.web.savedrequest.RequestCacheAwareFilter.doFilter(RequestCacheAwareFilter.java:63)
2019-02-20T16:57:46.180898243Z     at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
2019-02-20T16:57:46.180902002Z     at com.arcadeanalytics.security.jwt.JWTFilter.doFilter(JWTFilter.java:36)
2019-02-20T16:57:46.180905703Z     at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
2019-02-20T16:57:46.180909573Z     at org.springframework.web.filter.CorsFilter.doFilterInternal(CorsFilter.java:96)

ericmillsio avatar Feb 20 '19 17:02 ericmillsio

Having similar problems. Here's my config: arcade1 And here are several errors I get when trying to write gremlin queries and using the full-text-search. arcade2 Also the "test connection" button is greyed out. So far I just started the source using docker-compose -f src/main/docker/app.yml up

Auricopter avatar Feb 21 '19 11:02 Auricopter

try this: setup "na" (or any other not null value) as value for user, password and database. Let me know.

We are working to improve this and among over to improve our docs

robfrank avatar Feb 21 '19 13:02 robfrank

try this: setup "na" (or any other not null value) as value for user, password and database. Let me know.

We are working to improve this and among over to improve our docs

Now I can hit "Test Connection" but the connection is still dead. 2019-02-21 16_32_24- This config worked 2 months ago, when I used the Arcade Playground on your server as a SaaS.

I suspect the problem is, that somehow the docker container isn't allowed to communicate through the VPC of the EC2/ECS Cluster.

I just remembered that you need to place Arcade's public key into the authorized_keys folder of the EC2

Auricopter avatar Feb 21 '19 15:02 Auricopter

Yes, now it works! You need to add the rsa-ssh public key from Arcade Analytics into your authorized_keys file in your EC2 instance. E.g. connect to your EC2 instance via WinSCP. Navigate to home/ec2-user/.ssh (hidden folder). Open authorized_keys. Add the public key.

2019-02-21 17_07_56-arcade

arcade

Auricopter avatar Feb 21 '19 16:02 Auricopter

Where do you get the public key? This page's download link is broken for me: https://arcadeanalytics.com/usermanual/#how-to-integrate-with-your-database

ericmillsio avatar Feb 21 '19 17:02 ericmillsio

Another day and deeper I go. I don't think I need the public key or ssh for my situation. I've gotten a success on testing the connection (I was mistakenly testing with the neptune instance instead of cluster url)! When I try and index the Neptune DB I get this error. It seems like it is seeing my dataset now for sure because Account and Domain are labels in my DB. (This case is within the context of my EC2 instance.)

Getting closer though!

2019-02-21 18:18:57.009  INFO 1 --- [tics-Executor-2] c.a.p.gremlin.GremlinGraphProvider       : start indexing of data-source 1351 - total nodes:: 9346 
2019-02-21 18:18:59.368  INFO 1 --- [tics-Executor-2] c.a.index.ElasticBulkGraphIndexer        : sending bulk to index 1351 of size 85 -  total indexed:: 85 
2019-02-21 18:18:59.442  INFO 1 --- [tics-Executor-2] c.a.index.ElasticBulkGraphIndexer        : flushing index 1351 - total docs indexed::  85
2019-02-21 18:18:59.443  INFO 1 --- [tics-Executor-2] c.a.index.ElasticBulkGraphIndexer        : indexed by class:: {Account=50, Domain=35}
error while indexing datasource DataSource{id=1351, name='Graph', description='graph', type='GREMLIN_NEPTUNE', indexing='INDEXING', server='my-db.xxxxxxxxxxx.us-east-1.neptune.amazonaws.com', port=8182, database='na', username='na', password='na', remote='false', gateway='', sshPort=null, sshUser=''}
java.lang.IllegalStateException: Multiple properties exist for the provided key, use Vertex.properties(aliases)
        at org.apache.tinkerpop.gremlin.structure.Vertex$Exceptions.multiplePropertiesExistForProvidedKey(Vertex.java:179)
        at org.apache.tinkerpop.gremlin.structure.util.detached.DetachedVertex.property(DetachedVertex.java:109)
        at org.apache.tinkerpop.gremlin.structure.util.detached.DetachedVertex.property(DetachedVertex.java:50)
        at org.apache.tinkerpop.gremlin.structure.Element.value(Element.java:94)
        at com.arcadeanalytics.provider.gremlin.GremlinGraphProvider.lambda$provideNodes$0(GremlinGraphProvider.java:93)
        at java.lang.Iterable.forEach(Iterable.java:75)
        at java.util.Collections$UnmodifiableCollection.forEach(Collections.java:1080)
        at com.arcadeanalytics.provider.gremlin.GremlinGraphProvider.provideNodes(GremlinGraphProvider.java:93)
        at com.arcadeanalytics.provider.gremlin.GremlinGraphProvider.provideTo(GremlinGraphProvider.java:62)
        at com.arcadeanalytics.service.ElasticGraphIndexerService.index(ElasticGraphIndexerService.java:118)
        at com.arcadeanalytics.service.ElasticGraphIndexerService$$FastClassBySpringCGLIB$$7483a18d.invoke(<generated>)
        at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204)
        at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:738)
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157)
        at org.springframework.aop.interceptor.AsyncExecutionInterceptor$1.call(AsyncExecutionInterceptor.java:115)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at io.github.jhipster.async.ExceptionHandlingAsyncTaskExecutor.lambda$createWrappedRunnable$1(ExceptionHandlingAsyncTaskExecutor.java:68)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)

ericmillsio avatar Feb 21 '19 18:02 ericmillsio

Thank you very much for all your feedbacks.

As you can image, after opening up the project we received a lot of interest and we are trying to manage all.

BTW, we are working on a new release that will enable to provide your own keys to the application for ssh connection.

About the stack-trace, I investigated and discovered that we don't handle multivalued properties. We are on it as well.

The plans are to release a new version with these fixes and other on Monday.

robfrank avatar Feb 21 '19 22:02 robfrank

Awesome! There's no rush. I totally understand how a new public release like this can go. Just trying to provide as much detail as I can to be helpful for you and future users.

I totally understand, don't stress! Take your time and let me know how I can help. :)

ericmillsio avatar Feb 21 '19 22:02 ericmillsio

hi, the new images, even if not yet tagged with a stable version, are available.

They use the new version of our connectors: https://github.com/ArcadeAnalytics/arcade-connectors/releases/tag/v1.0.2

This version fixes the problem with multivalued properties. Let me know if it works

robfrank avatar Mar 05 '19 18:03 robfrank

On startup now of the app, I'm hitting this error where it can't find /content/media. Made a folder at that location, but didn't help.

2019-03-05 23:15:32.479  INFO 1 --- [           main] com.arcadeanalytics.util.MediaLoader     : start loading images from:: /content/media/ 
2019-03-05 23:15:32.482  WARN 1 --- [           main] ationConfigEmbeddedWebApplicationContext : Exception encountered during context initialization - cancelling refresh attempt: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'mediaLoader' defined in class path resource [com/arcadeanalytics/util/MediaLoaderConfiguration.class]: Bean instantiation via factory method failed; nested exception is org.springframework.beans.BeanInstantiationException: Failed to instantiate [com.arcadeanalytics.util.MediaLoader]: Factory method 'mediaLoader' threw exception; nested exception is java.io.FileNotFoundException: ServletContext resource [/content/media/] cannot be resolved to URL because it does not exist
2019-03-05 23:15:32.615 ERROR 1 --- [           main] o.s.boot.SpringApplication               : Application startup failed

org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'mediaLoader' defined in class path resource [com/arcadeanalytics/util/MediaLoaderConfiguration.class]: Bean instantiation via factory method failed; nested exception is org.springframework.beans.BeanInstantiationException: Failed to instantiate [com.arcadeanalytics.util.MediaLoader]: Factory method 'mediaLoader' threw exception; nested exception is java.io.FileNotFoundException: ServletContext resource [/content/media/] cannot be resolved to URL because it does not exist
        at org.springframework.beans.factory.support.ConstructorResolver.instantiateUsingFactoryMethod(ConstructorResolver.java:599)
        at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.instantiateUsingFactoryMethod(AbstractAutowireCapableBeanFactory.java:1177)
        at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBeanInstance(AbstractAutowireCapableBeanFactory.java:1071)
        at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:513)
        at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:483)
        at org.springframework.beans.factory.support.AbstractBeanFactory$1.getObject(AbstractBeanFactory.java:312)
        at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:230)
        at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:308)
        at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:197)
        at org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:761)
        at org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:867)
        at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:543)
        at org.springframework.boot.context.embedded.EmbeddedWebApplicationContext.refresh(EmbeddedWebApplicationContext.java:122)
        at org.springframework.boot.SpringApplication.refresh(SpringApplication.java:693)
        at org.springframework.boot.SpringApplication.refreshContext(SpringApplication.java:360)
        at org.springframework.boot.SpringApplication.run(SpringApplication.java:303)
        at com.arcadeanalytics.ArcadeanalyticsApp.main(ArcadeanalyticsApp.java:46)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at org.springframework.boot.loader.MainMethodRunner.run(MainMethodRunner.java:48)
        at org.springframework.boot.loader.Launcher.launch(Launcher.java:87)
        at org.springframework.boot.loader.Launcher.launch(Launcher.java:50)
        at org.springframework.boot.loader.WarLauncher.main(WarLauncher.java:59)
Caused by: org.springframework.beans.BeanInstantiationException: Failed to instantiate [com.arcadeanalytics.util.MediaLoader]: Factory method 'mediaLoader' threw exception; nested exception is java.io.FileNotFoundException: ServletContext resource [/content/media/] cannot be resolved to URL because it does not exist
        at org.springframework.beans.factory.support.SimpleInstantiationStrategy.instantiate(SimpleInstantiationStrategy.java:189)
        at org.springframework.beans.factory.support.ConstructorResolver.instantiateUsingFactoryMethod(ConstructorResolver.java:588)
        ... 24 common frames omitted
Caused by: java.io.FileNotFoundException: ServletContext resource [/content/media/] cannot be resolved to URL because it does not exist
        at org.springframework.web.context.support.ServletContextResource.getURL(ServletContextResource.java:156)
        at org.springframework.core.io.support.PathMatchingResourcePatternResolver.findPathMatchingResources(PathMatchingResourcePatternResolver.java:492)
        at org.springframework.core.io.support.PathMatchingResourcePatternResolver.getResources(PathMatchingResourcePatternResolver.java:293)
        at org.springframework.context.support.AbstractApplicationContext.getResources(AbstractApplicationContext.java:1295)
        at org.springframework.context.support.GenericApplicationContext.getResources(GenericApplicationContext.java:228)
        at com.arcadeanalytics.util.MediaLoader.loadImages(MediaLoader.kt:22)
        at com.arcadeanalytics.util.MediaLoaderConfiguration.mediaLoader(MediaLoaderConfiguration.kt:25)
        at com.arcadeanalytics.util.MediaLoaderConfiguration$$EnhancerBySpringCGLIB$$6c9a9b65.CGLIB$mediaLoader$0(<generated>)
        at com.arcadeanalytics.util.MediaLoaderConfiguration$$EnhancerBySpringCGLIB$$6c9a9b65$$FastClassBySpringCGLIB$$640334b0.invoke(<generated>)
        at org.springframework.cglib.proxy.MethodProxy.invokeSuper(MethodProxy.java:228)
        at org.springframework.context.annotation.ConfigurationClassEnhancer$BeanMethodInterceptor.intercept(ConfigurationClassEnhancer.java:358)
        at com.arcadeanalytics.util.MediaLoaderConfiguration$$EnhancerBySpringCGLIB$$6c9a9b65.mediaLoader(<generated>)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at org.springframework.beans.factory.support.SimpleInstantiationStrategy.instantiate(SimpleInstantiationStrategy.java:162)
        ... 25 common frames omitted

ericmillsio avatar Mar 06 '19 15:03 ericmillsio

in which way do you run the application? Are you using mvn ? (we moved to maven last week) Or the container?

robfrank avatar Mar 06 '19 16:03 robfrank

Hi I am also trying to get Arcade Analytics to run internally against Neptune. The application seems to install fine using Ubuntu 18.04 clean image with Java 1.8 using Maven. I am presented with:

    Application 'arcadeanalytics' is running! Access URLs:
    Local:          http://localhost:8080
    External:       http://172.18.11.192:8080
    Profile(s):     [swagger, dev]

Which looks correct. However when I go to the URL I get a 404 on http://172.18.11.192:8080/ckeditor/ckeditor.js.

The steps were pretty straight forward:

  1. Installed java
  2. Installed maven
  3. Installed git
  4. Cloned your github repo
  5. Ran Maven

Am I missing anything? Thanks!

msg-q avatar Mar 06 '19 20:03 msg-q

if you run from the sources you need two shell:

  • first shell: ./mvnw
  • second shell : 'yarn start'

so you need yarn/node installed too

robfrank avatar Mar 06 '19 21:03 robfrank

That got it running but there are a bunch of errors with the GUI connecting to localhost. Would it be possible for you guys build an AWS EC2 Image and just add it to the AWS Marketplace? It would be much easier for people to adopt.

msg-q avatar Mar 06 '19 23:03 msg-q

@msg-q we provide the docker image, is it enough?

robfrank avatar Mar 07 '19 06:03 robfrank

RobFrank, appreciate the work you are putting in here.

My opinion is it depends on who you want to adopt your software. If you want developers running it locally on their laptops to adopt, then the docker containers are fine. If you want businesses who are running in AWS, adding it to the marketplace as an AMI and having an image that is tested and easy to spin up would be better.

Just my opinion and I hope it helps!

msg-q avatar Mar 11 '19 23:03 msg-q

Is there a way to automate AMI creation as we do with Docker images? Any help in the form of script or tutorial link is appreciated.

robfrank avatar Mar 12 '19 08:03 robfrank

Hello everyone,

I cannot find the public key from arcade to add it to my authorized keys in EC2, can someone help me?

farari7 avatar Mar 15 '19 15:03 farari7

Automatic AMI and Docker Image creation is very easy to do with Packer.

ericmillsio avatar Mar 15 '19 15:03 ericmillsio

@farari7 please read the readme, I added some doc on how use your own ssh keys to connect to your own neptune.

robfrank avatar Mar 15 '19 15:03 robfrank

Ok I did all the steps (create the keys -with name id_rsa-, place it on .ssh directory and add the public key to my EC2 instance and I get the following error: java.nio.file.NoSuchFileException: /.arcade/.ssh/id_rsa .

It looks like I am getting in but then it cannot read my key and it drop out.

farari7 avatar Mar 18 '19 18:03 farari7

If the application emits the error:

java.nio.file.NoSuchFileException: /.arcade/.ssh/id_rsa

you placed the ssh keys in the wrong place. The path printed out isn't equal to the one provided in the compose file. Did you modify it?

This snippet is from the app.yml descriptor

      - JAVA_OPTS=-DSSH_PRIV_KEY=/arcade/.ssh/id_rsa -DSSH_PUB_KEY=/arcade/.ssh/id_rsa.pub
    ports:
      - 8080:8080
    volumes:
      - ~/.arcade/:/arcade/

from the readme: Create a directory in ~/.arcade/ named .ssh and put the private and public keys inside. The public key should be copied on the server used as ssh gateway too.

So, after the first run using the provided compose file, you should have in your home dir in the host a hidden directory called .arcade

I need more details to help you:

  • configuration files if you have modified them
  • commands used to run the application
  • operative system
  • any other useful detail

robfrank avatar Mar 19 '19 14:03 robfrank

Ok this is what I am doing:

  • I am launching Arcade with docker with: docker-compose -f src/main/docker/app.yml up. -My operating system is Mac OS High Sierra.
  • This is how I am trying to connect to my Neptune database (for the Gateway i am using the public IP of my EC2 instance and for the server the endpoint of my Neptune VPC):

Captura de pantalla 2019-03-19 a la(s) 12 47 57

I didn't move any file but I am going to erase it and download it again. Literally what I did was to create the keys, place them on the .ssh file (inside .arcade) and share the public key to my EC2 instance.

farari7 avatar Mar 19 '19 18:03 farari7