travis fails on dbLoad

[laurelnaiad]

travis has very unforgiving network and sometimes can't get a connection to developer.marklogic.com.

Isolate attempt to fetch seed data into a curl command so that the special "travis_retry" script may be used to download the seed data, and then override the gradle command to use the downloaded seed data instead of attempting the http download through java.

Charles has already done the gradle work and I've already done the other works, so this is ready to go.

No impact on end-users or non-travis scenarios, so the blast radius is confined only to travis and has just been tested/works in my branch. @gghai, may I merge it? (it's on branch laurelnaiad/seed-config)

[gghai]

Not impacting any of the QA testing , please go ahead with the check-in .Thanks

[grechaw]

You know, your solution makes sense generally -- fetch uses URL.openStream() which is not a fault-tolerant way to get files. If we get reports of issues outside of travis then we can consider a retryable http connection from the task too

[laurelnaiad]

Here is a stacktrace of the dbLoad failure on travis (this is when it is supposed to be loading the seed data from a local file). Obviously it's not telling us anything special -- no funky permissions problem or anything like that. It seems to work about 1 out of every 6-8 builds attempts.

./gradlew dbLoad -PseedDataUrl=file:///home/travis/build/marklogic/marklogic-samplestack/seed-data1.8.1.tgz --stacktrace
.............................................................
FAILURE:
Build failed with an exception.
* What went wrong:
Execution failed for task ':dbLoad'.
>
org.apache.http.NoHttpResponseException: The target server failed to respond
* Try:
Run with
--info
or
--debug option to get more log output.
* Exception is:
org.gradle.api.tasks.TaskExecutionException: Execution failed for task ':dbLoad'.
at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.executeActions(ExecuteActionsTaskExecuter.java:69)
    at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.execute(ExecuteActionsTaskExecuter.java:46)
at org.gradle.api.internal.tasks.execution.PostExecutionAnalysisTaskExecuter.execute(PostExecutionAnalysisTaskExecuter.java:35)
at org.gradle.api.internal.tasks.execution.SkipUpToDateTaskExecuter.execute(SkipUpToDateTaskExecuter.java:64)
at org.gradle.api.internal.tasks.execution.ValidatingTaskExecuter.execute(ValidatingTaskExecuter.java:58)
at org.gradle.api.internal.tasks.execution.SkipEmptySourceFilesTaskExecuter.execute(SkipEmptySourceFilesTaskExecuter.java:42)
at org.gradle.api.internal.tasks.execution.SkipTaskWithNoActionsExecuter.execute(SkipTaskWithNoActionsExecuter.java:52)
at org.gradle.api.internal.tasks.execution.SkipOnlyIfTaskExecuter.execute(SkipOnlyIfTaskExecuter.java:53)
at org.gradle.api.internal.tasks.execution.ExecuteAtMostOnceTaskExecuter.execute(ExecuteAtMostOnceTaskExecuter.java:43)
at org.gradle.api.internal.AbstractTask.executeWithoutThrowingTaskFailure(AbstractTask.java:305)
at org.gradle.execution.taskgraph.AbstractTaskPlanExecutor$TaskExecutorWorker.executeTask(AbstractTaskPlanExecutor.java:79)
at org.gradle.execution.taskgraph.AbstractTaskPlanExecutor$TaskExecutorWorker.processTask(AbstractTaskPlanExecutor.java:63)
at org.gradle.execution.taskgraph.AbstractTaskPlanExecutor$TaskExecutorWorker.run(AbstractTaskPlanExecutor.java:51)
at org.gradle.execution.taskgraph.DefaultTaskPlanExecutor.process(DefaultTaskPlanExecutor.java:23)
at org.gradle.execution.taskgraph.DefaultTaskGraphExecuter.execute(DefaultTaskGraphExecuter.java:88)
at org.gradle.execution.SelectedTaskExecutionAction.execute(SelectedTaskExecutionAction.java:29)
at org.gradle.execution.DefaultBuildExecuter.execute(DefaultBuildExecuter.java:62)
at org.gradle.execution.DefaultBuildExecuter.access$200(DefaultBuildExecuter.java:23)
at org.gradle.execution.DefaultBuildExecuter$2.proceed(DefaultBuildExecuter.java:68)
at org.gradle.execution.DryRunBuildExecutionAction.execute(DryRunBuildExecutionAction.java:32)
at org.gradle.execution.DefaultBuildExecuter.execute(DefaultBuildExecuter.java:62)
at org.gradle.execution.DefaultBuildExecuter.execute(DefaultBuildExecuter.java:55)
at org.gradle.initialization.DefaultGradleLauncher.doBuildStages(DefaultGradleLauncher.java:149)
at org.gradle.initialization.DefaultGradleLauncher.doBuild(DefaultGradleLauncher.java:106)
at org.gradle.initialization.DefaultGradleLauncher.run(DefaultGradleLauncher.java:86)
at org.gradle.launcher.exec.InProcessBuildActionExecuter$DefaultBuildController.run(InProcessBuildActionExecuter.java:80)
at org.gradle.launcher.cli.ExecuteBuildAction.run(ExecuteBuildAction.java:33)
at org.gradle.launcher.cli.ExecuteBuildAction.run(ExecuteBuildAction.java:24)
at org.gradle.launcher.exec.InProcessBuildActionExecuter.execute(InProcessBuildActionExecuter.java:36)
at org.gradle.launcher.exec.InProcessBuildActionExecuter.execute(InProcessBuildActionExecuter.java:26)
at org.gradle.launcher.cli.RunBuildAction.run(RunBuildAction.java:51)
at org.gradle.internal.Actions$RunnableActionAdapter.execute(Actions.java:171)
at org.gradle.launcher.cli.CommandLineActionFactory$ParseAndBuildAction.execute(CommandLineActionFactory.java:237)
at org.gradle.launcher.cli.CommandLineActionFactory$ParseAndBuildAction.execute(CommandLineActionFactory.java:210)
at org.gradle.launcher.cli.JavaRuntimeValidationAction.execute(JavaRuntimeValidationAction.java:35)
at org.gradle.launcher.cli.JavaRuntimeValidationAction.execute(JavaRuntimeValidationAction.java:24)
at org.gradle.launcher.cli.CommandLineActionFactory$WithLogging.execute(CommandLineActionFactory.java:206)
at org.gradle.launcher.cli.CommandLineActionFactory$WithLogging.execute(CommandLineActionFactory.java:169)
at org.gradle.launcher.cli.ExceptionReportingAction.execute(ExceptionReportingAction.java:33)
at org.gradle.launcher.cli.ExceptionReportingAction.execute(ExceptionReportingAction.java:22)
at org.gradle.launcher.Main.doAction(Main.java:33)
at org.gradle.launcher.bootstrap.EntryPoint.run(EntryPoint.java:45)
at org.gradle.launcher.bootstrap.ProcessBootstrap.runNoExit(ProcessBootstrap.java:54)
at org.gradle.launcher.bootstrap.ProcessBootstrap.run(ProcessBootstrap.java:35)
at org.gradle.launcher.GradleMain.main(GradleMain.java:23)
at org.gradle.wrapper.BootstrapMainStarter.start(BootstrapMainStarter.java:30)
at org.gradle.wrapper.WrapperExecutor.execute(WrapperExecutor.java:127)
at org.gradle.wrapper.GradleWrapperMain.main(GradleWrapperMain.java:56)
Caused by: com.sun.jersey.api.client.ClientHandlerException: org.apache.http.NoHttpResponseException: The target server failed to respond
at com.sun.jersey.client.apache4.ApacheHttpClient4Handler.handle(ApacheHttpClient4Handler.java:184)
at com.marklogic.client.impl.DigestChallengeFilter.handle(DigestChallengeFilter.java:34)
at com.sun.jersey.api.client.filter.HTTPDigestAuthFilter.handle(HTTPDigestAuthFilter.java:493)
at com.sun.jersey.api.client.Client.handle(Client.java:648)
at com.sun.jersey.api.client.WebResource.handle(WebResource.java:680)
at com.sun.jersey.api.client.WebResource.access$200(WebResource.java:74)
at com.sun.jersey.api.client.WebResource$Builder.post(WebResource.java:568)
at com.marklogic.client.impl.JerseyServices.doPost(JerseyServices.java:4192)
at com.marklogic.client.impl.JerseyServices.postResource(JerseyServices.java:3459)
at com.marklogic.client.impl.JerseyServices.postBulkDocuments(JerseyServices.java:3571)
at com.marklogic.client.impl.DocumentManagerImpl.write(DocumentManagerImpl.java:553)
at com.marklogic.client.impl.JSONDocumentImpl.write(JSONDocumentImpl.java:1)
at com.marklogic.client.impl.DocumentManagerImpl.write(DocumentManagerImpl.java:539)
at com.marklogic.client.impl.JSONDocumentImpl.write(JSONDocumentImpl.java:1)
at com.marklogic.client.document.DocumentManager$write$0.call(Unknown Source)
at MarkLogicSlurpTask$_load_closure2.doCall(MarkLogicSlurpTask.groovy:58)
at MarkLogicSlurpTask.load(MarkLogicSlurpTask.groovy:46)
at org.gradle.internal.reflect.JavaMethod.invoke(JavaMethod.java:63)
at org.gradle.api.internal.project.taskfactory.AnnotationProcessingTaskFactory$StandardTaskAction.doExecute(AnnotationProcessingTaskFactory.java:218)
at org.gradle.api.internal.project.taskfactory.AnnotationProcessingTaskFactory$StandardTaskAction.execute(AnnotationProcessingTaskFactory.java:211)
at org.gradle.api.internal.project.taskfactory.AnnotationProcessingTaskFactory$StandardTaskAction.execute(AnnotationProcessingTaskFactory.java:200)
at org.gradle.api.internal.AbstractTask$TaskActionWrapper.execute(AbstractTask.java:579)
at org.gradle.api.internal.AbstractTask$TaskActionWrapper.execute(AbstractTask.java:562)
at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.executeAction(ExecuteActionsTaskExecuter.java:80)
at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.executeActions(ExecuteActionsTaskExecuter.java:61)
... 47 more
Caused by: org.apache.http.NoHttpResponseException: The target server failed to respond
at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:95)
at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:62)
at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:254)
at org.apache.http.impl.AbstractHttpClientConnection.receiveResponseHeader(AbstractHttpClientConnection.java:289)
at org.apache.http.impl.conn.DefaultClientConnection.receiveResponseHeader(DefaultClientConnection.java:252)
at org.apache.http.impl.conn.AbstractClientConnAdapter.receiveResponseHeader(AbstractClientConnAdapter.java:219)
at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:300)
at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:127)
at org.apache.http.impl.client.DefaultRequestDirector.tryExecute(DefaultRequestDirector.java:712)
at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:517)
at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:906)
at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:827)
at com.sun.jersey.client.apache4.ApacheHttpClient4Handler.handle(ApacheHttpClient4Handler.java:170)
... 71 more
. 

[grechaw]

I think we worked this issue Friday -- and that this particular error came from the lack of absolute paths on the travis machine, right @laurelnaiad ?

[laurelnaiad]

No, it just fails most of the time and sometimes succeeds -- seems independent of whether we read from file: or http:.

I suspect a bug in their ubuntu stack's streaming capability or something like that. I think an flag to read from a "regular" local file instead of file: would probably be a reliable workaround.

[laurelnaiad]

I'm not dead set on 8.0-1 for this, BTW. In fact, I'm changing it to 8.0-2 and if anyone objects we can discuss.

[popzip]

was this meant to close?

[laurelnaiad]

No, that commit which was intended to close it didn't fix the issue. Current WIP is https://github.com/marklogic/marklogic-samplestack/pull/573. I would like to get some version of that PR merged because it makes travis a more realistic place to run tests and have some expectation of success. :)

[grechaw]

Hi @laurelnaiad I'm giving this to you just because you're more on top of this right now.