etl
etl copied to clipboard
Published
20 hours ago •
linkedpipes
linkedpipes
docker storage container keeps restarting
I'm proably doing something wrong here. I followed the readme on the letter. De application runs but I get errors on the pipeline page.
Can't load data
Request failed with status: 500.
Please try it again later.
Closer inspections shows an 500 internal server error in http://localhost:8080/api/v1/pipelines-list with the following response:
{
"error": {
"type": "errors.CONNECTION",
"source": "FRONTEND"
}
}
Closer looking at the docker containers the storage container keeps restarting I think that is related.
Sorry for the long docker log...
2024-02-02 16:52:18 15:52:18,100 [main ] DEBUG com.linkedpipes.etl.storage.cli.ConfigurationLoader - Reading configuration from: /opt/lp-etl/configuration/configuration.properties
2024-02-02 16:52:18 15:52:18,103 [main ] WARN com.linkedpipes.etl.storage.cli.ConfigurationLoader - Properties configuration files are deprecated.
2024-02-02 16:52:18 15:52:18,680 [main ] DEBUG com.linkedpipes.etl.storage.plugin.JavaPluginService - Loading plugins ...
2024-02-02 16:52:19 15:52:19,337 [main ] INFO com.linkedpipes.etl.storage.plugin.JavaPluginService - Loading plugins ... done with 97 plugins
2024-02-02 16:52:19 15:52:19,341 [main ] WARN org.springframework.context.support.ClassPathXmlApplicationContext - Exception encountered during context initialization - cancelling refresh attempt: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'storageService': Invocation of init method failed
2024-02-02 16:52:19 Exception in thread "main" org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'storageService': Invocation of init method failed
2024-02-02 16:52:19 at org.springframework.beans.factory.annotation.InitDestroyAnnotationBeanPostProcessor.postProcessBeforeInitialization(InitDestroyAnnotationBeanPostProcessor.java:196)
2024-02-02 16:52:19 at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.applyBeanPostProcessorsBeforeInitialization(AbstractAutowireCapableBeanFactory.java:419)
2024-02-02 16:52:19 at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1762)
2024-02-02 16:52:19 at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:598)
2024-02-02 16:52:19 at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:520)
2024-02-02 16:52:19 at org.springframework.beans.factory.support.AbstractBeanFactory.lambda$doGetBean$0(AbstractBeanFactory.java:326)
2024-02-02 16:52:19 at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:234)
2024-02-02 16:52:19 at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:324)
2024-02-02 16:52:19 at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:200)
2024-02-02 16:52:19 at org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:973)
2024-02-02 16:52:19 at org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:941)
2024-02-02 16:52:19 at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:608)
2024-02-02 16:52:19 at org.springframework.context.support.ClassPathXmlApplicationContext.<init>(ClassPathXmlApplicationContext.java:144)
2024-02-02 16:52:19 at org.springframework.context.support.ClassPathXmlApplicationContext.<init>(ClassPathXmlApplicationContext.java:85)
2024-02-02 16:52:19 at com.linkedpipes.etl.storage.Storage.startSpring(Storage.java:87)
2024-02-02 16:52:19 at com.linkedpipes.etl.storage.Storage.run(Storage.java:33)
2024-02-02 16:52:19 at com.linkedpipes.etl.storage.Storage.main(Storage.java:23)
2024-02-02 16:52:19 Caused by: com.linkedpipes.etl.storage.StorageException: Can't read repository info file '/data/lp-etl/storage/repository-info.json'.
2024-02-02 16:52:19 at com.linkedpipes.etl.storage.template.repository.RepositoryInfo.load(RepositoryInfo.java:44)
2024-02-02 16:52:19 at com.linkedpipes.etl.storage.template.repository.TemplateRepositoryFactory.loadRepositoryInfo(TemplateRepositoryFactory.java:86)
2024-02-02 16:52:19 at com.linkedpipes.etl.storage.template.repository.TemplateRepositoryFactory.create(TemplateRepositoryFactory.java:37)
2024-02-02 16:52:19 at com.linkedpipes.etl.storage.StorageService.initializeTemplateRepository(StorageService.java:109)
2024-02-02 16:52:19 at com.linkedpipes.etl.storage.StorageService.initialize(StorageService.java:72)
2024-02-02 16:52:19 at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
2024-02-02 16:52:19 at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
2024-02-02 16:52:19 at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
2024-02-02 16:52:19 at java.base/java.lang.reflect.Method.invoke(Method.java:568)
2024-02-02 16:52:19 at org.springframework.beans.factory.annotation.InitDestroyAnnotationBeanPostProcessor$LifecycleElement.invoke(InitDestroyAnnotationBeanPostProcessor.java:425)
2024-02-02 16:52:19 at org.springframework.beans.factory.annotation.InitDestroyAnnotationBeanPostProcessor$LifecycleMetadata.invokeInitMethods(InitDestroyAnnotationBeanPostProcessor.java:369)
2024-02-02 16:52:19 at org.springframework.beans.factory.annotation.InitDestroyAnnotationBeanPostProcessor.postProcessBeforeInitialization(InitDestroyAnnotationBeanPostProcessor.java:193)
2024-02-02 16:52:19 ... 16 more
2024-02-02 16:52:19 Caused by: com.fasterxml.jackson.databind.exc.MismatchedInputException: No content to map due to end-of-input
2024-02-02 16:52:19 at [Source: (File); line: 1, column: 0]
2024-02-02 16:52:19 at com.fasterxml.jackson.databind.exc.MismatchedInputException.from(MismatchedInputException.java:59)
2024-02-02 16:52:19 at com.fasterxml.jackson.databind.ObjectMapper._initForReading(ObjectMapper.java:4916)
2024-02-02 16:52:19 at com.fasterxml.jackson.databind.ObjectMapper._readMapAndClose(ObjectMapper.java:4818)
2024-02-02 16:52:19 at com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:3637)
2024-02-02 16:52:19 at com.linkedpipes.etl.storage.template.repository.RepositoryInfo.load(RepositoryInfo.java:42)
2024-02-02 16:52:19 ... 27 more
2024-02-02 16:53:21 Exception in thread "main" org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'storageService': Invocation of init method failed
2024-02-02 16:53:21 at org.springframework.beans.factory.annotation.InitDestroyAnnotationBeanPostProcessor.postProcessBeforeInitialization(InitDestroyAnnotationBeanPostProcessor.java:196)
2024-02-02 16:53:21 at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.applyBeanPostProcessorsBeforeInitialization(AbstractAutowireCapableBeanFactory.java:419)
2024-02-02 16:53:21 at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1762)
2024-02-02 16:53:21 at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:598)
2024-02-02 16:53:21 at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:520)
2024-02-02 16:53:21 at org.springframework.beans.factory.support.AbstractBeanFactory.lambda$doGetBean$0(AbstractBeanFactory.java:326)
2024-02-02 16:53:21 at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:234)
2024-02-02 16:53:21 at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:324)
2024-02-02 16:53:21 at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:200)
2024-02-02 16:53:21 at org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:973)
2024-02-02 16:53:21 at org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:941)
2024-02-02 16:53:21 at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:608)
2024-02-02 16:53:21 at org.springframework.context.support.ClassPathXmlApplicationContext.<init>(ClassPathXmlApplicationContext.java:144)
2024-02-02 16:53:21 at org.springframework.context.support.ClassPathXmlApplicationContext.<init>(ClassPathXmlApplicationContext.java:85)
2024-02-02 16:53:20 15:53:20,143 [main ] DEBUG com.linkedpipes.etl.storage.cli.ConfigurationLoader - Reading configuration from: /opt/lp-etl/configuration/configuration.properties
2024-02-02 16:53:20 15:53:20,147 [main ] WARN com.linkedpipes.etl.storage.cli.ConfigurationLoader - Properties configuration files are deprecated.
2024-02-02 16:53:20 15:53:20,916 [main ] DEBUG com.linkedpipes.etl.storage.plugin.JavaPluginService - Loading plugins ...
2024-02-02 16:53:21 15:53:21,808 [main ] INFO com.linkedpipes.etl.storage.plugin.JavaPluginService - Loading plugins ... done with 97 plugins
2024-02-02 16:53:21 15:53:21,815 [main ] WARN org.springframework.context.support.ClassPathXmlApplicationContext - Exception encountered during context initialization - cancelling refresh attempt: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'storageService': Invocation of init method failed
2024-02-02 16:53:21 at com.linkedpipes.etl.storage.Storage.startSpring(Storage.java:87)
2024-02-02 16:53:21 at com.linkedpipes.etl.storage.Storage.run(Storage.java:33)
2024-02-02 16:53:21 at com.linkedpipes.etl.storage.Storage.main(Storage.java:23)
2024-02-02 16:53:21 Caused by: com.linkedpipes.etl.storage.StorageException: Can't read repository info file '/data/lp-etl/storage/repository-info.json'.
2024-02-02 16:53:21 at com.linkedpipes.etl.storage.template.repository.RepositoryInfo.load(RepositoryInfo.java:44)
2024-02-02 16:53:21 at com.linkedpipes.etl.storage.template.repository.TemplateRepositoryFactory.loadRepositoryInfo(TemplateRepositoryFactory.java:86)
2024-02-02 16:53:21 at com.linkedpipes.etl.storage.template.repository.TemplateRepositoryFactory.create(TemplateRepositoryFactory.java:37)
2024-02-02 16:53:21 at com.linkedpipes.etl.storage.StorageService.initializeTemplateRepository(StorageService.java:109)
2024-02-02 16:53:21 at com.linkedpipes.etl.storage.StorageService.initialize(StorageService.java:72)
2024-02-02 16:53:21 at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
2024-02-02 16:53:21 at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
2024-02-02 16:53:21 at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
2024-02-02 16:53:21 at java.base/java.lang.reflect.Method.invoke(Method.java:568)
2024-02-02 16:53:21 at org.springframework.beans.factory.annotation.InitDestroyAnnotationBeanPostProcessor$LifecycleElement.invoke(InitDestroyAnnotationBeanPostProcessor.java:425)
2024-02-02 16:53:21 at org.springframework.beans.factory.annotation.InitDestroyAnnotationBeanPostProcessor$LifecycleMetadata.invokeInitMethods(InitDestroyAnnotationBeanPostProcessor.java:369)
2024-02-02 16:53:21 at org.springframework.beans.factory.annotation.InitDestroyAnnotationBeanPostProcessor.postProcessBeforeInitialization(InitDestroyAnnotationBeanPostProcessor.java:193)
2024-02-02 16:53:21 ... 16 more
2024-02-02 16:53:21 Caused by: com.fasterxml.jackson.databind.exc.MismatchedInputException: No content to map due to end-of-input
2024-02-02 16:53:21 at [Source: (File); line: 1, column: 0]
2024-02-02 16:53:21 at com.fasterxml.jackson.databind.exc.MismatchedInputException.from(MismatchedInputException.java:59)
2024-02-02 16:53:21 at com.fasterxml.jackson.databind.ObjectMapper._initForReading(ObjectMapper.java:4916)
2024-02-02 16:53:21 at com.fasterxml.jackson.databind.ObjectMapper._readMapAndClose(ObjectMapper.java:4818)
2024-02-02 16:53:21 at com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:3637)
2024-02-02 16:53:21 at com.linkedpipes.etl.storage.template.repository.RepositoryInfo.load(RepositoryInfo.java:42)
2024-02-02 16:53:21 ... 27 more
2024-02-02 16:54:24 Exception in thread "main" org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'storageService': Invocation of init method failed
2024-02-02 16:54:24 at org.springframework.beans.factory.annotation.InitDestroyAnnotationBeanPostProcessor.postProcessBeforeInitialization(InitDestroyAnnotationBeanPostProcessor.java:196)
2024-02-02 16:54:24 at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.applyBeanPostProcessorsBeforeInitialization(AbstractAutowireCapableBeanFactory.java:419)
2024-02-02 16:54:24 at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1762)
2024-02-02 16:54:24 at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:598)
2024-02-02 16:54:24 at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:520)
2024-02-02 16:54:24 at org.springframework.beans.factory.support.AbstractBeanFactory.lambda$doGetBean$0(AbstractBeanFactory.java:326)
2024-02-02 16:54:24 at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:234)
2024-02-02 16:54:24 at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:324)
2024-02-02 16:54:24 at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:200)
2024-02-02 16:54:24 at org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:973)
2024-02-02 16:54:24 at org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:941)
2024-02-02 16:54:24 at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:608)
2024-02-02 16:54:24 at org.springframework.context.support.ClassPathXmlApplicationContext.<init>(ClassPathXmlApplicationContext.java:144)
2024-02-02 16:54:24 at org.springframework.context.support.ClassPathXmlApplicationContext.<init>(ClassPathXmlApplicationContext.java:85)
2024-02-02 16:54:24 at com.linkedpipes.etl.storage.Storage.startSpring(Storage.java:87)
2024-02-02 16:54:24 at com.linkedpipes.etl.storage.Storage.run(Storage.java:33)
2024-02-02 16:54:24 at com.linkedpipes.etl.storage.Storage.main(Storage.java:23)
2024-02-02 16:54:24 Caused by: com.linkedpipes.etl.storage.StorageException: Can't read repository info file '/data/lp-etl/storage/repository-info.json'.
2024-02-02 16:54:24 at com.linkedpipes.etl.storage.template.repository.RepositoryInfo.load(RepositoryInfo.java:44)
2024-02-02 16:54:24 at com.linkedpipes.etl.storage.template.repository.TemplateRepositoryFactory.loadRepositoryInfo(TemplateRepositoryFactory.java:86)
2024-02-02 16:54:24 at com.linkedpipes.etl.storage.template.repository.TemplateRepositoryFactory.create(TemplateRepositoryFactory.java:37)
2024-02-02 16:54:22 15:54:22,649 [main ] DEBUG com.linkedpipes.etl.storage.cli.ConfigurationLoader - Reading configuration from: /opt/lp-etl/configuration/configuration.properties
2024-02-02 16:54:22 15:54:22,653 [main ] WARN com.linkedpipes.etl.storage.cli.ConfigurationLoader - Properties configuration files are deprecated.
2024-02-02 16:54:23 15:54:23,372 [main ] DEBUG com.linkedpipes.etl.storage.plugin.JavaPluginService - Loading plugins ...
2024-02-02 16:54:24 15:54:24,185 [main ] INFO com.linkedpipes.etl.storage.plugin.JavaPluginService - Loading plugins ... done with 97 plugins
2024-02-02 16:54:24 15:54:24,190 [main ] WARN org.springframework.context.support.ClassPathXmlApplicationContext - Exception encountered during context initialization - cancelling refresh attempt: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'storageService': Invocation of init method failed
2024-02-02 16:54:24 at com.linkedpipes.etl.storage.StorageService.initializeTemplateRepository(StorageService.java:109)
2024-02-02 16:54:24 at com.linkedpipes.etl.storage.StorageService.initialize(StorageService.java:72)
2024-02-02 16:54:24 at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
2024-02-02 16:54:24 at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
2024-02-02 16:54:24 at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
2024-02-02 16:54:24 at java.base/java.lang.reflect.Method.invoke(Method.java:568)
2024-02-02 16:54:24 at org.springframework.beans.factory.annotation.InitDestroyAnnotationBeanPostProcessor$LifecycleElement.invoke(InitDestroyAnnotationBeanPostProcessor.java:425)
2024-02-02 16:54:24 at org.springframework.beans.factory.annotation.InitDestroyAnnotationBeanPostProcessor$LifecycleMetadata.invokeInitMethods(InitDestroyAnnotationBeanPostProcessor.java:369)
2024-02-02 16:54:24 at org.springframework.beans.factory.annotation.InitDestroyAnnotationBeanPostProcessor.postProcessBeforeInitialization(InitDestroyAnnotationBeanPostProcessor.java:193)
2024-02-02 16:54:24 ... 16 more
2024-02-02 16:54:24 Caused by: com.fasterxml.jackson.databind.exc.MismatchedInputException: No content to map due to end-of-input
2024-02-02 16:54:24 at [Source: (File); line: 1, column: 0]
2024-02-02 16:54:24 at com.fasterxml.jackson.databind.exc.MismatchedInputException.from(MismatchedInputException.java:59)
2024-02-02 16:54:24 at com.fasterxml.jackson.databind.ObjectMapper._initForReading(ObjectMapper.java:4916)
2024-02-02 16:54:24 at com.fasterxml.jackson.databind.ObjectMapper._readMapAndClose(ObjectMapper.java:4818)
2024-02-02 16:54:24 at com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:3637)
2024-02-02 16:54:24 at com.linkedpipes.etl.storage.template.repository.RepositoryInfo.load(RepositoryInfo.java:42)
2024-02-02 16:54:24 ... 27 more
2024-02-02 16:55:25 15:55:25,034 [main ] DEBUG com.linkedpipes.etl.storage.cli.ConfigurationLoader - Reading configuration from: /opt/lp-etl/configuration/configuration.properties
2024-02-02 16:55:25 15:55:25,037 [main ] WARN com.linkedpipes.etl.storage.cli.ConfigurationLoader - Properties configuration files are deprecated.
2024-02-02 16:55:25 15:55:25,622 [main ] DEBUG com.linkedpipes.etl.storage.plugin.JavaPluginService - Loading plugins ...
2024-02-02 16:55:26 15:55:26,345 [main ] INFO com.linkedpipes.etl.storage.plugin.JavaPluginService - Loading plugins ... done with 97 plugins
2024-02-02 16:55:26 15:55:26,349 [main ] WARN org.springframework.context.support.ClassPathXmlApplicationContext - Exception encountered during context initialization - cancelling refresh attempt: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'storageService': Invocation of init method failed
2024-02-02 16:55:26 Exception in thread "main" org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'storageService': Invocation of init method failed
2024-02-02 16:55:26 at org.springframework.beans.factory.annotation.InitDestroyAnnotationBeanPostProcessor.postProcessBeforeInitialization(InitDestroyAnnotationBeanPostProcessor.java:196)
2024-02-02 16:55:26 at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.applyBeanPostProcessorsBeforeInitialization(AbstractAutowireCapableBeanFactory.java:419)
2024-02-02 16:55:26 at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1762)
2024-02-02 16:55:26 at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:598)
2024-02-02 16:55:26 at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:520)
2024-02-02 16:55:26 at org.springframework.beans.factory.support.AbstractBeanFactory.lambda$doGetBean$0(AbstractBeanFactory.java:326)
2024-02-02 16:55:26 at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:234)
2024-02-02 16:55:26 at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:324)
2024-02-02 16:55:26 at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:200)
2024-02-02 16:55:26 at org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:973)
2024-02-02 16:55:26 at org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:941)
2024-02-02 16:55:26 at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:608)
2024-02-02 16:55:26 at org.springframework.context.support.ClassPathXmlApplicationContext.<init>(ClassPathXmlApplicationContext.java:144)
2024-02-02 16:55:26 at org.springframework.context.support.ClassPathXmlApplicationContext.<init>(ClassPathXmlApplicationContext.java:85)
2024-02-02 16:55:26 at com.linkedpipes.etl.storage.Storage.startSpring(Storage.java:87)
2024-02-02 16:55:26 at com.linkedpipes.etl.storage.Storage.run(Storage.java:33)
2024-02-02 16:55:26 at com.linkedpipes.etl.storage.Storage.main(Storage.java:23)
2024-02-02 16:55:26 Caused by: com.linkedpipes.etl.storage.StorageException: Can't read repository info file '/data/lp-etl/storage/repository-info.json'.
2024-02-02 16:55:26 at com.linkedpipes.etl.storage.template.repository.RepositoryInfo.load(RepositoryInfo.java:44)
2024-02-02 16:55:26 at com.linkedpipes.etl.storage.template.repository.TemplateRepositoryFactory.loadRepositoryInfo(TemplateRepositoryFactory.java:86)
2024-02-02 16:55:26 at com.linkedpipes.etl.storage.template.repository.TemplateRepositoryFactory.create(TemplateRepositoryFactory.java:37)
2024-02-02 16:55:26 at com.linkedpipes.etl.storage.StorageService.initializeTemplateRepository(StorageService.java:109)
2024-02-02 16:55:26 at com.linkedpipes.etl.storage.StorageService.initialize(StorageService.java:72)
2024-02-02 16:55:26 at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
2024-02-02 16:55:26 at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
2024-02-02 16:55:26 at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
2024-02-02 16:55:26 at java.base/java.lang.reflect.Method.invoke(Method.java:568)
2024-02-02 16:55:26 at org.springframework.beans.factory.annotation.InitDestroyAnnotationBeanPostProcessor$LifecycleElement.invoke(InitDestroyAnnotationBeanPostProcessor.java:425)
2024-02-02 16:55:26 at org.springframework.beans.factory.annotation.InitDestroyAnnotationBeanPostProcessor$LifecycleMetadata.invokeInitMethods(InitDestroyAnnotationBeanPostProcessor.java:369)
2024-02-02 16:55:26 at org.springframework.beans.factory.annotation.InitDestroyAnnotationBeanPostProcessor.postProcessBeforeInitialization(InitDestroyAnnotationBeanPostProcessor.java:193)
2024-02-02 16:55:26 ... 16 more
2024-02-02 16:55:26 Caused by: com.fasterxml.jackson.databind.exc.MismatchedInputException: No content to map due to end-of-input
2024-02-02 16:55:26 at [Source: (File); line: 1, column: 0]
2024-02-02 16:55:26 at com.fasterxml.jackson.databind.exc.MismatchedInputException.from(MismatchedInputException.java:59)
2024-02-02 16:55:26 at com.fasterxml.jackson.databind.ObjectMapper._initForReading(ObjectMapper.java:4916)
2024-02-02 16:55:26 at com.fasterxml.jackson.databind.ObjectMapper._readMapAndClose(ObjectMapper.java:4818)
2024-02-02 16:55:26 at com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:3637)
2024-02-02 16:55:26 at com.linkedpipes.etl.storage.template.repository.RepositoryInfo.load(RepositoryInfo.java:42)
2024-02-02 16:55:26 ... 27 more
Is there something I'm missing/doing wrong here?
Dear @Pin0 thank you for your report. To be sure, you just run following locally?
git clone https://github.com/linkedpipes/etl.git
cd etl
docker compose up
@Pin0 can you describe your system where you are trying this? I just tried and works for me on:
Windows 11 Docker Desktop 4.27.1 (136059)
PS C:\Users\Kuba> git clone https://github.com/linkedpipes/etl.git
Cloning into 'etl'...
remote: Enumerating objects: 57363, done.
remote: Counting objects: 100% (8509/8509), done.
remote: Compressing objects: 100% (3542/3542), done.
remote: Total 57363 (delta 2637), reused 8204 (delta 2421), pack-reused 48854
Receiving objects: 100% (57363/57363), 21.17 MiB | 3.16 MiB/s, done.
Resolving deltas: 100% (21096/21096), done.
Updating files: 100% (2584/2584), done.
PS C:\Users\Kuba> cd .\eclipse-workspace\
PS C:\Users\Kuba\eclipse-workspace> cd ..
PS C:\Users\Kuba> cd etl
PS C:\Users\Kuba\etl> docker compose up
[+] Running 33/14
✔ executor 10 layers [⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿] 0B/0B Pulled 24.0s
✔ storage 4 layers [⣿⣿⣿⣿] 0B/0B Pulled 22.2s
✔ frontend 12 layers [⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿] 0B/0B Pulled 63.6s
✔ executor-monitor 3 layers [⣿⣿⣿] 0B/0B Pulled 22.9s
[+] Running 9/9
✔ Network etl_default Created 0.1s
✔ Volume "etl_data-logs" Created 0.1s
✔ Volume "etl_configuration" Created 0.1s
✔ Volume "etl_data-storage" Created 0.0s
✔ Volume "etl_data-execution" Created 0.1s
✔ Container etl-executor-monitor-1 Created 0.9s
✔ Container etl-executor-1 Created 0.9s
✔ Container etl-storage-1 Created 0.9s
✔ Container etl-frontend-1 Created 0.9s
Attaching to executor-1, executor-monitor-1, frontend-1, storage-1
executor-1 | 12:30:27,447 [main ] DEBUG com.linkedpipes.etl.executor.cli.ConfigurationLoader
- Reading configuration from: /opt/lp-etl/configuration/configuration.properties
executor-1 | 12:30:27,480 [main ] WARN com.linkedpipes.etl.executor.cli.ConfigurationLoader
- Properties configuration files are deprecated.
executor-1 | 12:30:27,652 [main ] DEBUG com.linkedpipes.etl.executor.Executor
- Banned components: []
storage-1 | 12:30:27,679 [main ] DEBUG com.linkedpipes.etl.storage.cli.ConfigurationLoader
- Reading configuration from: /opt/lp-etl/configuration/configuration.properties
storage-1 | 12:30:27,684 [main ] WARN com.linkedpipes.etl.storage.cli.ConfigurationLoader
- Properties configuration files are deprecated.
frontend-1 |
frontend-1 | > [email protected] start
frontend-1 | > node server/server.production.js
frontend-1 |
executor-monitor-1 | 12:30:28,097 [main ] DEBUG com.linkedpipes.etl.executor.monitor.cli.ConfigurationLoader - Reading configuration from: /opt/lp-etl/configuration/configuration.properties
executor-monitor-1 | 12:30:28,106 [main ] WARN com.linkedpipes.etl.executor.monitor.cli.ConfigurationLoader - Properties configuration files are deprecated.
frontend-1 | Loading configuration from: /opt/lp-etl/configuration/configuration.properties
frontend-1 | 2024-02-09 12:30:28 [INFO] Starting production server ...
frontend-1 | 2024-02-09 12:30:28 [INFO] Listening on port: 8080
storage-1 | 12:30:29,335 [main ] DEBUG com.linkedpipes.etl.storage.plugin.JavaPluginService
- Loading plugins ...
executor-monitor-1 | 12:30:29,643 [main ] INFO com.linkedpipes.etl.executor.monitor.events.EventListenerAggregator - Initializing event listeners ...
executor-monitor-1 | 12:30:30,805 [main ] INFO com.linkedpipes.etl.executor.monitor.web.WebServer
- Starting web server on port: 8081
executor-1 | 12:30:30,870 [main ] DEBUG com.linkedpipes.etl.executor.plugin.osgi.OsgiPluginService - Loading libraries from '/opt/lp-etl/osgi'.
executor-monitor-1 | 12:30:31,213 [main ] INFO org.eclipse.jetty.server.Server
- jetty-11.0.15; built: 2023-04-11T18:37:53.775Z; git: 5bc5e562c8d05c5862505aebe5cf83a61bdbcb96; jvm 17.0.1+12-39
executor-monitor-1 | 12:30:31,337 [main ] INFO org.eclipse.jetty.server.handler.ContextHandler.ROOT
- Initializing Spring root WebApplicationContext
storage-1 | 12:30:31,584 [main ] INFO com.linkedpipes.etl.storage.plugin.JavaPluginService
- Loading plugins ... done with 97 plugins
storage-1 | 12:30:31,604 [main ] DEBUG com.linkedpipes.etl.storage.template.repository.file.FileTemplateRepository - Loading repository ...
storage-1 | 12:30:31,620 [main ] DEBUG com.linkedpipes.etl.storage.template.repository.file.FileTemplateRepository - Loading repository ... done
storage-1 | 12:30:31,758 [main ] DEBUG com.linkedpipes.etl.storage.template.PluginTemplateService - Initializing plugin service ...
storage-1 | 12:30:31,759 [main ] INFO com.linkedpipes.etl.storage.template.PluginTemplateService - Initializing plugin service ... done with 98 templates
storage-1 | 12:30:31,759 [main ] DEBUG com.linkedpipes.etl.storage.template.ReferenceTemplateService - Initializing reference service ...
storage-1 | 12:30:31,759 [main ] INFO com.linkedpipes.etl.storage.template.ReferenceTemplateService - Initializing reference service ... done with 0 templates
storage-1 | 12:30:31,762 [main ] DEBUG com.linkedpipes.etl.storage.pipeline.repository.file.FilePipelineRepository - Loading repository ...
storage-1 | 12:30:31,763 [main ] DEBUG com.linkedpipes.etl.storage.pipeline.repository.file.FilePipelineRepository - Loading repository ... done (files: 0, loaded: 0, migrated: 0, failed: 0)
storage-1 | 12:30:31,763 [main ] DEBUG com.linkedpipes.etl.storage.pipeline.PipelineService
- Initializing pipeline service ...
storage-1 | 12:30:31,764 [main ] INFO com.linkedpipes.etl.storage.pipeline.PipelineService
- Initializing pipeline service ... done with 0 pipelines
storage-1 | 12:30:31,764 [main ] DEBUG com.linkedpipes.etl.storage.StorageService
- Initialization done in 2s
storage-1 | 12:30:31,912 [main ] INFO com.linkedpipes.etl.storage.http.HttpServer
- Starting HTTP server on port 8083
storage-1 | 12:30:32,399 [main ] INFO org.eclipse.jetty.server.Server
- jetty-11.0.15; built: 2023-04-11T18:37:53.775Z; git: 5bc5e562c8d05c5862505aebe5cf83a61bdbcb96; jvm 17.0.1+12-39
storage-1 | 12:30:32,592 [main ] INFO org.eclipse.jetty.server.handler.ContextHandler.ROOT
- Initializing Spring root WebApplicationContext
executor-monitor-1 | 12:30:32,846 [main ] INFO org.eclipse.jetty.server.handler.ContextHandler.ROOT
- Initializing Spring DispatcherServlet 'org.springframework.web.servlet.DispatcherServlet-3703bf3c'
executor-monitor-1 | 12:30:32,890 [main ] INFO org.eclipse.jetty.server.handler.ContextHandler
- Started o.e.j.s.ServletContextHandler@71978f46{/,jar:file:/opt/lp-etl/executor-monitor/executor-monitor.jar!/web/,AVAILABLE}
executor-monitor-1 | 12:30:32,921 [main ] INFO org.eclipse.jetty.server.AbstractConnector
- Started ServerConnector@30404dba{HTTP/1.1, (http/1.1)}{0.0.0.0:8081}
executor-monitor-1 | 12:30:32,944 [main ] INFO org.eclipse.jetty.server.Server
- Started Server@51ec2df1{STARTING}[11.0.15,sto=0] @6377ms
executor-monitor-1 | 12:30:32,945 [main ] INFO com.linkedpipes.etl.executor.monitor.web.FtpServer
- Starting FTP server on ports: 2221, 2222 - 2225
executor-1 | 12:30:33,038 [main ] DEBUG org.apache.jena.util.FileManager
- Add location: LocatorFile
executor-1 | 12:30:33,039 [main ] DEBUG org.apache.jena.util.FileManager
- Add location: ClassLoaderLocator
executor-1 | 12:30:33,040 [main ] DEBUG org.apache.jena.util.LocationMapper
- Failed to find configuration: file:location-mapping.rdf;file:location-mapping.n3;file:location-mapping.ttl;file:etc/location-mapping.rdf;file:etc/location-mapping.n3;file:etc/location-mapping.ttl
executor-1 | 12:30:33,041 [main ] DEBUG org.apache.jena.util.FileManager
- Add location: LocatorFile
executor-1 | 12:30:33,041 [main ] DEBUG org.apache.jena.util.FileManager
- Add location: LocatorURL
executor-1 | 12:30:33,041 [main ] DEBUG org.apache.jena.util.FileManager
- Add location: ClassLoaderLocator
executor-1 | 12:30:33,054 [main ] DEBUG org.apache.jena.riot.system.stream.JenaIOEnvironment
- Failed to find configuration: location-mapping.ttl;location-mapping.rdf;etc/location-mapping.rdf;etc/location-mapping.ttl
executor-monitor-1 | 12:30:33,098 [main ] DEBUG org.apache.ftpserver.impl.DefaultFtpServerContext
- Intializing shared thread pool executor with max threads of 4
executor-monitor-1 | 12:30:33,151 [main ] INFO org.apache.ftpserver.impl.DefaultFtpServer
- FTP server started
executor-1 | 12:30:33,597 [main ] DEBUG org.apache.jena.info
- System architecture: 64 bit
executor-1 | 12:30:33,633 [main ] DEBUG org.apache.jena.dboe.System
- System architecture: 64 bit
executor-1 | 12:30:33,757 [main ] DEBUG com.linkedpipes.etl.executor.plugin.osgi.OsgiPluginService - Loading plugins from '/opt/lp-etl/components'.
storage-1 | 12:30:33,831 [main ] INFO org.eclipse.jetty.server.handler.ContextHandler.ROOT
- Initializing Spring DispatcherServlet 'org.springframework.web.servlet.DispatcherServlet-73ab3aac'
storage-1 | 12:30:33,859 [main ] INFO org.eclipse.jetty.server.handler.ContextHandler
- Started o.e.j.s.ServletContextHandler@3c98781a{/,jar:file:/opt/lp-etl/storage/storage.jar!/web/,AVAILABLE}
storage-1 | 12:30:33,882 [main ] INFO org.eclipse.jetty.server.AbstractConnector
- Started ServerConnector@29a4f594{HTTP/1.1, (http/1.1)}{0.0.0.0:8083}
storage-1 | 12:30:33,889 [main ] INFO org.eclipse.jetty.server.Server
- Started Server@1e54cb33{STARTING}[11.0.15,sto=0] @7499ms
executor-1 | 12:30:34,409 [main ] WARN com.linkedpipes.etl.executor.plugin.osgi.OsgiClassLoader - Can't load class: com.linkedpipes.plugin.loader.wikibase.WikibaseApiConnection
executor-1 | java.lang.NoClassDefFoundError: org/wikidata/wdtk/wikibaseapi/BasicApiConnection
executor-1 | at java.base/java.lang.ClassLoader.defineClass1(Native Method)
executor-1 | at java.base/java.lang.ClassLoader.defineClass(ClassLoader.java:1012)
executor-1 | at org.eclipse.osgi.internal.loader.ModuleClassLoader.defineClass(ModuleClassLoader.java:276)
executor-1 | at org.eclipse.osgi.internal.loader.classpath.ClasspathManager.defineClass(ClasspathManager.java:632)
executor-1 | at org.eclipse.osgi.internal.loader.classpath.ClasspathManager.findClassImpl(ClasspathManager.java:555)
executor-1 | at org.eclipse.osgi.internal.loader.classpath.ClasspathManager.findLocalClassImpl(ClasspathManager.java:514)
executor-1 | at org.eclipse.osgi.internal.loader.classpath.ClasspathManager.findLocalClass(ClasspathManager.java:501)
executor-1 | at org.eclipse.osgi.internal.loader.ModuleClassLoader.findLocalClass(ModuleClassLoader.java:328)
executor-1 | at org.eclipse.osgi.internal.loader.BundleLoader.findLocalClass(BundleLoader.java:392)
executor-1 | at org.eclipse.osgi.internal.loader.BundleLoader.findClassInternal(BundleLoader.java:470)
executor-1 | at org.eclipse.osgi.internal.loader.BundleLoader.findClass(BundleLoader.java:419)
executor-1 | at org.eclipse.osgi.internal.loader.BundleLoader.findClass(BundleLoader.java:411)
executor-1 | at org.eclipse.osgi.internal.loader.ModuleClassLoader.loadClass(ModuleClassLoader.java:150)
executor-1 | at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:520)
executor-1 | at org.eclipse.osgi.internal.framework.EquinoxBundle.loadClass(EquinoxBundle.java:609)
executor-1 | at com.linkedpipes.etl.executor.plugin.osgi.OsgiClassLoader.tryToLoadAndRegisterClass(OsgiClassLoader.java:84)
executor-1 | at com.linkedpipes.etl.executor.plugin.osgi.OsgiClassLoader.scan(OsgiClassLoader.java:59)
executor-1 | at com.linkedpipes.etl.executor.plugin.osgi.OsgiClassLoader.load(OsgiClassLoader.java:42)
executor-1 | at com.linkedpipes.etl.executor.plugin.osgi.OsgiPluginService.loadPlugin(OsgiPluginService.java:174)
executor-1 | at com.linkedpipes.etl.executor.plugin.osgi.OsgiPluginService.loadPlugins(OsgiPluginService.java:148)
executor-1 | at com.linkedpipes.etl.executor.plugin.PluginServiceHolder.start(PluginServiceHolder.java:133)
executor-1 | at com.linkedpipes.etl.executor.plugin.PluginServiceHolder.onApplicationEvent(PluginServiceHolder.java:119)
executor-1 | at org.springframework.context.event.SimpleApplicationEventMulticaster.doInvokeListener(SimpleApplicationEventMulticaster.java:172)
executor-1 | at org.springframework.context.event.SimpleApplicationEventMulticaster.invokeListener(SimpleApplicationEventMulticaster.java:165)
executor-1 | at org.springframework.context.event.SimpleApplicationEventMulticaster.multicastEvent(SimpleApplicationEventMulticaster.java:143)
executor-1 | at org.springframework.context.support.AbstractApplicationContext.publishEvent(AbstractApplicationContext.java:437)
executor-1 | at org.springframework.context.support.AbstractApplicationContext.publishEvent(AbstractApplicationContext.java:370)
executor-1 | at org.springframework.context.support.AbstractApplicationContext.start(AbstractApplicationContext.java:1443)
executor-1 | at com.linkedpipes.etl.executor.Executor.startSpring(Executor.java:97)
executor-1 | at com.linkedpipes.etl.executor.Executor.run(Executor.java:35)
executor-1 | at com.linkedpipes.etl.executor.Executor.main(Executor.java:23)
executor-1 | Caused by: java.lang.ClassNotFoundException: org.wikidata.wdtk.wikibaseapi.BasicApiConnection cannot be found by com.linkedpipes.l-wikibase_0.0.0
executor-1 | at org.eclipse.osgi.internal.loader.BundleLoader.findClassInternal(BundleLoader.java:508)
executor-1 | at org.eclipse.osgi.internal.loader.BundleLoader.findClass(BundleLoader.java:419)
executor-1 | at org.eclipse.osgi.internal.loader.BundleLoader.findClass(BundleLoader.java:411)
executor-1 | at org.eclipse.osgi.internal.loader.ModuleClassLoader.loadClass(ModuleClassLoader.java:150)
executor-1 | at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:520)
executor-1 | ... 31 common frames omitted
executor-1 | 12:30:37,529 [main ] INFO com.linkedpipes.etl.executor.plugin.osgi.OsgiPluginService - Loaded '98' component bundles from '97' files.
executor-1 | 12:30:37,529 [main ] INFO com.linkedpipes.etl.executor.web.WebServer
- Starting server on port: 8085
executor-1 | 12:30:37,765 [main ] INFO org.eclipse.jetty.server.Server
- jetty-11.0.15; built: 2023-04-11T18:37:53.775Z; git: 5bc5e562c8d05c5862505aebe5cf83a61bdbcb96; jvm 17.0.1+12-39
executor-1 | 12:30:37,886 [main ] INFO org.eclipse.jetty.server.handler.ContextHandler.ROOT
- Initializing Spring root WebApplicationContext
executor-1 | 12:30:38,899 [main ] INFO org.eclipse.jetty.server.handler.ContextHandler.ROOT
- Initializing Spring DispatcherServlet 'org.springframework.web.servlet.DispatcherServlet-5f166f2e'
executor-1 | 12:30:38,916 [main ] INFO org.eclipse.jetty.server.handler.ContextHandler
- Started o.e.j.s.ServletContextHandler@1a7a21d0{/,jar:file:/opt/lp-etl/executor/executor.jar!/web/,AVAILABLE}
executor-1 | 12:30:38,944 [main ] INFO org.eclipse.jetty.server.AbstractConnector
- Started ServerConnector@2430cf17{HTTP/1.1, (http/1.1)}{0.0.0.0:8085}
executor-1 | 12:30:38,967 [main ] INFO org.eclipse.jetty.server.Server
- Started Server@6d6f6ca9{STARTING}[11.0.15,sto=0] @12741ms
Docker version 24.0.6, build ed223bc Model Name: MacBook Pro Model Identifier: MacBookPro14,3 Processor Name: Quad-Core Intel Core i7 Processor Speed: 2,8 GHz Number of Processors: 1 Total Number of Cores: 4 L2 Cache (per Core): 256 KB L3 Cache: 6 MB Hyper-Threading Technology: Enabled Memory: 16 GB System Version: macOS 13.6.3 (22G436) Kernel Version: Darwin 22.6.0
@Pin0 thanks. Unfortunately, we do not have a Mac to test this on, if it turns out to be a Mac-specific issue.
@Pin0 Can you execute following commands and share the output?
# In the LP:ETL directory, start the storage container.
docker compose run -it storage bash
# In the container, check content and existence of the data directory.
ls -la /data/lp-etl/storage
# Print content of the file
cat /data/lp-etl/storage/repository-info.json
# Exit container
exit