alibabacloud-jindodata icon indicating copy to clipboard operation
alibabacloud-jindodata copied to clipboard

利用jindofs-sdk包初始化FileSystem对象实例失败

Open baisui1981 opened this issue 2 years ago • 3 comments
trafficstars

按照 https://github.com/aliyun/alibabacloud-jindodata/blob/master/docs/user/3.x/hadoop/jindosdk_on_hadoop.md 文档所示,

  1. 下载jindofs-sdk,并且加入classpath中
  2. 在阿里云OSS中创建了基于OSS HDFS bucket,
  3. 本地依赖hadoop:3.2.1 版本相关的依赖包代码

希望能够初始化org.apache.hadoop.fs.FileSystem 实例对象。

在执行过程中报告以下错误,不知道是哪儿出错了,本机是MacOS机器

Caused by: java.lang.RuntimeException: JbootNative loading failed: java.lang.UnsatisfiedLinkError: /private/tmp/libjboot-3397ed5b1f5602cb_20211216_104547.dylib: dlopen(/private/tmp/libjboot-3397ed5b1f5602cb_20211216_104547.dylib, 1): no suitable image found.  Did find:
	/private/tmp/libjboot-3397ed5b1f5602cb_20211216_104547.dylib: cannot load 'libjboot-3397ed5b1f5602cb_20211216_104547.dylib' (load command 0x80000034 is unknown)
	/private/tmp/libjboot-3397ed5b1f5602cb_20211216_104547.dylib: cannot load 'libjboot-3397ed5b1f5602cb_20211216_104547.dylib' (load command 0x80000034 is unknown)
	at java.lang.ClassLoader$NativeLibrary.load(Native Method)
	at java.lang.ClassLoader.loadLibrary0(ClassLoader.java:1941)
	at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1824)
	at java.lang.Runtime.load0(Runtime.java:809)
	at java.lang.System.load(System.java:1086)
	at com.alibaba.jboot.NativeCodeLoader.loadLibrary(NativeCodeLoader.java:125)
	at com.alibaba.jboot.NativeCodeLoader.loadLibrary(NativeCodeLoader.java:82)
	at com.alibaba.jboot.NativeCodeLoader.load(NativeCodeLoader.java:44)
	at com.alibaba.jboot.JbootNative.<clinit>(JbootNative.java:19)
	at com.alibaba.jboot.JbootCache.<init>(JbootCache.java:11)
	at com.aliyun.emr.fs.internal.ossnative.OssFileletSystem.<init>(OssFileletSystem.java:36)
	at com.aliyun.emr.fs.internal.ossnative.OssNativeStore$Context.<init>(OssNativeStore.java:99)
	at com.aliyun.emr.fs.oss.JindoOssFileSystem.initNativeStoreContext(JindoOssFileSystem.java:140)
	at com.aliyun.emr.fs.oss.JindoOssFileSystem.initialize(JindoOssFileSystem.java:135)
	at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:2669)
	at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:94)
	at org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:2703)
	at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:2685)
	at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:373)
	at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:172)
	at com.qlangtech.tis.hdfs.impl.HdfsFileSystemFactory$HdfsUtils.getFileSystem(HdfsFileSystemFactory.java:320)
	at com.qlangtech.tis.hdfs.impl.HdfsFileSystemFactory$1.create(HdfsFileSystemFactory.java:173)
	at com.qlangtech.tis.hdfs.impl.HdfsFileSystemFactory$1.visit(HdfsFileSystemFactory.java:149)
	at com.qlangtech.tis.hdfs.impl.HdfsFileSystemFactory$1.visit(HdfsFileSystemFactory.java:145)
	at com.qlangtech.tis.config.authtoken.impl.OffUserToken.accept(OffUserToken.java:37)
	at com.qlangtech.tis.hdfs.impl.HdfsFileSystemFactory.createFS(HdfsFileSystemFactory.java:145)
	at com.qlangtech.tis.hdfs.impl.HdfsFileSystemFactory.getFileSystem(HdfsFileSystemFactory.java:113)
	at com.qlangtech.tis.hdfs.impl.TestAliayunJindoFSFactory.testListChild(TestAliayunJindoFSFactory.java:77)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59)
	at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
	at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56)
	at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
	at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306)
	at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100)
	at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366)
	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103)
	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63)
	at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331)
	at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79)
	at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329)
	at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66)
	at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293)
	at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306)
	at org.junit.runners.ParentRunner.run(ParentRunner.java:413)
	at org.junit.runner.JUnitCore.run(JUnitCore.java:137)
	at com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:68)
	at com.intellij.rt.execution.junit.IdeaTestRunner$Repeater.startRunnerWithArgs(IdeaTestRunner.java:47)
	at com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:242)
	at com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:70)

	at com.alibaba.jboot.JbootNative.checkLoaded(JbootNative.java:25)
	at com.alibaba.jboot.JbootNative.instance(JbootNative.java:30)
	at com.alibaba.jboot.JbootNative.initOnceNativeClass(JbootNative.java:74)
	at com.alibaba.jboot.JbootCache.<init>(JbootCache.java:11)
	at com.aliyun.emr.fs.internal.ossnative.OssFileletSystem.<init>(OssFileletSystem.java:36)
	at com.aliyun.emr.fs.internal.ossnative.OssNativeStore$Context.<init>(OssNativeStore.java:99)
	at com.aliyun.emr.fs.oss.JindoOssFileSystem.initNativeStoreContext(JindoOssFileSystem.java:140)
	at com.aliyun.emr.fs.oss.JindoOssFileSystem.initialize(JindoOssFileSystem.java:135)
	at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:2669)
	at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:94)
	at org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:2703)
	at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:2685)
	at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:373)
	at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:172)
	at com.qlangtech.tis.hdfs.impl.HdfsFileSystemFactory$HdfsUtils.getFileSystem(HdfsFileSystemFactory.java:320)
	... 32 more

baisui1981 avatar Jul 27 '23 14:07 baisui1981

@baisui1981 你是不是 mac m1/m2 的机型?是的话,可能得用jindosdk-4.6.4-linux-el7-aarch64.tar.gz

yanbinyang avatar Jul 27 '23 15:07 yanbinyang

@baisui1981 你是不是 mac m1/m2 的机型?是的话,可能得用jindosdk-4.6.4-linux-el7-aarch64.tar.gz

谢谢如此快速的回复!我现在用的 jindofs-sdk包是否只能在centOS服务器上使用?我现在就按照你说的,试试

baisui1981 avatar Jul 27 '23 15:07 baisui1981

@baisui1981 你是不是 mac m1/m2 的机型?是的话,可能得用jindosdk-4.6.4-linux-el7-aarch64.tar.gz

谢谢如此快速的回复!我现在用的 jindofs-sdk包是否只能在centOS服务器上使用?我现在就按照你说的,试试

不是,我们支持多平台使用的,4.x所有版本,基本上都支持macos x86、大部分linux发行版。 如果是macos arm版本,由于使用的人不多,目前只有4.6.4版本做了发布,以供demo使用。 下载链接参见jindodata_download.md。 安装说明参见install_dependeny_jindosdk.md

另外,你看的文档是我们的老版本3.x文档。目前最新4.x版本的文档参见jindosdk_ide_hadoop.md

yanbinyang avatar Jul 27 '23 15:07 yanbinyang