alluxio icon indicating copy to clipboard operation
alluxio copied to clipboard

Active Syncing is not supported on this UFS type: hdfs

Open Crazybean-lwb opened this issue 2 years ago • 5 comments

Alluxio Version: Alluxio version: v2.8.0

Other tools Version: k8s version: v1.20.11 docker version: 20.10.11 hdfs version: 3.3.2

Some background to add hdfs was deployed by binary file hdfs was started by user root:supergroup alluxio was deployed in k8s by helm run user root:root

Describe the bug [bug1] I can't sync metadata and raw data from hdfs to alluxio. output exception: Active Syncing is not supported on this UFS type: hdfs [bug2] I can't write data in ReadWriteMany PVC created by alluxio csi output xxx.txt" E514: write error (file system full?) To Reproduce

  1. start sync alluxio dir in alluxio master pod image
  2. error log in alluxio master image

  1. I can touch new file in alluxio pvc, but can not write data in when write something into touched new file, error occured: image or can not write into swap file
  2. error log in alluxio master image

Problem

  1. I found out hdfs version 3.3.2 have enabled support for ACL, can not understand Active Syncing is not supported on this UFS type: hdfs
  2. I do not understand the procession of write data, .swap file id needed, but alluxio can not sync it.

Crazybean-lwb avatar Jul 28 '22 09:07 Crazybean-lwb

Make sure the parent path of your startSync path /liuweibin in the mount table specifies the hdfs version. you can use "alluxio fs mount --option alluxio.underfs.version=xxx"

yohengyang avatar Aug 01 '22 13:08 yohengyang

Make sure the parent path of your startSync path /liuweibin in the mount table specifies the hdfs version. you can use "alluxio fs mount --option alluxio.underfs.version=xxx"

I have tried, output as follows: image

Crazybean-lwb avatar Aug 09 '22 02:08 Crazybean-lwb

Make sure the parent path of your startSync path /liuweibin in the mount table specifies the hdfs version. you can use "alluxio fs mount --option alluxio.underfs.version=xxx"

I have tried many times mount hdfs with under.version, no differences happend.

Crazybean-lwb avatar Aug 11 '22 08:08 Crazybean-lwb

Make sure the parent path of your startSync path /liuweibin in the mount table specifies the hdfs version. you can use "alluxio fs mount --option alluxio.underfs.version=xxx"

I have tried many times mount hdfs with under.version, no differences happend.


I found sollution to [1. start sync alluxio dir in alluxio master pod] alluxio latest version surpport hdfs version 3.3.1 by default, while my hdfs version is 3.3.2. I build alluxio-underfs-hdfs jar following the document, I can resolve the problem origin_img_v2_0ed46b3f-91af-4454-a30a-f88d520bb9bg

bug2 still can not be resolved. I still can not write data in ReadWriteMany PVC created by alluxio csi output xxx.txt" E514: write error (file system full?)

Crazybean-lwb avatar Aug 29 '22 06:08 Crazybean-lwb

Make sure the parent path of your startSync path /liuweibin in the mount table specifies the hdfs version. you can use "alluxio fs mount --option alluxio.underfs.version=xxx"

I have tried many times mount hdfs with under.version, no differences happend.

I found sollution to [1. start sync alluxio dir in alluxio master pod]

alluxio latest version surpport hdfs version 3.3.1 by default, while my hdfs version is 3.3.2. I build alluxio-underfs-hdfs jar following the document, I can resolve the problem origin_img_v2_0ed46b3f-91af-4454-a30a-f88d520bb9bg bug2 still can not be resolved. I still can not write data in ReadWriteMany PVC created by alluxio csi output xxx.txt" E514: write error (file system full?)

get some reply for bugs 2 image

Crazybean-lwb avatar Aug 31 '22 03:08 Crazybean-lwb

This issue has been automatically marked as stale because it has not had recent activity. It will be closed in two weeks if no further activity occurs. Thank you for your contributions.

github-actions[bot] avatar Feb 01 '23 15:02 github-actions[bot]