hudi icon indicating copy to clipboard operation
hudi copied to clipboard

[SUPPORT] Hudi cli got empty result for command show fsview all

Open paul8263 opened this issue 2 years ago • 2 comments

Describe the problem you faced

Hudi cli got empty result after running command show fsview all.

image

The type of table t1 is COW and I am sure that the parquet file is actually generated inside data folder. Also, the parquet files are not damaged as the data could be retrieved correctly by reading as Hudi table or directly reading each parquet file(using Spark).

To Reproduce

Steps to reproduce the behavior:

  1. Enter Flink SQL client.
  2. Execute the SQL and check the data was written successfully.
CREATE TABLE t1(
  uuid VARCHAR(20),
  name VARCHAR(10),
  age INT,
  ts TIMESTAMP(3),
  `partition` VARCHAR(20)
)
PARTITIONED BY (`partition`)
WITH (
  'connector' = 'hudi',
  'path' = 'hdfs:///path/to/table/',
  'table.type' = 'COPY_ON_WRITE'
);

-- insert data using values
INSERT INTO t1 VALUES
  ('id1','Danny',23,TIMESTAMP '1970-01-01 00:00:01','par1'),
  ('id2','Stephen',33,TIMESTAMP '1970-01-01 00:00:02','par1'),
  ('id3','Julian',53,TIMESTAMP '1970-01-01 00:00:03','par2'),
  ('id4','Fabian',31,TIMESTAMP '1970-01-01 00:00:04','par2'),
  ('id5','Sophia',18,TIMESTAMP '1970-01-01 00:00:05','par3'),
  ('id6','Emma',20,TIMESTAMP '1970-01-01 00:00:06','par3'),
  ('id7','Bob',44,TIMESTAMP '1970-01-01 00:00:07','par4'),
  ('id8','Han',56,TIMESTAMP '1970-01-01 00:00:08','par4');
  1. Enter Hudi cli and execute show fsview all

Expected behavior

show fsview all in Hudi cli should return all file slices.

Environment Description

  • Hudi version : 0.11.1

  • Spark version : 3.1.1

  • Hive version : 3.1.0

  • Hadoop version : 3.1.1

  • Storage (HDFS/S3/GCS..) : HDFS

  • Running on Docker? (yes/no) : no

Additional context

No.

Stacktrace

N/A

paul8263 avatar Jul 22 '22 02:07 paul8263

Hi @xushiyan , Finally I got where the problem locates. Could you please assign this issue to me? Thanks.

paul8263 avatar Jul 27 '22 02:07 paul8263

cool. I have created a jira. Let me know your apache id. I can assign the jira to you. and we can close out the github issuue.

nsivabalan avatar Aug 10 '22 03:08 nsivabalan

@paul8263 : gentle ping.

nsivabalan avatar Aug 16 '22 07:08 nsivabalan

@paul8263 : another ping.

nsivabalan avatar Aug 16 '22 13:08 nsivabalan

Hi @nsivabalan , Sorry for my late response. I have created the issue in jira HUDI-4485.

paul8263 avatar Aug 17 '22 00:08 paul8263