databend icon indicating copy to clipboard operation
databend copied to clipboard

[feat] replace into table on (x) select * from @stage/file does not support NDJSON format

Open hantmac opened this issue 1 year ago • 2 comments

Summary

reproduce step

  1. create stage and generate ndjson file
CREATE STAGE sjhstage;

copy into @sjhstage FROM 
(
    SELECT * 
    FROM 
    (
        VALUES 
        (1, 'Chengdu', 80),
        (3, 'Chongqing', 90),
        (6, 'Hangzhou', 92),
        (9, 'Hong Kong', 88)
    )
)
FILE_FORMAT = (TYPE = NDJSON);
  1. create target table
CREATE TABLE sample
(
    id      INT,
    city    VARCHAR,
    score   INT,
    country VARCHAR DEFAULT 'China'
);
  1. replace into target table
REPLACE INTO sample
    (id, city, score) 
ON
    (Id)
SELECT * from @sjhstage/data_4e264479-077b-44ff-ae68-cec7a974e882_0000_00000000.ndjson;

or

REPLACE INTO sample
    (id, city, score) 
ON
    (Id)
SELECT * from @sjhstage (FILE_FORMAT => 'ndjson');

We will get the following error:

Query ID: 46d42da4-193d-43ce-9c5e-a0681ad2020a
1065=>
select * from file only support Parquet format

image

How about supporting ndjson and csv file in relace into x select * from stage file ?

hantmac avatar Jun 11 '24 06:06 hantmac

Query stage with json not support select * , Please read: https://docs.databend.com/guides/load-data/transform/querying-ndjson

wubx avatar Jun 11 '24 23:06 wubx

@hantmac this issue in fact not a bug, but a feature request 😄

youngsofun avatar Jun 13 '24 03:06 youngsofun