databend icon indicating copy to clipboard operation
databend copied to clipboard

feat: basic support of schema evolution in copy for parquet

Open youngsofun opened this issue 2 weeks ago โ€ข 0 comments

I hereby agree to the terms of the CLA available at: https://docs.databend.com/dev/policies/cla/

Summary

feats:

  1. table alter and data append is committed together atomicly.

example:

# prepare data
create or replace table t1(a int);
create or replace stage s1;
copy into @s1 from (select 1 as a, 2 as b,  3 as c)

# ENABLE_SCHEMA_EVOLUTION
ALTER TABLE t1 SET options(ENABLE_SCHEMA_EVOLUTION = true);

# do copy
copy into t1 from @s1/ file_format=(type=parquet);

not supported yet:

  • adding field in tuple type
  • merge types, e.g. int8 and int16

Tests

  • [ ] Unit Test
  • [x] Logic Test
  • [ ] Benchmark Test
  • [ ] No Test - Explain why

Type of change

  • [] Bug Fix (non-breaking change which fixes an issue)
  • [x] New Feature (non-breaking change which adds functionality)
  • [ ] Breaking Change (fix or feature that could cause existing functionality not to work as expected)
  • [ ] Documentation Update
  • [ ] Refactoring
  • [ ] Performance Improvement
  • [ ] Other (please describe):

This change isโ€‚Reviewable

youngsofun avatar Dec 11 '25 02:12 youngsofun