paimon icon indicating copy to clipboard operation
paimon copied to clipboard

fix: The increment query result is not in the correct order

Open Pandas886 opened this issue 7 months ago • 0 comments

Purpose

Linked issue: close #5709

fix branch : paimon-1.2-snapshot

Tests

  spark.sql(
      """
        |create table my_table (
        |    k int,
        |    v string
        |) tblproperties (
        |    'primary-key' = 'k','changelog-producer' = 'lookup','bucket'='4'
        |);
        |""".stripMargin)
 
    spark.sql(" insert into my_table values (1, 'a'), (2, 'b')")
 
    spark.sql("delete from my_table where k = 1")
 
    spark.sql("insert into my_table values (11, 'a'), (2, 'bb')")

    spark.sql(
      """
        |SELECT * FROM paimon_incremental_query('`my_table$audit_log`', '0', '6')
        |""".stripMargin).show()

Before Fix , the result is not in correct order.

+-------+---+---+
|rowkind|  k|  v|
+-------+---+---+
|     -D|  1|  a|
|     +I|  1|  a|
|     -U|  2|  b|
|     +U|  2| bb|
|     +I|  2|  b|
|     +I| 11|  a|
+-------+---+---+

After the modification, the order of the #5709 results is correct now.

+-------+---+---+
|rowkind|  k|  v|
+-------+---+---+
|     +I|  1|  a|
|     -D|  1|  a|
|     +I|  2|  b|
|     -U|  2|  b|
|     +U|  2| bb|
|     +I| 11|  a|
+-------+---+---+

API and Format

Documentation

Pandas886 avatar Jun 08 '25 01:06 Pandas886