[Bug] Inconsistent parameter key if use dataframe write

[Askwang]

Search before asking

• [X] I searched in the issues and found nothing similar.

Paimon version

0.8

Compute Engine

spark 3.2

Minimal reproduce step

1. Can get corresponding value 'testid-100'.

        spark.sql("""
                  |CREATE TABLE tb (id int, dt string) using paimon
                  |TBLPROPERTIES ('file.format'='parquet', 'primary-key'='id', 'bucket'='1',
                  | 'commit.callback.org.apache.paimon.spark.CustomCommitCallback.param' = 'testid-100')
                  |""".stripMargin)

2. The value is null. Use spark dataframe writes data into paimon, Spark will convert the uppercase key to lowercase. If the key contains uppercase key, we will not get the corresponding value through the default key.

     spark.sql("""
                  |CREATE TABLE tb (id int, dt string) using paimon
                  |TBLPROPERTIES ('file.format'='parquet', 'primary-key'='id', 'bucket'='1')
                  |""".stripMargin)

     val _spark = spark
      import _spark.implicits._
      val df = Seq((1, "a"), (2, "b")).toDF("a", "b")
      df.write
        .format("paimon")
        .option(CoreOptions.COMMIT_CALLBACKS.key(), classOf[CustomCommitCallback].getName)
        .option(
          CoreOptions.COMMIT_CALLBACK_PARAM
            .key()
            .replace("#", classOf[CustomCommitCallback].getName),
          "testid-100")
        .mode("append")
        .save(location)

What doesn't meet your expectations?

Incorrect constructor call.

Anything else?

No response

Are you willing to submit a PR?

• [X] I'm willing to submit a PR!