seatunnel icon indicating copy to clipboard operation
seatunnel copied to clipboard

[Bug] [Source] Temporary files are not ignored

Open xiaofan2022 opened this issue 2 years ago • 1 comments

Search before asking

  • [X] I had searched in the issues and found no similar issues.

What happened

ignored Temporary files

SeaTunnel Version

2.3.1

SeaTunnel Config

env {
  # You can set spark configuration here
  # see available properties defined by spark: https://spark.apache.org/docs/latest/configuration.html#available-properties
  #job.mode = BATCH
  job.name = "SeaTunnel"
  spark.executor.instances = 1
  spark.executor.cores = 1
  spark.executor.memory = "1g"
  spark.master = local
}
source {
  Hive {
    table_name = "test.test_count"
    metastore_uri = "thrift://master1:9083"
    hdfs_site_path = "/data/conf/cdh/hadoop/hdfs-site.xml"
    read_partitions = ["dt=2023-01-01"]
    result_table_name = "hive_table"
  }
}
transform {
  #Sql {
  #  source_table_name = "hive_table"
   # result_table_name = "trans_table"
   # query = "select * from hive_table"
  #}
}

sink {
  console{
  }
}

Running Command

seatunnel.sh --config test.conf --deploy-mode local

Error Exception

.hive-staging_hive*  Contains data, will dirty read。

Flink or Spark Version

No response

Java or Scala Version

No response

Screenshots

Snipaste_2023-06-02_10-30-36

Are you willing to submit PR?

  • [ ] Yes I am willing to submit a PR!

Code of Conduct

xiaofan2022 avatar Jun 02 '23 02:06 xiaofan2022

Check whether there is a problem PTAL: @EricJoy2048 ,I'll fix it if I can

zhilinli123 avatar Jun 02 '23 03:06 zhilinli123

This issue has been automatically marked as stale because it has not had recent activity for 30 days. It will be closed in next 7 days if no further activity occurs.

github-actions[bot] avatar Jul 03 '23 00:07 github-actions[bot]

This issue has been closed because it has not received response for too long time. You could reopen it if you encountered similar problems in the future.

github-actions[bot] avatar Jul 10 '23 00:07 github-actions[bot]