hudi
hudi copied to clipboard
apache
Upserts, Deletes And Incremental Processing on Big Data.
Running Deltastreamer with Cloudwatch Metrics isn't shutting down properly. This is in NON continous mode. DeltaSync and spark context say they are closing, but the JVM is not exiting, everything...
**Summary** During clustering, Hudi creates duplicate parquet file with the same file group ID and identical content. One of the two files are later marked as a duplicate and deleted....
### Describe the problem I'm using a Spark job running on EMR to insert data using hudi (0.9.0). The inserts are working as expected and it stores parquet files in...
Hudi version: 0.11.1 Spark version: 3.1.1 Storage: S3 AWS Glue: 3 Function ```scala import org.apache.spark.sql.{functions => fn} def readAndShow(path: String) { val df = spark.read.format("hudi").load(path) df.select(fn.min(fn.col("updated_at")), fn.min(fn.col("_hoodie_commit_time"))) show false val...
**Describe the problem you faced** Hudi cli got empty result after running command show fsview all.  The type of table t1 is COW and I am sure that the...
**Describe the problem you faced** A clear and concise description of the problem. Upgrading to 0.11.1 , the deltastreamer is failing to write to a 6GB bucket. It is failing...
**_Tips before filing an issue_** - Have you gone through our [FAQs](https://hudi.apache.org/learn/faq/)? - Join the mailing list to engage in conversations and get faster support at [email protected]. - If you...
**Describe the problem you faced** Our Hudi data lake is heavily partitioned by datasource, year, and month. We have 1000 datasources currently loaded into the lake, and are looking to...
**Describe the problem you faced** I'm using Hudi Delta streamer in continuous mode with Kafka source. Whenever Kafka offset got expired the job will fail with offset out of range...
**_Tips before filing an issue_** - Have you gone through our [FAQs](https://hudi.apache.org/learn/faq/)? - Join the mailing list to engage in conversations and get faster support at [email protected]. - If you...