kazdy comments

Results 8 comments of


                                            kazdy

[SUPPORT] ALL PARQUET FILES FROM BASE PATH GOT DELETED BY CLEANER

As discussed in slack, OCC was not enabled. Let's try with OCC enabled since these two jobs run in parallel.

Align policy naming with serverless framework convention?

Hi, yes I think this can break stuff if someone adjusted their deployment role for how this plugin builds policy names. Would it be ok to introduce a feature flag...

[SUPPORT] Clustering created files with modified schema resulting in corrupted table

I inspected all files that were source files for clustering and all have the same schema. The file created as result of clustering has changed schema, column named "year" was...

[SUPPORT] Clustering created files with modified schema resulting in corrupted table

@nsivabalan I would really appreciate any support or guidance here, I have two broken tables in prod now and need to come up with a fix or workaround. Thanks!

[SUPPORT] Clustering created files with modified schema resulting in corrupted table

EDIT: I'm (now) aware there's a regression with async clustering and MDT, I was not earlier, but now it's disabled. So i guess I'm fine there and should not miss...

[SUPPORT] Clustering created files with modified schema resulting in corrupted table

Hi @nsivabalan I see that Danny assigned you to this ticket. I was able to replicate this exact case. Previous one was not exactly exposing the issue. ~I'll update the...

[SUPPORT] Clustering created files with modified schema resulting in corrupted table

@mansipp I was able to replicate this case using Hudi 0.13 as well. Files created by clustering have changed schema. Interestingly there's no issue if eg the order of types...

[SUPPORT] Clustering created files with modified schema resulting in corrupted table

It was 0.12.2 not 0.13, sorry for the confusion. The issue does not occur with 0.13