hudi icon indicating copy to clipboard operation
hudi copied to clipboard

[HUDI-4894] Fix ClassCastException when using fixed type defining dec…

Open wangxianghu opened this issue 3 years ago • 2 comments

when use fixed type to define decimal column, hudi will throw a ClassCastException

Caused by: java.lang.ClassCastException: java.lang.Double cannot be cast to java.util.List
at org.apache.hudi.avro.MercifulJsonConverter$9.convert(MercifulJsonConverter.java:254)
at org.apache.hudi.avro.MercifulJsonConverter$JsonToAvroFieldProcessor.convertToAvro(MercifulJsonConverter.java:151)
at org.apache.hudi.avro.MercifulJsonConverter.convertJsonToAvroField(MercifulJsonConverter.java:140)
at org.apache.hudi.avro.MercifulJsonConverter.convertJsonToAvro(MercifulJsonConverter.java:107)
at org.apache.hudi.avro.MercifulJsonConverter.convert(MercifulJsonConverter.java:96)
at org.apache.hudi.utilities.sources.helpers.AvroConvertor.fromJs

schema for decimal column

{
    "name": "column_name",
    "type": ["null", {
        "type": "fixed",
        "name": "fixed",
        "size": 5,
        "logicalType": "decimal",
        "precision": 10,
        "scale": 2
    }],
    "default": null
}

Change Logs

Describe context and summary for this change. Highlight if any code was copied.

Impact

Describe any public API or user-facing feature change or any performance impact.

Risk level: none | low | medium | high

Choose one. If medium or high, explain what verification was done to mitigate the risks.

Contributor's checklist

  • [ ] Read through contributor's guide
  • [ ] Change Logs and Impact were stated clearly
  • [ ] Adequate tests were added if applicable
  • [ ] CI passed

wangxianghu avatar Sep 22 '22 01:09 wangxianghu

@hudi-bot run azure

wangxianghu avatar Sep 23 '22 01:09 wangxianghu

cancelling all azure CI runs for now to investigate CI flakiness. will retrigger build once we are in stable state. sorry about the inconvenience.

nsivabalan avatar Sep 23 '22 16:09 nsivabalan

@shenh062326 can you also take a look at this ?

wangxianghu avatar Sep 28 '22 02:09 wangxianghu

CI report:

  • a41207c6914f7e5778658abc361f4c8cb50be13a Azure: FAILURE
Bot commands @hudi-bot supports the following commands:
  • @hudi-bot run azure re-run the last Azure build

hudi-bot avatar Oct 04 '22 05:10 hudi-bot

@xushiyan : PR is assigned to you. Will you be following up?

nsivabalan avatar Nov 02 '22 00:11 nsivabalan