paimon icon indicating copy to clipboard operation
paimon copied to clipboard

[Bug] ArrowFormatCWriter will raise java.lang.IllegalArgumentException while writing arrow format with empty schema

Open jichen20210919 opened this issue 2 months ago • 0 comments

Search before asking

  • [x] I searched in the issues and found nothing similar.

Paimon version

When we write rows to arrow format with empty paimon schema,following exception will raise java.lang.IllegalArgumentException at org.apache.arrow.util.Preconditions.checkArgument(Preconditions.java:122) at org.apache.arrow.vector.VectorSchemaRoot.getVector(VectorSchemaRoot.java:185) at org.apache.paimon.arrow.ArrowUtils.serializeToCStruct(ArrowUtils.java:224) at org.apache.paimon.arrow.vector.ArrowFormatCWriter.toCStruct(ArrowFormatCWriter.java:72) at org.apache.paimon.arrow.vector.ArrowFormatWriterTest.testArrowFormatCWriterWithEmptySchema(ArrowFormatWriterTest.java:292) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

Compute Engine

JavaAPI

Minimal reproduce step

    try (RootAllocator rootAllocator = new RootAllocator();
            BufferAllocator allocator =
                    rootAllocator.newChildAllocator("paimonWriter", 0, Long.MAX_VALUE);
            ArrowFormatCWriter writer =
                    new ArrowFormatCWriter(emptyschema, 4096, true, allocator)) {
        for (int i = 0; i < 100; i++) {
            writer.write(GenericRow.of());
        }
        writer.flush();
        ArrowCStruct cStruct = writer.toCStruct();
        Assertions.assertThat(cStruct).isNotNull();
        writer.release();
    }

What doesn't meet your expectations?

It should work as normal.

Anything else?

No response

Are you willing to submit a PR?

  • [x] I'm willing to submit a PR!

jichen20210919 avatar Oct 22 '25 10:10 jichen20210919