[Java] Unable to update VectorSchemaRoot field vectors.

[hnasrullakhan]

Describe the usage question you have. Please include as many useful details as possible.

Hi community,

I have a trying to modify data buffers of string column. if i print the buffer readable bytes after buffers.set(index, compressedBuf);, i see the right value.

 List<ArrowBuf> buffers = fieldVector.getFieldBuffers();
                    int index = buffers.size() - 1;
                    ArrowBuf compressedBuf = null;
                    ArrowBuf originalBuf = buffers.get(index);
                    ByteBuffer originalBuffer = originalBuf.nioBuffer(0, (int) originalBuf.readableBytes());

                    // Compress the buffer using LZ4
                    byte[] originalBytes = new byte[originalBuffer.remaining()];
                    System.out.println("This is buffer id: "+ index + " originalbytes: "+ originalBuf.readableBytes());
                    originalBuffer.get(originalBytes);

                    int maxCompressedLength = compressor.maxCompressedLength(originalBytes.length);
                    byte[] compressedBytes = new byte[maxCompressedLength];
                    int compressedLength = compressor.compress(originalBytes, 0, originalBytes.length, compressedBytes, 0, maxCompressedLength);

                    // Allocate a new ArrowBuf for the compressed data
                    compressedBuf = allocator.buffer(compressedLength);

                    compressedBuf.writeBytes(compressedBytes, 0, compressedLength);
                    compressedBuf.writerIndex(compressedLength);
                    System.out.println("This is buffer id: "+ index+ " compressedbytes: "+ compressedBuf.readableBytes());

                    buffers.set(index, compressedBuf);  // Replace with compressed buffer

When i update my fieldvector.

int nullCount = fieldVector.getNullCount();  // Get the null count
                    ArrowFieldNode fieldNode = new ArrowFieldNode(fieldVectorValueCount, nullCount);
                    ArrowBuf validityBuffer = fieldVector.getValidityBuffer(); // Assuming it exists
                    ArrowBuf offsetBuffer = fieldVector.getOffsetBuffer(); // Assuming it exists

                    // Update FieldVector with the new compressed buffers
                    fieldVector.clear();  // Clear existing buffers (to prevent memory leaks)
                    fieldVector.loadFieldBuffers(fieldNode,java.util.Arrays.asList(validityBuffer, offsetBuffer, compressedBuf));  // Load the new (compressed) buffers
                    fieldVector.setValueCount(fieldVectorValueCount);  // Reset value count after buffer replacement
                    Field field1 = fieldVector.getField();

and i check the buffers again then buffers reflect old value.

please advise .

Component(s)

Java