pulsar-io-cloud-storage

pulsar-io-cloud-storage copied to clipboard

**Is your feature request related to a problem? Please describe.** Now that the new partitioner doesn't add a datetime based prefix to the file it would be useful if we...

asher-goldberg

**Describe the bug** ``` │ pulsar-sink java.lang.NullPointerException: Cannot read the array length because "array" is null │ │ pulsar-sink at java.nio.ByteBuffer.wrap(ByteBuffer.java:437) ~[?:?] │ │ pulsar-sink at org.apache.pulsar.io.jcloud.util.MetadataUtil.parseSchemaVersionFromBytes(MetadataUtil.java:169) ~[vFdZUZqkBvBdIxBFOEzjGw/:?] │ │...

RobertIndie

**Is your feature request related to a problem? Please describe.** > Would it be possible to add another configuration element that includes the original topic in the metadata when withMetadata...

nlu90

**Describe the bug** Similar to #493, currently the `maxTimeoutMs` is applied for all topics at once, instead of having an independent timer for each topic. **Expected behavior** Each topic should...

alpreu

**Describe the bug** The current sink implementation uses one `batchSize` summing the number of records of all topics together. This results in fluctuating real 'batch' sizes for each topic. **Expected...

alpreu

[BUG] Time Partitioner not working as expected for GCS

1

**Data Sent to GCS Bucket is not in the correct date folder.** I have configured `timePartitionDuration: 1d` It was working fine but after a few days dates folder was not...

shaileshjha061

[BUG] error while writing data in Parquet format using JsonSchema as schema format

3

**Describe the bug** I was using the Cloud Storage Sink to collect data from Pulsar and write it to AWS S3 in **Parquet**. Messages were produced using a `JsonSchema` format....

grandimk

GCS Sink Connector not able to write as parquet when published data using golang client for pulsar.. works fine with java client

2

@freeznet when I am writing data using pulsar golang client .. the below is the program I am using to write on pulsar topic and its able to publish on...

mdsadimnadeem

Bumps org.apache.hadoop:hadoop-common from 3.3.6 to 3.4.0. [](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a...

dependabot[bot]

Add batch modle

1

Draft

shibd