[BUG] Time Partitioner not working as expected for GCS

Open shaileshjha061 opened this issue 3 years ago • 1 comments

Data Sent to GCS Bucket is not in the correct date folder. I have configured timePartitionDuration: 1d It was working fine but after a few days dates folder was not being created (say after 2022-06-23 folders had not been created) and the data was being sent to the 2022-06-23 folder. Data being sent to GCS.

To Reproduce I am using Function Mesh version: 0.1.11-rc3 Pulsar IO Cloud storage sink connector Version: 2.9.2.21

Expected behavior Data Sent to GCS Bucket should be in the respective data folder.

Screenshots Screenshot 2022-06-28 204511

Additional context Add any other context about the problem here.

Jun 28 '22 15:06 shaileshjha061

Hi @shaileshjha061, can you confirm whether your records Message objects have a publishTime associated?

Oct 11 '22 15:10 alpreu