wal2json icon indicating copy to clipboard operation
wal2json copied to clipboard

LSN not picking from "withStartPosition" in format version-2

Open anilgol opened this issue 1 year ago • 2 comments

we are working with the wal2json format version 2 on Amazon RDS for PostgreSQL 11.19 we are using withStartPosition(LogSequenceNumber.valueOf(lsn))method to get the last LSN value. The LSN value here comes from the confirmed_flush_lsn obtained from the database. Ideally, we expect the next event to start from this position, but currently, it begins from the beginning.

Our specific requirement is to establish a connection to the database from AWS Lambda. Since Lambda requires a restart every 15 minutes, we need to capture records from the last LSN where the Lambda previously stopped. To achieve this, we make use of the withStartPosition(LogSequenceNumber.valueOf(lsn)) method and we are flushing last LSN

Is there any parameter/option we can utilize to make it read from the last LSN position?

Example :

LSN{1ECC/1D0438} --> BEGIN
LSN{1ECC/1D0456} 
LSN{1ECC/1D0536}
LSN{1ECC/1D0876} --> Although I intend to set this value as the starting position, it should ideally initiate from here. However it doesnt start from here, even though this LSN has already been flushed to the database
LSN{1ECC/1D0746}
LSN{1ECC/1D0837}
LSN{1ECC/1D0349} --> COMMIT

anilgol avatar Jan 17 '24 03:01 anilgol

@anilgol Read this comment.

eulerto avatar Jan 17 '24 14:01 eulerto

Thanks a lot!

anilgol avatar Jan 22 '24 12:01 anilgol