Acknowledge - Is it working ?

[dmarro89]

trafficstars

Hi, I might not fully understand how 'acknowledge' works, but I'll try to explain my situation. I've created a new Logical Replication Service like this: new LogicalReplicationService(config, { acknowledge: { auto: false, timeoutSeconds: 10, }, })

Then, I subscribe to event changes and receive one event change, but I'm not manually sending any acknowledgment. When I restart the subscription, I don't receive the events that haven't received an acknowledgment. Why is that? Am I using this feature incorrectly, or is there a bug?

[AgustLuj]

Im in the same. Do you have some solution or use some another package ?

[whygee-dev]

@AgustLuj facing the same issue, did you have any luck with solving it ?

[AgustLuj]

@whygee-dev nop, the only I know if i wait a few minutes when I call the function. The ack is automatic for all the events.

[dmarro89]

I've found the solution: change the project you're working on 😂

[kibae]

Hello, guys :)

I have understood your question while writing the relevant test code. You can find it here: https://github.com/kibae/pg-logical-replication/commit/c5dbda3088443c8ad1dcd225386f76aebf1ab68b#diff-746957515044ada07b4076effa099157208566b32c2a0bdcf581fd8255f434c1

const service = new LogicalReplicationService(config, { acknowledge: { auto: false, timeoutSeconds: 10 } });
const plugin = new Wal2JsonPlugin({});

service.subscribe(plugin, slotName, lastLsn /* optional */);

When calling service.subscribe, the 3rd argument is lastLsn. If not provided, the default value is '0/00000000', which means all data in the corresponding logical replication slot could be received. You need to manage the LSN of the data changes you stream separately. This allows you to only receive data following the last received point the next time you subscribe. Therefore, if you are managing your lastLsn well, you can simply pass it on the next subscribe() call.

Please examine the test code where it performs subscribe -> stop -> subscribe. If you have any other issues, feel free to ask questions at any time.

[stephen-dahl]

Your test suggests to me that acknowledge doesn't actually do what I thought it does. I thought that it was a way of marking how far in the replication log I have processed. If my client crashes without acknowledging it would reprocess those logs when it restarts. Is this not how it works?

the test you added doesn't show calling acknowledge manually. What is the effect of calling or not calling acknowledge?

[kibae]

SELECT *
FROM pg_create_logical_replication_slot('slot_a', 'test_decoding')

From this point, PostgreSQL begins to save changes in the WAL, which includes an ID called the LSN for each storage point. As data changes occur, they are logged in the WAL and begin to be streamed to the client, including the current data's LSN. If you do not send an acknowledgment signal, the server will not delete the WAL and will retain it. The files will only be deleted when you drop the slot or send an acknowledgment.

sequenceDiagram
    participant Client
    participant PostgreSQL

    note over Client,PostgreSQL: Create logical replication slot named "slot_a"
    Client->>PostgreSQL: SELECT * FROM pg_create_logical_replication_slot('slot_a', 'test_decoding')
    PostgreSQL->>Client: Success response with start LSN(Log Sequence Number)

    loop Data changed
        PostgreSQL-->>Client: changed data with LSN
        alt
            Client-->>PostgreSQL: acknowledge with LSN
            PostgreSQL->>PostgreSQL: OK. I can delete the WAL files under my control.
            Client->>Client: Save the last LSN for continue to get data in the future.
        end
    end

    Client->>Client: Disconnect
    Client->>PostgreSQL: Connect again and resume replication with last LSN
    loop Data changed
        PostgreSQL-->>Client: changed data with LSN
        alt
            Client-->>PostgreSQL: acknowledge with LSN
            PostgreSQL->>PostgreSQL: OK. I can delete the WAL files under my control.
            Client->>Client: Save the last LSN for continue to get data in the future.
        end
    end