gcp pubsub source: at_least_once

[AyWa]

Description

Following on https://github.com/quickwit-oss/quickwit/pull/3720 and https://github.com/quickwit-oss/quickwit/pull/3744 for https://github.com/quickwit-oss/quickwit/issues/1107

• [x] implements the missing check_connectivity
• [x] Goal of this PR, is to support at_least_once delivery, and add test for it.
• [x] add tests

How was this PR tested?

It is adding some test, to ensure that the message are ACK in order

Next step

In next PR we can focus on:

• ensure we extends message ack deadline (extends) or ensure that the subscription has a deadline of at least twice bigger than the flush
• benchmark perfs

[AyWa]

Looking good so far. This PR opens the door for proper unit testing of the checkpointing logic.

Actually I am now looking how to test the suggest_truncate. Because in the current test, nothing is calling suggest_truncate. Because there is no uploader right ? Does there is a test util to trigger suggest_truncate or other ?

[guilload]

Looking good so far. This PR opens the door for proper unit testing of the checkpointing logic.

Actually I am now looking how to test the suggest_truncate. Because in the current test, nothing is calling suggest_truncate. Because there is no uploader right ? Does there is a test util to trigger suggest_truncate or other ?

You can move the ack logic outside the suggest_truncate method of the Source trait and call the method directly on the GcpPubSubSource. Some Kafka tests do that. Or you can send SuggestTruncate messages to the actor` and perform some asserts via the observed state.