pulsar icon indicating copy to clipboard operation
pulsar copied to clipboard

[Bug] bookie-2 is not able to recover after lossing the filesystem

Open cccdemon opened this issue 1 year ago • 0 comments

Search before asking

  • [X] I searched in the issues and found nothing similar.

Read release policy

  • [X] I understand that unsupported versions don't get bug fixes. I will attempt to reproduce the issue on a supported version of Pulsar client and Pulsar broker.

Version

Kubernetes 1.29 Pulsar 3.2.3

Minimal reproduce step

scale down to 1 bookies delete the related pvc/pv for bookie-2

What did you expect to see?

a full recover by 1 bookie

What did you see instead?

2024-07-17T10:10:23,047+0000 [main] ERROR org.apache.bookkeeper.bookie.LegacyCookieValidation - There are directories without a cookie, and this is neither a new environment, nor is storage expansion enabled. Empty directories are [/pulsar/data/bookkeeper/journal/current, /pulsar/data/bookkeeper/ledgers/current] 2024-07-17T10:10:23,048+0000 [main] ERROR org.apache.bookkeeper.server.Main - Failed to build bookie server org.apache.bookkeeper.bookie.BookieException$InvalidCookieException: at org.apache.bookkeeper.bookie.LegacyCookieValidation.checkCookies(LegacyCookieValidation.java:113) ~[org.apache.bookkeeper-bookkeeper-server-4.16.5.jar:4.16.5] at org.apache.bookkeeper.server.EmbeddedServer$Builder.build(EmbeddedServer.java:408) ~[org.apache.bookkeeper-bookkeeper-server-4.16.5.jar:4.16.5] at org.apache.bookkeeper.server.Main.buildBookieServer(Main.java:277) ~[org.apache.bookkeeper-bookkeeper-server-4.16.5.jar:4.16.5] at org.apache.bookkeeper.server.Main.doMain(Main.java:216) ~[org.apache.bookkeeper-bookkeeper-server-4.16.5.jar:4.16.5] at org.apache.bookkeeper.server.Main.main(Main.java:199) ~[org.apache.bookkeeper-bookkeeper-server-4.16.5.jar:4.16.5]

Anything else?

No response

Are you willing to submit a PR?

  • [x] I'm willing to submit a PR!

cccdemon avatar Jul 17 '24 10:07 cccdemon