[BUG] S3WAL failed with 'WALHeader magic code not match' error

Open superhx opened this issue 8 months ago • 1 comments

[2025-04-27 07:21:26,610] ERROR S3Storage start fail (com.automq.stream.s3.S3Storage)
com.automq.stream.s3.wal.exception.UnmarshalException: WALHeader magic code not match, Recovered: [0] expect: [305419896]
        at com.automq.stream.s3.wal.impl.object.WALObjectHeader.unmarshal(WALObjectHeader.java:63)
        at com.automq.stream.s3.wal.impl.object.ObjectWALService$RecoverIterator.loadNextBuffer(ObjectWALService.java:225)
        at com.automq.stream.s3.wal.impl.object.ObjectWALService$RecoverIterator.next(ObjectWALService.java:257)
        at com.automq.stream.s3.wal.impl.object.ObjectWALService$RecoverIterator.next(ObjectWALService.java:191)
        at com.automq.stream.s3.S3Storage.recoverContinuousRecords0(S3Storage.java:298)
        at com.automq.stream.s3.S3Storage.recoverContinuousRecords(S3Storage.java:271)
        at com.automq.stream.s3.S3Storage.recoverContinuousRecords(S3Storage.java:221)
        at com.automq.stream.s3.S3Storage.recover0(S3Storage.java:395)
        at com.automq.stream.s3.S3Storage.recover(S3Storage.java:375)

Apr 27 '25 07:04 superhx

The bug is caused by PR https://github.com/AutoMQ/automq/pull/2356 . The fast retry success before normal path and the normal path upload the dirty data.

Apr 27 '25 12:04 superhx