rocketmq icon indicating copy to clipboard operation
rocketmq copied to clipboard

[ISSUE #3257] Ignore irrelevant commit log files on startup

Open schnell18 opened this issue 3 years ago • 3 comments

  • Prevent irrelevant files under $storePathCommitLog failing startup
  • Make RocketMQ easy to operate and bring less surprise to average administrators

Make sure set the target branch to develop

What is the purpose of the change

Make RocketMQ more tolerant to minor administrator mistake and reduce service outage in case less experienced admin create irrelevant files under $storePathCommitLog directory.

Brief changelog

Add regex (20-digit nubmer) filter irrelevant MappedFile names. Ignore files don't match the regex.

Verifying this change

Separate unit test is created to simulate irrelevant files present under commitlog and consumequeue directory. Unit test is passed.

Follow this checklist to help us incorporate your contribution quickly and easily. Notice, it would be helpful if you could finish the following 5 checklist(the last one is not necessary)before request the community to review your PR.

  • [x] Make sure there is a Github issue filed for the change (usually before you start working on it). Trivial changes like typos do not require a Github issue. Your pull request should address just this issue, without pulling in other changes - one PR resolves one issue.
  • [x] Format the pull request title like [ISSUE #123] Fix UnknownException when host config not exist. Each commit in the pull request should have a meaningful subject line and body.
  • [x] Write a pull request description that is detailed enough to understand what the pull request does, how, and why.
  • [x] Write necessary unit-test(over 80% coverage) to verify your logic correction, more mock a little better when cross module dependency exist. If the new feature or significant change is committed, please remember to add integration-test in test module.
  • [x] Run mvn -B clean apache-rat:check findbugs:findbugs checkstyle:checkstyle to make sure basic checks pass. Run mvn clean install -DskipITs to make sure unit-test pass. Run mvn clean test-compile failsafe:integration-test to make sure integration-test pass.
  • [ ] If this contribution is large, please file an Apache Individual Contributor License Agreement.

schnell18 avatar Aug 12 '21 15:08 schnell18

Coverage Status

Coverage increased (+0.02%) to 54.244% when pulling 7f8aa99b70a260e38065ba91f7bd5d0c2efce09f on schnell18:master into fea769da7c721e7497e892d3aed38115f8091679 on apache:develop.

coveralls avatar Aug 12 '21 16:08 coveralls

Codecov Report

Merging #3256 (a061fae) into master (a20f31b) will decrease coverage by 2.81%. The diff coverage is 50.00%.

Impacted file tree graph

@@             Coverage Diff              @@
##             master    #3256      +/-   ##
============================================
- Coverage     47.72%   44.91%   -2.82%     
+ Complexity     4544     3489    -1055     
============================================
  Files           552      470      -82     
  Lines         36628    28869    -7759     
  Branches       4844     3740    -1104     
============================================
- Hits          17482    12967    -4515     
+ Misses        16917    14199    -2718     
+ Partials       2229     1703     -526     
Impacted Files Coverage Δ
.../main/java/org/apache/rocketmq/common/UtilAll.java 38.18% <50.00%> (+0.52%) :arrow_up:
...org/apache/rocketmq/common/stats/StatsItemSet.java 43.28% <0.00%> (-7.47%) :arrow_down:
.../apache/rocketmq/logging/inner/LoggingBuilder.java 63.92% <0.00%> (ø)
...ketmq/client/impl/consumer/PullMessageService.java 75.55% <0.00%> (ø)
...rocketmq/test/client/rmq/RMQAsyncSendProducer.java
...rg/apache/rocketmq/store/config/FlushDiskType.java
...etmq/test/client/rmq/RMQTransactionalProducer.java
...est/listener/rmq/concurrent/RMQNormalListener.java
...ache/rocketmq/store/AllocateMappedFileService.java
...ache/rocketmq/store/config/MessageStoreConfig.java
... and 91 more

Continue to review full report at Codecov.

Legend - Click here to learn more Δ = absolute <relative> (impact), ø = not affected, ? = missing data Powered by Codecov. Last update a20f31b...a061fae. Read the comment docs.

codecov-commenter avatar Aug 19 '21 03:08 codecov-commenter

IMO, MappedFileQueue and MappedFile are basic components which may be used in other storages. Checking their names will decrease the flexibility of these components.

HScarb avatar Mar 01 '22 17:03 HScarb

This PR is stale because it has been open for 365 days with no activity. It will be closed in 3 days if no further activity occurs. If you wish not to mark it as stale, please leave a comment in this PR.

github-actions[bot] avatar Jul 20 '23 00:07 github-actions[bot]

This PR was closed because it has been inactive for 3 days since being marked as stale.

github-actions[bot] avatar Jul 23 '23 00:07 github-actions[bot]