activemq icon indicating copy to clipboard operation
activemq copied to clipboard

AMQ-8049: Failed to start Apache ActiveMQ (mKahaDB / JMX)

Open lraczko opened this issue 2 years ago • 7 comments

When using filtered adapter for topics or queues with perDestination on catch all, after restart it is possible to have duplicated adapters when there are destinations in metadata and as a result start will fail. Perdestination should not start adapter for directory read from filesystem which is already defined in configured adapters.

lraczko avatar Mar 01 '23 11:03 lraczko

Hey @jbonofre

I need help and consultation :). I keep getting some random fails in tests (locally on my machine I had few successes but also fails. On your pipeline there were only fails). They do not seem to be connected to my change (and when I rerun them locally then they work), but still I am not fan of giving to code review commit with failed verification. Do you have some hints what should I do and how you guys proceed in such cases?

lraczko avatar Apr 26 '23 11:04 lraczko

Let me take a look on this one. For the tests, it's possible we still have some flaky tests, let me do a pass on these tests.

jbonofre avatar Jun 06 '23 12:06 jbonofre

How does the error occur when the adapters are added via directory listing? There should not be duplication.

mattrpav avatar Jun 06 '23 17:06 mattrpav

When you have configuration like:

	<mKahaDB directory="${activemq.data}/kahadb">
		<filteredPersistenceAdapters>
			<!-- kahaDB per destinations -->
			<filteredKahaDB perDestination="true">
				<persistenceAdapter>
					<kahaDB>
						...
					</kahaDB>
				</persistenceAdapter>
			</filteredKahaDB>
			<filteredKahaDB topic="STOMP.>">
				<persistenceAdapter>
					<kahaDB>
						...
					</kahaDB>
				</persistenceAdapter>
			</filteredKahaDB>
		</filteredPersistenceAdapters>
	</mKahaDB>
</persistenceAdapter>

On first start everything works fine, but after restart in org.apache.activemq.store.kahadb.MultiKahaDBPersistenceAdapter#doStart you first go through 'matchAll' and for this example topic store in org.apache.activemq.store.kahadb.MultiKahaDBPersistenceAdapter#registerExistingAdapter you register and start adapter (as destinations.size() != 0) and when you get to

        if (candidates != null) {
            for (File candidate : candidates) {
                registerExistingAdapter(template, candidate);
            }
        }

in mentioned 'doStart' then it will try to register and start same adapter second time and corrupt index. And after that another restart will make AMQ fail to start because of that corrupted index.

lraczko avatar Jun 07 '23 08:06 lraczko

@jbonofre - Did you have time to take a look?

lraczko avatar Jun 16 '23 06:06 lraczko

@jbonofre - any news? :D

lraczko avatar Jul 10 '23 13:07 lraczko

I'm resuming my test/investigation on this one.

jbonofre avatar Oct 20 '23 06:10 jbonofre