incubator-seata icon indicating copy to clipboard operation
incubator-seata copied to clipboard

dead lock when system starting

Open robinZhao opened this issue 5 years ago • 9 comments

  • [ ] I have searched the issues of this repository and believe that this is not a duplicate.

Ⅰ. Issue Description

dead lock when system starting with a thread operate database

thread1 lock(dataSourceProxyMap){ lock(springfactory ); }

thread2
lock (springfactory ){ lock (dataSourceProxyMap) }

Ⅱ. Describe what happened

If there is an exception, please attach the exception trace:

Found one Java-level deadlock:
=============================
"pool-3-thread-3":
  waiting to lock monitor 0x000002327c895708 (object 0x00000006c19c2808, a java.util.concurrent.ConcurrentHashMap),
  which is held by "main"
"main":
  waiting to lock monitor 0x000002327c86bc78 (object 0x00000007ad8ce650, a java.util.concurrent.ConcurrentHashMap$ReservationNode),
  which is held by "pool-3-thread-3"

Java stack information for the threads listed above:
===================================================
"pool-3-thread-3":
        at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.getSingletonFactoryBeanForTypeCheck(AbstractAutowireCapableBeanFactory.java:986)
        - waiting to lock <0x00000006c19c2808> (a java.util.concurrent.ConcurrentHashMap)
        at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.getTypeForFactoryBean(AbstractAutowireCapableBeanFactory.java:884)
        at org.springframework.beans.factory.support.AbstractBeanFactory.isTypeMatch(AbstractBeanFactory.java:619)
        at org.springframework.beans.factory.support.DefaultListableBeanFactory.doGetBeanNamesForType(DefaultListableBeanFactory.java:536)
        at org.springframework.beans.factory.support.DefaultListableBeanFactory.getBeanNamesForType(DefaultListableBeanFactory.java:503)
        at org.springframework.beans.factory.support.DefaultListableBeanFactory.getBeanNamesForType(DefaultListableBeanFactory.java:480)
        at org.springframework.beans.factory.support.DefaultListableBeanFactory.getBeanNamesForType(DefaultListableBeanFactory.java:473)
        at org.springframework.beans.factory.support.DefaultListableBeanFactory.resolveNamedBean(DefaultListableBeanFactory.java:1159)
        at org.springframework.beans.factory.support.DefaultListableBeanFactory.resolveBean(DefaultListableBeanFactory.java:420)
        at org.springframework.beans.factory.support.DefaultListableBeanFactory.getBean(DefaultListableBeanFactory.java:350)
        at org.springframework.beans.factory.support.DefaultListableBeanFactory.getBean(DefaultListableBeanFactory.java:343)
        at org.springframework.context.support.AbstractApplicationContext.getBean(AbstractApplicationContext.java:1127)
        at io.seata.spring.boot.autoconfigure.provider.SpringBootConfigurationProvider.get(SpringBootConfigurationProvider.java:93)
        at io.seata.spring.boot.autoconfigure.provider.SpringBootConfigurationProvider.access$100(SpringBootConfigurationProvider.java:43)
        at io.seata.spring.boot.autoconfigure.provider.SpringBootConfigurationProvider$1.intercept(SpringBootConfigurationProvider.java:56)
        at io.seata.config.FileConfiguration$$EnhancerByCGLIB$$862af1eb.getConfig(<generated>)
        at io.seata.discovery.registry.RegistryFactory.buildRegistryService(RegistryFactory.java:52)
        at io.seata.discovery.registry.RegistryFactory.getInstance(RegistryFactory.java:43)
        - locked <0x00000007ad418230> (a java.lang.Class for io.seata.discovery.registry.RegistryFactory)
        at io.seata.core.rpc.netty.NettyClientChannelManager.getAvailServerList(NettyClientChannelManager.java:216)
        at io.seata.core.rpc.netty.NettyClientChannelManager.reconnect(NettyClientChannelManager.java:162)
        at io.seata.core.rpc.netty.RmNettyRemotingClient.registerResource(RmNettyRemotingClient.java:181)
        at io.seata.rm.AbstractResourceManager.registerResource(AbstractResourceManager.java:121)
        at io.seata.rm.datasource.DataSourceManager.registerResource(DataSourceManager.java:146)
        at io.seata.rm.DefaultResourceManager.registerResource(DefaultResourceManager.java:114)
        at io.seata.rm.datasource.DataSourceProxy.init(DataSourceProxy.java:99)
        at io.seata.rm.datasource.DataSourceProxy.<init>(DataSourceProxy.java:85)
        at io.seata.rm.datasource.DataSourceProxy.<init>(DataSourceProxy.java:74)
        at io.seata.spring.annotation.datasource.DataSourceProxyHolder$$Lambda$673/773999389.apply(Unknown Source)
        at java.util.concurrent.ConcurrentHashMap.computeIfAbsent(ConcurrentHashMap.java:1660)
        - locked <0x00000007ad8ce650> (a java.util.concurrent.ConcurrentHashMap$ReservationNode)
        at io.seata.spring.annotation.datasource.DataSourceProxyHolder.putDataSource(DataSourceProxyHolder.java:64)
        at io.seata.spring.annotation.datasource.SeataAutoDataSourceProxyAdvice.invoke(SeataAutoDataSourceProxyAdvice.java:34)
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
        at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:749)
        at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:691)
        at org.apache.commons.dbcp.BasicDataSource$$EnhancerBySpringCGLIB$$48e7303.getConnection(<generated>)
        at org.springframework.jdbc.datasource.lookup.AbstractRoutingDataSource.getConnection(AbstractRoutingDataSource.java:170)
        at org.springframework.jdbc.datasource.DataSourceUtils.fetchConnection(DataSourceUtils.java:158)
        at org.springframework.jdbc.datasource.DataSourceUtils.doGetConnection(DataSourceUtils.java:116)
        at org.springframework.jdbc.datasource.DataSourceUtils.getConnection(DataSourceUtils.java:79)
        at org.mybatis.spring.transaction.SpringManagedTransaction.openConnection(SpringManagedTransaction.java:80)
        at org.mybatis.spring.transaction.SpringManagedTransaction.getConnection(SpringManagedTransaction.java:67)
        at org.apache.ibatis.executor.BaseExecutor.getConnection(BaseExecutor.java:337)
        at org.apache.ibatis.executor.SimpleExecutor.prepareStatement(SimpleExecutor.java:86)
        at org.apache.ibatis.executor.SimpleExecutor.doQuery(SimpleExecutor.java:62)
        at org.apache.ibatis.executor.BaseExecutor.queryFromDatabase(BaseExecutor.java:325)
        at org.apache.ibatis.executor.BaseExecutor.query(BaseExecutor.java:156)
        at org.apache.ibatis.executor.CachingExecutor.query(CachingExecutor.java:109)
        at com.github.pagehelper.PageInterceptor.intercept(PageInterceptor.java:143)
        at org.apache.ibatis.plugin.Plugin.invoke(Plugin.java:61)
        at com.sun.proxy.$Proxy293.query(Unknown Source)
        at org.apache.ibatis.session.defaults.DefaultSqlSession.selectList(DefaultSqlSession.java:147)
        at org.apache.ibatis.session.defaults.DefaultSqlSession.selectList(DefaultSqlSession.java:140)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at org.mybatis.spring.SqlSessionTemplate$SqlSessionInterceptor.invoke(SqlSessionTemplate.java:426)
        at com.sun.proxy.$Proxy110.selectList(Unknown Source)
        at org.mybatis.spring.SqlSessionTemplate.selectList(SqlSessionTemplate.java:223)
        at org.apache.ibatis.binding.MapperMethod.executeForMany(MapperMethod.java:147)
        at org.apache.ibatis.binding.MapperMethod.execute(MapperMethod.java:80)
        at org.apache.ibatis.binding.MapperProxy$PlainMethodInvoker.invoke(MapperProxy.java:152)
        at org.apache.ibatis.binding.MapperProxy.invoke(MapperProxy.java:85)
        at com.sun.proxy.$Proxy175.listScheduling(Unknown Source)
        at com.yky.cis.service.changeshifts.impl.ChangeshiftsServiceImpl.listScheduling(ChangeshiftsServiceImpl.java:92)
        at com.yky.cis.service.changeshifts.impl.ChangeshiftsServiceImpl$$FastClassBySpringCGLIB$$580714c5.invoke(<generated>)
        at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218)
        at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:771)
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)
        at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:749)
        at org.springframework.aop.framework.adapter.AfterReturningAdviceInterceptor.invoke(AfterReturningAdviceInterceptor.java:55)
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
        at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:749)
        at org.springframework.aop.framework.adapter.MethodBeforeAdviceInterceptor.invoke(MethodBeforeAdviceInterceptor.java:56)
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
        at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:749)
        at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:95)
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
        at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:749)
        at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:691)
        at com.yky.cis.service.changeshifts.impl.ChangeshiftsServiceImpl$$EnhancerBySpringCGLIB$$a34e7e97.listScheduling(<generated>)
        at com.yky.cis.job.StatisticsDataJob.getStatisticItems(StatisticsDataJob.java:172)
        at com.yky.cis.job.StatisticsDataJob.access$100(StatisticsDataJob.java:47)
        at com.yky.cis.job.StatisticsDataJob$2.run(StatisticsDataJob.java:100)
        at java.lang.Thread.run(Thread.java:748)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)
"main":
        at java.util.concurrent.ConcurrentHashMap.computeIfAbsent(ConcurrentHashMap.java:1674)
        - waiting to lock <0x00000007ad8ce650> (a java.util.concurrent.ConcurrentHashMap$ReservationNode)
        at io.seata.spring.annotation.datasource.DataSourceProxyHolder.putDataSource(DataSourceProxyHolder.java:64)
        at io.seata.spring.annotation.datasource.SeataAutoDataSourceProxyAdvice.invoke(SeataAutoDataSourceProxyAdvice.java:34)
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
        at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:749)
        at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:691)
        at org.apache.commons.dbcp.BasicDataSource$$EnhancerBySpringCGLIB$$48e7303.getConnection(<generated>)
        at org.springframework.jdbc.datasource.DataSourceUtils.fetchConnection(DataSourceUtils.java:158)
        at org.springframework.jdbc.datasource.DataSourceUtils.doGetConnection(DataSourceUtils.java:116)
        at org.springframework.jdbc.datasource.DataSourceUtils.getConnection(DataSourceUtils.java:79)
        at org.springframework.jdbc.support.JdbcUtils.extractDatabaseMetaData(JdbcUtils.java:336)
        at org.springframework.jdbc.support.JdbcUtils.extractDatabaseMetaData(JdbcUtils.java:373)
        at org.springframework.scheduling.quartz.LocalDataSourceJobStore.initialize(LocalDataSourceJobStore.java:150)
        at org.quartz.impl.StdSchedulerFactory.instantiate(StdSchedulerFactory.java:1368)
        at org.quartz.impl.StdSchedulerFactory.getScheduler(StdSchedulerFactory.java:1579)
        at org.springframework.scheduling.quartz.SchedulerFactoryBean.createScheduler(SchedulerFactoryBean.java:597)
        - locked <0x00000007ad2c7a90> (a org.quartz.impl.SchedulerRepository)
        at org.springframework.scheduling.quartz.SchedulerFactoryBean.afterPropertiesSet(SchedulerFactoryBean.java:480)
        at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.invokeInitMethods(AbstractAutowireCapableBeanFactory.java:1853)
        at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1790)
        at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:594)
        at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:516)
        at org.springframework.beans.factory.support.AbstractBeanFactory.lambda$doGetBean$0(AbstractBeanFactory.java:324)
        at org.springframework.beans.factory.support.AbstractBeanFactory$$Lambda$199/1875108260.getObject(Unknown Source)
        at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:226)
        - locked <0x00000006c19c2808> (a java.util.concurrent.ConcurrentHashMap)
        at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:322)
        at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:202)
        at org.springframework.beans.factory.config.DependencyDescriptor.resolveCandidate(DependencyDescriptor.java:276)
        at org.springframework.beans.factory.support.DefaultListableBeanFactory.doResolveDependency(DefaultListableBeanFactory.java:1307)
        at org.springframework.beans.factory.support.DefaultListableBeanFactory.resolveDependency(DefaultListableBeanFactory.java:1227)
        at org.springframework.beans.factory.annotation.AutowiredAnnotationBeanPostProcessor$AutowiredFieldElement.inject(AutowiredAnnotationBeanPostProcessor.java:640)
        at org.springframework.beans.factory.annotation.InjectionMetadata.inject(InjectionMetadata.java:130)
        at org.springframework.beans.factory.annotation.AutowiredAnnotationBeanPostProcessor.postProcessProperties(AutowiredAnnotationBeanPostProcessor.java:399)
        at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.populateBean(AbstractAutowireCapableBeanFactory.java:1420)
        at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:593)
        at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:516)
        at org.springframework.beans.factory.support.AbstractBeanFactory.lambda$doGetBean$0(AbstractBeanFactory.java:324)
        at org.springframework.beans.factory.support.AbstractBeanFactory$$Lambda$199/1875108260.getObject(Unknown Source)
        at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:226)
        - locked <0x00000006c19c2808> (a java.util.concurrent.ConcurrentHashMap)
        at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:322)
        at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:202)
        at org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:897)
        at org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:879)
        at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:551)
        - locked <0x00000006c169cf38> (a java.lang.Object)
        at org.springframework.boot.web.servlet.context.ServletWebServerApplicationContext.refresh(ServletWebServerApplicationContext.java:143)
        at org.springframework.boot.SpringApplication.refresh(SpringApplication.java:758)
        at org.springframework.boot.SpringApplication.refresh(SpringApplication.java:750)
        at org.springframework.boot.SpringApplication.refreshContext(SpringApplication.java:397)
        at org.springframework.boot.SpringApplication.run(SpringApplication.java:315)
        at org.springframework.boot.SpringApplication.run(SpringApplication.java:1237)
        at org.springframework.boot.SpringApplication.run(SpringApplication.java:1226)
        at com.yky.cis.CisApplication.main(CisApplication.java:14)

Found 1 deadlock.

Ⅲ. Describe what you expected to happen

starting normally

Ⅳ. How to reproduce it (as minimally and precisely as possible)

  1. xxx
  2. xxx
  3. xxx

Ⅴ. Anything else we need to know?

Ⅵ. Environment:

  • JDK version :
  • OS :
  • Others:

robinZhao avatar Mar 08 '21 01:03 robinZhao

请问你这边有热更新吗?以及部署的各种版本能否也说明一下?

caohdgege avatar Mar 08 '21 03:03 caohdgege

你是在启动过程中吗?为何会出现pool-3-thread-3这个线程

funky-eyes avatar Mar 08 '21 07:03 funky-eyes

你是在启动过程中吗?为何会出现pool-3-thread-3这个线程 因为有人在@PostConstruct中开了新线程操作数据库

robinZhao avatar Mar 08 '21 15:03 robinZhao

请问你这边有热更新吗?以及部署的各种版本能否也说明一下?

1.3.1 不是热更新时出的问题

robinZhao avatar Mar 08 '21 15:03 robinZhao

DataSourceProxyHolder#putDataSource deadlock (#3568) 这个问题描述有点相似

robinZhao avatar Apr 16 '21 14:04 robinZhao

public abstract class AbstractAutowireCapableBeanFactory extends AbstractBeanFactory
		implements AutowireCapableBeanFactory {
		.......
/**
	 * Obtain a "shortcut" singleton FactoryBean instance to use for a
	 * {@code getObjectType()} call, without full initialization of the FactoryBean.
	 * @param beanName the name of the bean
	 * @param mbd the bean definition for the bean
	 * @return the FactoryBean instance, or {@code null} to indicate
	 * that we couldn't obtain a shortcut FactoryBean instance
	 */
	@Nullable
	private FactoryBean<?> getSingletonFactoryBeanForTypeCheck(String beanName, RootBeanDefinition mbd) {
		synchronized (getSingletonMutex()) {
			BeanWrapper bw = this.factoryBeanInstanceCache.get(beanName);
			if (bw != null) {
				return (FactoryBean<?>) bw.getWrappedInstance();
			}
			Object beanInstance = getSingleton(beanName, false);
			if (beanInstance instanceof FactoryBean) {
				return (FactoryBean<?>) beanInstance;
			}
.........
        } 
}

/**
	 * Exposes the singleton mutex to subclasses and external collaborators.
	 * <p>Subclasses should synchronize on the given Object if they perform
	 * any sort of extended singleton creation phase. In particular, subclasses
	 * should <i>not</i> have their own mutexes involved in singleton creation,
	 * to avoid the potential for deadlocks in lazy-init situations.
	 */
	@Override
	public final Object getSingletonMutex() {
		return this.singletonObjects;
	}
}

经过查看代码,这个加锁的对象是存在于这个类中 是因为DataSourceProxyHolder中使用this.dataSourceProxyMap.computeIfAbsent(dataSource, DataSourceProxy::new),在对datasourceproxymap加锁后,在datasourceProxy的构造时又调用上述的方法,对SingletonMutex加锁 而另一个线程是在对beanFactory中的singletonMutex加锁后又调用了dataSourceProxyMap.computeIfAbsent,此时需要等待第一个线程持有的锁,从而造成死锁 这样分析,上面的改动并不能解决根本问题 建议在putDatasource时改用对beanFactory的singletonMutex加锁,共用周一个锁,不会造成死锁

robinZhao avatar Apr 16 '21 15:04 robinZhao

通过覆盖默认配置解决死锁的问题

@Configuration
@ConditionalOnClass(GlobalTransactional.class)
@ConditionalOnProperty(prefix = StarterConstants.SEATA_PREFIX, name = "enabled", havingValue = "true", matchIfMissing = true)
@AutoConfigureBefore(SeataAutoConfiguration.class)
public class SeataAutoConfig {
    Logger logger = LoggerFactory.getLogger(SeataAutoConfig.class);

    @Bean(BEAN_NAME_SEATA_AUTO_DATA_SOURCE_PROXY_CREATOR)
    @ConditionalOnProperty(prefix = StarterConstants.SEATA_PREFIX, name = {"enableAutoDataSourceProxy", "enable-auto-data-source-proxy"}, havingValue = "true", matchIfMissing = true)
    @ConditionalOnMissingBean(SeataAutoDataSourceProxyCreator.class)
    public SeataAutoDataSourceProxyCreator seataAutoDataSourceProxyCreator(SeataProperties seataProperties, AbstractBeanFactory abstractBeanFactory) {
        return new SeataAutoDataSourceProxyCreator(seataProperties.isUseJdkProxy(),seataProperties.getExcludesForAutoProxying()) {
            private Advisor fixAdvisor = new DefaultIntroductionAdvisor(new SeataAutoDataSourceProxyAdvice(){
                private HashMap<DataSource, DataSourceProxy> dataSourceProxyMap=new HashMap<>(8);
                @Override
                public Object invoke(MethodInvocation invocation) throws Throwable {
                    DataSource originDatasource = (DataSource) invocation.getThis();
                    DataSourceProxy dataSourceProxy = dataSourceProxyMap.get(originDatasource);
                    if(null==dataSourceProxy){
                        synchronized (abstractBeanFactory.getSingletonMutex()){
                            if(null==dataSourceProxyMap.get(originDatasource)){
                                dataSourceProxy=new DataSourceProxy(originDatasource);
                                dataSourceProxyMap.put(originDatasource, dataSourceProxy);
                            }
                        }
                    }
                    Method method = invocation.getMethod();
                    Object[] args = invocation.getArguments();
                    Method m = BeanUtils.findDeclaredMethod(DataSourceProxy.class, method.getName(), method.getParameterTypes());
                    if (m != null) {
                        return m.invoke(dataSourceProxy, args);
                    } else {
                        return invocation.proceed();
                    }
                }
            });
            @Override
            protected Object[] getAdvicesAndAdvisorsForBean(Class<?> beanClass, String beanName, TargetSource customTargetSource) throws BeansException {
                if (logger.isInfoEnabled()) {
                    logger.info("Auto proxy of ["+beanName+"]");
                }
                return new Object[]{fixAdvisor};
            }
        };
    }
}

robinZhao avatar Apr 19 '21 08:04 robinZhao

通过覆盖默认配置解决死锁的问题

@Configuration
@ConditionalOnClass(GlobalTransactional.class)
@ConditionalOnProperty(prefix = StarterConstants.SEATA_PREFIX, name = "enabled", havingValue = "true", matchIfMissing = true)
@AutoConfigureBefore(SeataAutoConfiguration.class)
public class SeataAutoConfig {
    Logger logger = LoggerFactory.getLogger(SeataAutoConfig.class);

    @Bean(BEAN_NAME_SEATA_AUTO_DATA_SOURCE_PROXY_CREATOR)
    @ConditionalOnProperty(prefix = StarterConstants.SEATA_PREFIX, name = {"enableAutoDataSourceProxy", "enable-auto-data-source-proxy"}, havingValue = "true", matchIfMissing = true)
    @ConditionalOnMissingBean(SeataAutoDataSourceProxyCreator.class)
    public SeataAutoDataSourceProxyCreator seataAutoDataSourceProxyCreator(SeataProperties seataProperties, AbstractBeanFactory abstractBeanFactory) {
        return new SeataAutoDataSourceProxyCreator(seataProperties.isUseJdkProxy(),seataProperties.getExcludesForAutoProxying()) {
            private Advisor fixAdvisor = new DefaultIntroductionAdvisor(new SeataAutoDataSourceProxyAdvice(){
                private HashMap<DataSource, DataSourceProxy> dataSourceProxyMap=new HashMap<>(8);
                @Override
                public Object invoke(MethodInvocation invocation) throws Throwable {
                    DataSource originDatasource = (DataSource) invocation.getThis();
                    DataSourceProxy dataSourceProxy = dataSourceProxyMap.get(originDatasource);
                    if(null==dataSourceProxy){
                        synchronized (abstractBeanFactory.getSingletonMutex()){
                            if(null==dataSourceProxyMap.get(originDatasource)){
                                dataSourceProxy=new DataSourceProxy(originDatasource);
                                dataSourceProxyMap.put(originDatasource, dataSourceProxy);
                            }
                        }
                    }
                    Method method = invocation.getMethod();
                    Object[] args = invocation.getArguments();
                    Method m = BeanUtils.findDeclaredMethod(DataSourceProxy.class, method.getName(), method.getParameterTypes());
                    if (m != null) {
                        return m.invoke(dataSourceProxy, args);
                    } else {
                        return invocation.proceed();
                    }
                }
            });
            @Override
            protected Object[] getAdvicesAndAdvisorsForBean(Class<?> beanClass, String beanName, TargetSource customTargetSource) throws BeansException {
                if (logger.isInfoEnabled()) {
                    logger.info("Auto proxy of ["+beanName+"]");
                }
                return new Object[]{fixAdvisor};
            }
        };
    }
}

SeataAutoDataSourceProxyCreator 是分支事务代理数据源的aop,其实他走到了SeataAutoDataSourceProxyAdvice的invoke方法这里已经持有了spring的单例锁,您又去synchronized (abstractBeanFactory.getSingletonMutex())似乎并不能解决问题

lrydl avatar Jul 22 '22 09:07 lrydl

seata启动timeCheck线程和main线程死锁问题:

seata的GlobalTransactionScanner类的initClient方法最终会调用到AbstractNettyRemotingClient#init,如下: 图片 reconnect方法最终调用到NacosRegistryServiceImpl#lookup,如下: 图片

综上所述:timecheck线程(延迟60s执行,间隔10s)会先请求seata LOCK_OBJ,再请求spring单例锁。

seata的代理数据源增强aop代码如下: 图片 图片 图片 只要main线程代码走到了SeataAutoDataSourceProxyAdvice的invoke方法,那么此时已经持有了spring单例锁,当他new DataSourceProxy的时候又想去获取seata的LOCK_OBJ锁。

综上所述:main线程持有了spring锁,想获取seata的LOCK_OBJ锁。

综上两个综上所述:timecheck线程先获取seata的LOCK_OBJ锁,再获取spring锁,main线程先获取spring锁,再获取seata的LOCK_OBJ锁。故当应用启动慢时(60s还没启动成功,就有可能发生main线程和timecheck线程死锁的现象)。

通过如下修改,可以让我们本机回显这个bug: 图片

官方能否尽快修复这个bug

lrydl avatar Jul 22 '22 09:07 lrydl

1.5.x及以上版本已修复

funky-eyes avatar Feb 08 '23 07:02 funky-eyes