release v2.0.0-alpha-1,测试过程中出现的一些问题
1、多表同步到一张表,在检测源表和目标表时出现报错 test_nvwi_0000.common = public.common test_nvwi_0001.common = public.common test_nvwi_0002.common = public.common test_nvwi_0003.common = public.common test_nvwi_0004.common = public.common test_nvwi_0005.common = public.common test_nvwi_0006.common = public.common test_nvwi_0007.common = public.common 报错信息,Greenplum table and MySQL table size are inconsistent!
手动注释掉以下相关检测代码,才可以启动 if (table_map.size() != tableMap.size()) { String message = "Greenplum table and MySQL table size are inconsistent!"; throw new BiremeException(message); } else { logger.info("MySQL、Greenplum table check completed, the state is okay!"); }
if (table_map.size() != tableMap.size()) {
String message = "some tables do not have primary keys!";
throw new BiremeException(message);
} else {
logger.info("Greenplum table primary key check is completed, the state is okay!");
}
2、很多表同步时,仍然出现表数目过多时,即pipeline过多时,表数据无法同步
Bireme "hang" if many table is configured:
- Greenplum is slow to delete/copy. At lease 16 segments is required in our test case. And if the workload is heavy, using resource queue to ensure bireme workload has enough resource.
- I assume heap table is used, right?
- Configure more database connection for Bireme by
loader.conn_pool.size, default is 10.
我在测试1.0版本bireme的时候,maxwell+kafka的方式,配置文件超过10个表的时候,就报错,报 PANIC: ERRORDATA_STACK_SIZE exceeded 。我把loader.conn_pool.size 改成20都没用。 请问这种情况怎么处理,上述说的using resource queue to ensure bireme workload has enough resource这个是啥。
PANIC: ERRORDATA_STACK_SIZE exceeded is an error throw by Greenplum, please ask in Greenplum community.
resource queue is a feature of Greenplum, please refer its documents.
PANIC: ERRORDATA_STACK_SIZE exceededis an error throw by Greenplum, please ask in Greenplum community.
resource queueis a feature of Greenplum, please refer its documents.
你们之前开发的时候使用的是哪个版本的greenplum?或者哪些个版本的postgre或者greenplum是支持的?我是真的没办法了。。头疼。。
PANIC: ERRORDATA_STACK_SIZE exceeded usually means a bug of Greenplum. Please report an issue to https://github.com/greenplum-db/gpdb. Currently Greenplum 4.3.x and 5.x is stable.
PANIC: ERRORDATA_STACK_SIZE exceededusually means a bug of Greenplum. Please report an issue to https://github.com/greenplum-db/gpdb. Currently Greenplum 4.3.x and 5.x is stable. 我们现在使用的是greenplum 5.10.2(PostgreSQL 8.3.23),我上greenplum上面去问问吧,谢谢~