bireme icon indicating copy to clipboard operation
bireme copied to clipboard

release v2.0.0-alpha-1,测试过程中出现的一些问题

Open dwwang1992 opened this issue 7 years ago • 6 comments

1、多表同步到一张表,在检测源表和目标表时出现报错 test_nvwi_0000.common = public.common test_nvwi_0001.common = public.common test_nvwi_0002.common = public.common test_nvwi_0003.common = public.common test_nvwi_0004.common = public.common test_nvwi_0005.common = public.common test_nvwi_0006.common = public.common test_nvwi_0007.common = public.common 报错信息,Greenplum table and MySQL table size are inconsistent!

手动注释掉以下相关检测代码,才可以启动 if (table_map.size() != tableMap.size()) { String message = "Greenplum table and MySQL table size are inconsistent!"; throw new BiremeException(message); } else { logger.info("MySQL、Greenplum table check completed, the state is okay!"); }

  if (table_map.size() != tableMap.size()) {
    String message = "some tables do not have primary keys!";
    throw new BiremeException(message);
  } else {
    logger.info("Greenplum table primary key check is completed, the state is okay!");
  }

2、很多表同步时,仍然出现表数目过多时,即pipeline过多时,表数据无法同步

dwwang1992 avatar Aug 13 '18 09:08 dwwang1992

Bireme "hang" if many table is configured:

  1. Greenplum is slow to delete/copy. At lease 16 segments is required in our test case. And if the workload is heavy, using resource queue to ensure bireme workload has enough resource.
  2. I assume heap table is used, right?
  3. Configure more database connection for Bireme by loader.conn_pool.size, default is 10.

wangzw avatar Aug 25 '18 09:08 wangzw

我在测试1.0版本bireme的时候,maxwell+kafka的方式,配置文件超过10个表的时候,就报错,报 PANIC: ERRORDATA_STACK_SIZE exceeded 。我把loader.conn_pool.size 改成20都没用。 请问这种情况怎么处理,上述说的using resource queue to ensure bireme workload has enough resource这个是啥。

chengshao1987 avatar Nov 15 '18 03:11 chengshao1987

PANIC: ERRORDATA_STACK_SIZE exceeded is an error throw by Greenplum, please ask in Greenplum community.

resource queue is a feature of Greenplum, please refer its documents.

wangzw avatar Nov 15 '18 03:11 wangzw

PANIC: ERRORDATA_STACK_SIZE exceeded is an error throw by Greenplum, please ask in Greenplum community.

resource queue is a feature of Greenplum, please refer its documents.

你们之前开发的时候使用的是哪个版本的greenplum?或者哪些个版本的postgre或者greenplum是支持的?我是真的没办法了。。头疼。。

chengshao1987 avatar Nov 15 '18 10:11 chengshao1987

PANIC: ERRORDATA_STACK_SIZE exceeded usually means a bug of Greenplum. Please report an issue to https://github.com/greenplum-db/gpdb. Currently Greenplum 4.3.x and 5.x is stable.

wangzw avatar Nov 15 '18 10:11 wangzw

PANIC: ERRORDATA_STACK_SIZE exceeded usually means a bug of Greenplum. Please report an issue to https://github.com/greenplum-db/gpdb. Currently Greenplum 4.3.x and 5.x is stable. 我们现在使用的是greenplum 5.10.2(PostgreSQL 8.3.23),我上greenplum上面去问问吧,谢谢~

chengshao1987 avatar Nov 15 '18 10:11 chengshao1987