gravitino icon indicating copy to clipboard operation
gravitino copied to clipboard

[Bug report] Doris table operation integration test failed occasionally

Open FANNG1 opened this issue 1 year ago • 4 comments

Version

PR: https://github.com/datastrato/gravitino/pull/3307

Describe what's wrong

doris table operation failed in ci occasionally, seems the state of doris is not healthy.

Failed ci: https://github.com/datastrato/gravitino/actions/runs/9025764657/job/24801995236.

Error message and/or stacktrace

DorisTableOperationsIT > testAlterTable() FAILED
    com.datastrato.gravitino.exceptions.GravitinoRuntimeException: errCode = 2, detailMessage = Table[doris_alter_test_table_9de02715]'s state is not NORMAL. Do not allow doing ALTER ops
        at app//com.datastrato.gravitino.catalog.doris.converter.DorisExceptionConverter.toGravitinoException(DorisExceptionConverter.java:66)
        at app//com.datastrato.gravitino.catalog.jdbc.operation.JdbcTableOperations.alterTable(JdbcTableOperations.java:247)
        at app//com.datastrato.gravitino.catalog.doris.integration.test.DorisTableOperationsIT.testAlterTable(DorisTableOperationsIT.java:184)

        Caused by:
        java.sql.SQLException: errCode = 2, detailMessage = Table[doris_alter_test_table_9de02715]'s state is not NORMAL. Do not allow doing ALTER ops
            at com.mysql.cj.jdbc.exceptions.SQLError.createSQLException(SQLError.java:129)
            at com.mysql.cj.jdbc.exceptions.SQLExceptionsMapping.translateException(SQLExceptionsMapping.java:122)
            at com.mysql.cj.jdbc.StatementImpl.executeUpdateInternal(StatementImpl.java:1333)
            at com.mysql.cj.jdbc.StatementImpl.executeLargeUpdate(StatementImpl.java:2106)
            at com.mysql.cj.jdbc.StatementImpl.executeUpdate(StatementImpl.java:1243)
            at org.apache.commons.dbcp2.DelegatingStatement.executeUpdate(DelegatingStatement.java:330)
            at org.apache.commons.dbcp2.DelegatingStatement.executeUpdate(DelegatingStatement.java:330)
            at com.datastrato.gravitino.catalog.jdbc.utils.JdbcConnectorUtils.executeUpdate(JdbcConnectorUtils.java:28)
            at com.datastrato.gravitino.catalog.jdbc.operation.JdbcTableOperations.alterTable(JdbcTableOperations.java:244)

How to reproduce

CI

Additional context

No response

FANNG1 avatar May 10 '24 01:05 FANNG1

seems couldn't reproduce on main branch, but easy to reproduce on PR: https://github.com/datastrato/gravitino/pull/3307

FANNG1 avatar May 10 '24 03:05 FANNG1

This bug is a known bug that occasionally occurs, but not frequently. The core reason is that alter operation in Doris is asynchronous and increasing the wait time will alleviate the problem, but it can't completely eliminate it. The following are related similar cases: https://github.com/datastrato/gravitino/actions/runs/9013450467/job/24764301966?pr=3100 https://github.com/datastrato/gravitino/actions/runs/9013889875/job/24765516186

yuqi1129 avatar May 10 '24 03:05 yuqi1129

seems not the same problem, the problem you mentioned is alter operation does not takes affect. but in this case, alter operation is not allowed for state is not NORMAL

FANNG1 avatar May 10 '24 03:05 FANNG1

Maybe it's a bug of Doris 1.2.7, in some cases, the Doris tablet is damaged.

I will track this issue.

zhoukangcn avatar May 10 '24 11:05 zhoukangcn

This issue has been temporarily resolved by increasing the wait time, so we will close it.

yuqi1129 avatar Aug 01 '24 13:08 yuqi1129