Error when creating test cases if the table contains the name COLUMN

[triquinielas]

DATA QUALITY TEST CASE

Describe the bug If the name of the table contains the word COLUMN it fails when creating a test case, it has been tested with different tables that contain the word and all of them fail, but if they do not contain the word COLUMN it works correctly.

To Reproduce

Expected behavior The performance is expected to be the same because a table can contain the name column, we have +180 tables with this nomenclature.

Version:

• OpenMetadata version: 1.3.2

Additional context we assume it will happen with more reserved words, but we have detected COLUMN

[TeddyCr]

@triquinielas I am not able to reproduce your error. Would you have the server logs to share by any chance

[triquinielas]

Hi @TeddyCr , thanks for your feedback, I will try to do some more testing and let you know, see if I am able to find any patterns.

[triquinielas]

hI @TeddyCr ,

I keep getting the same error, I have tried the latest version 1.3.3.

I have created in snowflake the following tables to try to make quality test, I have tried different combinations of uppercase, lowercase, and the term columns,column and col included.

CREATE OR REPLACE TABLE DIM_DATA_QUALITY_COLUMN AS SELECT 1 AS ID; CREATE OR REPLACE TABLE foo_column AS SELECT 1 AS ID; CREATE OR REPLACE TABLE foo_columns AS SELECT 1 AS ID; CREATE OR REPLACE TABLE foo_column AS SELECT 1 AS ID; CREATE OR REPLACE TABLE foo_column AS SELECT 1 AS ID CREATE OR REPLACE TABLE DIM_DATA_COLUMNS AS SELECT 1 AS ID;

CREATE OR REPLACE TABLE foo_col AS SELECT 1 AS ID; CREATE OR REPLACE TABLE foo_col AS SELECT 1 AS ID CREATE OR REPLACE TABLE DIM_DATA_QUALITY_COL AS SELECT 1 AS ID;

Only let me do quality test in foo_col and DIM_DATA_QUALITY_COL (last two), the rest gives me errors.

If I create a "table" test, I can do test case in all but if I do add test column, it fails me if it has the word COLUMN or COLUMNS.

Look at the error that gives open_metadata: table instance for HBX_SNOWFLAKE.HBG_GOVERNANCE.COMMUNITY_ANALYTICS.DIM_DATA_IDS not found

It's as if the word COLUMN disappears from the composition, when it should be HBX_SNOWFLAKE.HBG_GOVERNANCE.COMMUNITY_ANALYTICS.DIM_DATA_COLUMNS

thanks, Carlos

[TeddyCr]

Thanks @triquinielas let me try to replicate it in Snowflake

[TeddyCr]

I was able to reproduce it. Thanks @triquinielas for the details.

Error seems to come from the entityLink payload that is passed from the UI

{
  "name": "ID_column_value_lengths_to_be_between_7Fi9",
  "displayName": "ID_column_value_lengths_to_be_between_7Fi9",
  "entityLink": "<#E::table::MySQL.default.openmetadata_db.DIM_DATA_ID::columns::COLUMN>",
  "parameterValues": [
    {
      "name": "maxLength",
      "value": 2
    }
  ],
  "testDefinition": "columnValueLengthsToBeBetween",
  "testSuite": "MySQL.default.openmetadata_db.DIM_DATA_COLUMNS.testSuite",
  "owner": {
    "id": "f256b4a3-6dfd-454e-bd6a-66dab94ed3c4",
    "type": "user"
  }
}

[triquinielas]

Hi @TeddyCr, thanks for your effort and for trying identify the issue.

[TeddyCr]

@triquinielas in the meantime you should be able to create/run your test cases for this table via the CLI tool. More info here

source:
  type: TestSuite
  serviceName: MySQL
  sourceConfig:
    config:
      type: TestSuite
      entityFullyQualifiedName: MySQL.default.openmetadata_db.DIM_DATA_COLUMN

processor:
  type: "orm-test-runner"
  config:
    testCases:
      - name: myTestName
        description: test description
        columnName: ID
        testDefinitionName: columnValuesToBeUnique
        computePassedFailedRowCount: true

sink:
  type: metadata-rest
  config: {}
workflowConfig:
  openMetadataServerConfig:
    hostPort: "http://localhost:8585/api"
    authProvider: openmetadata
    securityConfig:
      jwtToken: <token>