starrocks icon indicating copy to clipboard operation
starrocks copied to clipboard

[BUG] Elasticsearch category table does not exist when the es index contains dynamic_templates in the mapping

Open jdzchenxin opened this issue 1 year ago • 1 comments

Steps to reproduce the behavior (Required)

  1. Create elasticsearch index "test_index"
PUT test_index
{
  "mappings": {
    "dynamic_templates": [
       {
        "tags_template": {
          "match_mapping_type": "string",
          "match":   "tags*",
          "mapping": {
            "type": "keyword"
          }
        }
      }
    ],
    "properties": {
      "id": {
        "type": "keyword"
      },
      "title": {
        "type": "text"
      }
    }
  }
}
  1. Create elasticsearch catalog in starrocks
CREATE EXTERNAL CATALOG es_catalog
comment "ES CATALOG"
PROPERTIES (
	"hosts"  =  "https://127.0.0.1:9200",
	"type"  =  "es",
	"user"  =  "xxxxxx",
	"password" = "xxxxxx",
	"es.nodes.wan.only" = "true"
)
  1. Verify table and select data in starrocks
mysql> SET CATALOG es8_catalog;
mysql> USE default_db;
mysql> show tables like 'test_index%';
mysql> select * from test_index;

Expected behavior (Required)

  1. show table return 1 result
  2. "select * from test_index" return success;

Real behavior (Required)

  1. show table return 0 result
  2. "select * from test_index" return :"ERROR 1064 (HY000): Getting analyzing error. Detail message: Unknown table 'default_db.test_index'."

Later, I drop the es_catalog, and rebuild it without part of "dynamic_templates", the table appeared in show tables list, and i success select it with the SQL.

StarRocks version (Required)

4.2.4-613f0b5

Elasticsearch version

8.8.2

jdzchenxin avatar Apr 22 '24 10:04 jdzchenxin

We have marked this issue as stale because it has been inactive for 6 months. If this issue is still relevant, removing the stale label or adding a comment will keep it active. Otherwise, we'll close it in 10 days to keep the issue queue tidy. Thank you for your contribution to StarRocks!

github-actions[bot] avatar Oct 21 '24 11:10 github-actions[bot]

Same problem starrocks v3.3.4 , I think documentation is not complete about elasticsearch external catalog. Only mention function esquery() when is an external table function (deprecated) in starrocks.

image Or maybe is mysql client not compatible with table names of external catalog?

jmorcar avatar Oct 30 '24 17:10 jmorcar

We have marked this issue as stale because it has been inactive for 6 months. If this issue is still relevant, removing the stale label or adding a comment will keep it active. Otherwise, we'll close it in 10 days to keep the issue queue tidy. Thank you for your contribution to StarRocks!

github-actions[bot] avatar May 05 '25 11:05 github-actions[bot]