metabase-driver icon indicating copy to clipboard operation
metabase-driver copied to clipboard

The Browse data menu does not show database's schema and tables when use aws glue hive metastore

Open ahululu opened this issue 7 months ago • 0 comments

Metastore: AWS Glue

trino helm settings:

image:
  tag: "450"
server:
  workers: 1
  exchangeManager:
    baseDir: "s3://xxxx"
  autoscaling:
    enabled: true
    maxReplicas: 20
    targetCPUUtilizationPercentage: 70
  config:
    query:
      maxMemory: "4GB"
service:
  type: ClusterIP
  port: 8080
coordinator:
  jvm:
    maxHeapSize: "4G"
  additionalJVMConfig:
    - "-XX:+UnlockDiagnosticVMOptions"
  config:
    query:
      maxMemoryPerNode: "2GB"
worker:
  jvm:
    maxHeapSize: "12G"
  additionalJVMConfig:
    - "-XX:+UnlockDiagnosticVMOptions"
  config:
    query:
      maxMemoryPerNode: "8GB"
  resources:
    requests:
      cpu: 4000m
      memory: 12Gi
additionalConfigProperties:
  - "retry-policy=TASK"
  - "exchange.compression-enabled=true"
  - "query.low-memory-killer.delay=0s"
  - "query.remote-task.max-error-duration=1m"
  - "query.hash-partition-count=50"
additionalExchangeManagerProperties:
  - "exchange.s3.region=xxxxx"
  - "exchange.s3.iam-role=xxxxxx"
additionalCatalogs:
  hive: |-
    connector.name=hive
    hive.metastore=glue
    hive.metastore.glue.region=xxxxx
    hive.metastore.glue.endpoint-url=https://glue.xxxxx.amazonaws.com
    hive.metastore.glue.max-connections=30
    hive.metastore.glue.max-error-retries=3
    hive.metastore.glue.catalogid=1111111111
    hive.metastore.glue.iam-role=xxxxx
serviceAccount:
  create: true
  name: trino-sa
accessControl:
  type: configmap
  refreshPeriod: 60s
  configFile: "rules.json"
  rules:
    rules.json: |-
      {
        "catalogs": [
          {
            "user": "xxxxx",
            "catalog": ".*",
            "allow": "all"
          }
        ],
        "schemas": [
          {
            "user": "xxxxx",
            "schema": ".*",
            "owner": true
          }

      ],
        "tables": [
          {
            "user": "xxxx",
            "privileges": ["SELECT", "OWNERSHIP"]
          }
        ]
      }
ingress:
  enabled: true
  annotations:
    alb.ingress.kubernetes.io/backend-protocol: HTTP
    alb.ingress.kubernetes.io/certificate-arn: xxx
    alb.ingress.kubernetes.io/group.name: xxx
    alb.ingress.kubernetes.io/healthcheck-path: /
    alb.ingress.kubernetes.io/healthcheck-protocol: HTTP
    alb.ingress.kubernetes.io/listen-ports: '[{"HTTPS":443}]'
    alb.ingress.kubernetes.io/subnets: sxxx
    alb.ingress.kubernetes.io/success-codes: xx
    alb.ingress.kubernetes.io/target-type: ip
    kubernetes.io/ingress.class: alb
  hosts:
  - paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: trino
            port:
              number: 8080
    host: xx.xx.xx

Metabase database settings and operation:

1. Database type: Starburst
2. Host: xxxx
3. Port: 443
4. Catalog: hive
5. Schema: 
6. Username: xxxxx
7. Use a secure connection (SSL): enable
8. Rerun queries for simple explorations: enable

click button:  'Sync database schema now' and 'Re-scan field values now' 
Then go back to the metabase home page, Browse data select the corresponding database, there is no schema display, but you can use 'Sql query' to query the data

ahululu avatar Jul 17 '24 12:07 ahululu