How to query hudi and iceberg in the Onetable.dev Docker Demo

[alberttwong]

Depends on: https://github.com/onetable-io/onetable/pull/320

Hudi Instructions

Note StarRocks is having problems with Hudi query. See https://github.com/onetable-io/onetable/issues/319

StarRocks > CREATE EXTERNAL CATALOG hudi_catalog_hms
    -> PROPERTIES
    -> (
    ->     "type" = "hudi",
    ->     "hive.metastore.type" = "hive",
    ->     "hive.metastore.uris" = "thrift://hive-metastore:9083",
    ->     "aws.s3.use_instance_profile" = "false",
    ->     "aws.s3.access_key" = "admin",
    ->     "aws.s3.secret_key" = "password",
    ->     "aws.s3.region" = "us-east-1",
    ->     "aws.s3.enable_ssl" = "false",
    ->     "aws.s3.enable_path_style_access" = "true",
    ->     "aws.s3.endpoint" = "http://minio:9000"
    -> );
Query OK, 0 rows affected (0.32 sec)

StarRocks > set catalog hudi_catalog_hms;
Query OK, 0 rows affected (0.00 sec)

StarRocks > show databases;
+--------------------+
| Database           |
+--------------------+
| default            |
| demo               |
| information_schema |
+--------------------+
3 rows in set (0.47 sec)

StarRocks > use demo
;
Reading table information for completion of table and column names
You can turn off this feature to get a quicker startup with -A

Database changed
StarRocks > use demo;
Database changed
StarRocks > show tables;
+------------------+
| Tables_in_demo   |
+------------------+
| hudi_dimcustomer |
+------------------+
1 row in set (0.05 sec)

StarRocks > select * from hudi_dimcustomer;
Empty set (1.86 sec)

Iceberg instructions

StarRocks > drop catalog iceberg;
Query OK, 0 rows affected (0.02 sec)

StarRocks > CREATE EXTERNAL CATALOG iceberg
    -> PROPERTIES
    -> (
    ->     "type" = "iceberg",
    ->     "iceberg.catalog.type" = "hive",
    ->     "hive.metastore.uris" = "thrift://hive-metastore:9083",
    ->     "aws.s3.use_instance_profile" = "false",
    ->     "aws.s3.access_key" = "admin",
    ->     "aws.s3.secret_key" = "password",
    ->     "aws.s3.region" = "us-east-1",
    ->     "aws.s3.enable_ssl" = "false",
    ->     "aws.s3.enable_path_style_access" = "true",
    ->     "aws.s3.endpoint" = "http://minio:9000"
    -> );
Query OK, 0 rows affected (0.02 sec)

StarRocks > set catalog iceberg;
Query OK, 0 rows affected (0.00 sec)

StarRocks > show databases;
+--------------------+
| Database           |
+--------------------+
| default            |
| demo               |
| information_schema |
+--------------------+
3 rows in set (0.17 sec)

StarRocks > use demo;
Reading table information for completion of table and column names
You can turn off this feature to get a quicker startup with -A

Database changed
StarRocks > show tables;
+------------------+
| Tables_in_demo   |
+------------------+
| hudi_dimcustomer |
+------------------+
1 row in set (0.05 sec)

StarRocks > select * from hudi_dimcustomer;
+------+-------------+--------------+-------------+---------------+------------+---------------+--------+--------------+---------------+----------------------+---------------------+----------------+----------------+-----------------+
| _c0  | CustomerKey | GeographyKey | FirstName   | LastName      | BirthDate  | MaritalStatus | Gender | YearlyIncome | TotalChildren | NumberChildrenAtHome | Education           | Occupation     | HouseOwnerFlag | NumberCarsOwned |
+------+-------------+--------------+-------------+---------------+------------+---------------+--------+--------------+---------------+----------------------+---------------------+----------------+----------------+-----------------+
| 4096 | 12965       | 543          | Arturo      | Nara          | 1934-09-22 | S             | M      | 10000.0      | 1.0           | 0.0                  | Graduate Degree     | Manual         | 1.0            | 0.0             |
| 4097 | 12966       | 483          | Dustin      | Shan          | 1962-07-26 | S             | M      | 10000.0      | 1.0           | 0.0                  | Graduate Degree     | Manual         | 0.0            | 0.0             |
| 4098 | 12967       | 498          | Grant       | Sharma        | 1962-03-03 | M             | M      | 20000.0      | 1.0           | 0.0                  | Graduate Degree     | Manual         | 0.0            | 0.0             |