MyTile

MariaDB storage engine based on TileDB.

Quickstart Usage

Docker

Docker images are available on Docker Hub for quick testing of the mytile storage engine.

Variants

• tiledb-mariadb launches an interactive shell
• tiledb-mariadb-server provides a more traditional (non-interactive) mariadb server

Supported tags

• latest: latest stable release (recommended)
• dev: development version
• v0.x.x for a specific version

Examples

The tiledb-mariadb image starts a mariadb server and connects to it from the shell for you.

docker run --rm -e MYSQL_ALLOW_EMPTY_PASSWORD=1 -it tiledb/tiledb-mariadb

If you want to access arrays on s3, you will need to add your AWS keys as env variables

docker run --rm -e MYSQL_ALLOW_EMPTY_PASSWORD=1 -e AWS_ACCESS_KEY_ID="<key>" -e AWS_SECRET_ACCESS_KEY="<secret>" -it tiledb/tiledb-mariadb

or mount a local array into the Docker container with the -v option:

docker run -e MYSQL_ALLOW_EMPTY_PASSWORD=1 -it --rm -v /local/array/path:/data/local_array tiledb/tiledb-mariadb

Example SQL Usage

TileDB Arrays can be access directly via their URI.

Show create table:

show create table `s3://my/array`\G

Basic select:

select * from `s3://my/array`;

Use assisted table discovery for long uris:

create table my_array ENGINE=mytile uri='s3://my_bucket/arrays/sub_prefix/path_is_longer_than_64_chars/my_array_1';
select * from my_array;

Create table:

CREATE TABLE `s3://bucket/regions`(
  regionkey bigint WITH (dimension=true),
  name varchar,
  comment varchar
  ) uri = 's3://bucket/region' array_type='SPARSE';

Installation

Docker

To build either docker image locally simply use the following:

git clone https://github.com/TileDB-Inc/TileDB-MariaDB.git
cd TileDB-MariaDB

# tiledb-mariadb
docker build -t mytile -f docker/Dockerfile .

# tiledb-mariadb-server
docker build -t mytile -f docker/Dockerfile-server .

Requirements

Requires MariaDB 10.4.8 or newer.

Inside MariaDB Source Tree

git clone [email protected]:MariaDB/server.git -b mariadb-10.5.8
cd server
git submodule add https://github.com/TileDB-Inc/TileDB-MariaDB.git storage/mytile
mkdir build && cd build
cmake ..
make -j4

Running Unit Test

Once MariaDB has been build you can run unit tests from the build directory:

./mysql-test/mtr --suite mytile

That will run all unit tests defined for mytile

Features

• Based on TileDB arrays
• Supports basic pushdown of predicates for dimensions
• Supports basic pushdown of query conditions for attributes
• Supports basic pushdown of aggregates (SUM, AVG, MAX, MIN) for attributes
• Create arrays through CREATE TABLE syntax.
• Existing arrays can be dynamically queried
• Supports all datatypes

Known Issues

• Condition pushdown only works for constants not sub selects
• Buffers will double in size for incomplete queries with zero results
• MyTile is not capable of binlogging currently both stmt and row based is disabled at the storage engine level
• Aggregates on multi-valued attributes are not supported.
• Aggregate pushdown does not work when using multiple functions on a column. e.g. SELECT COALESCE(SUM(count), 0) from allele;. Such queries revert back to MariaDB filtering.
• Aggregate pushdown does not work with GROUP BYs. Such queries revert back to MariaDB filtering.