Trino Gateway Routing Framework (WIP)

[oneonestar]

Trino Gateway Routing Framework

Summary

Introduce Trino Gateway Routing Framework that simplifies the extension of the routing function, facilitates testing and verification. This allows routing logics to be implemented as plugins.

Motivation

The routing logic required can vary significantly depending on the use case. The types of workloads that need to be handled and the methods for operating Trino clusters differ for each user.

Although the gateway supports some level of customization of routing logic using MVEL, Java code may be preferred due to its compile-time checking and ease of testing.

Additionally, the current default routing logic is hard-coded, making it difficult to understand or modify according to specific needs. Modularizing the default routing logic would facilitate better testing and customization.

Goals

• Extensible - Define a set of APIs that allow routing logics to be implemented as plugins.
• Testable - Each routing logic implemented as plugins can be tested individually. The framework should provide the necessary tools for testing.
• Stackable - Multiple plugins can be chained together to accommodate various needs. Each plugin decides either to make a routing decision or to pass it to the next plugin. Framework provides a mechanism allow data to be passed between plugins.
• Configurable - Allow each plugin to be configured individually. The execution order of the plugins are also adjustable.
• Efficient - Use lazy evaluation to avoid heavy computation when a routing decision can be made without it.
• Logging - The framework provides essential logging and tracing, enabling unified logging and configuration.

Non-goals

Remove or replace MVEL (currently supported by using Easy Rules). Defining routing logics using MVEL will continue to be supported.

Proposal

Architecture

The routing framework defines a set of APIs that allow routing logics to be implemented as plugins. These plugins can be chained in different order according to the configuration. Each plugin decides either to make a routing decision or to pass it to the next plugin.

Information can be passed from earlier plugins to later ones, allowing them to work together to form complex logic.

Lazy evaluation

Different plugin requires different information. Most of the information can be derived from the request. Lazy evaluation will be used to minimize the computation cost. For example, Route by queryID plugin needs a query ID to make a decision. The plugin try to get a query ID, which trigger a computation to try to extract query ID from the request. The computed result could be stored for other plugins to use later.

---

(Below are WIP & random ideas)

Example 1: X-Trino-Routing-Group

Input:
Request: SELECT 1, X-Trino-Routing-Group: adhoc
ClusterInfo:
Trino1: Routing group: "batch", healthy
Trino2: Routing group: "batch", healthy
Trino3: Routing group: "adhoc", healthy
Trino4: Routing group: "adhoc", unhealthy
Trino5: Routing group: "adhoc", healthy

Plugin 1: Filter unhealthy clusters
=> Decision: undecided
=> Remaining cluster: {Trino1, Trino2, Trino3, Trino5}

Plugin 2: Route by X-Trino-Routing-Group
=> Decision: undecided
=> Remaining cluster: {Trino3, Trino5}

Plugin 3: Route by User
=> Decision: undecided
=> Remaining cluster: {Trino3, Trino5}

Plugin 4: Random Pick one
=> Decision: decided
=> Remaining cluster: {Trino3}

Example 2: Route by User

Input:
Request: SELECT 1, X-Trino-User: batch_job_account
ClusterInfo:
Trino1: Routing group: "batch", healthy
Trino2: Routing group: "batch", healthy
Trino3: Routing group: "adhoc", healthy
Trino4: Routing group: "adhoc", unhealthy
Trino5: Routing group: "adhoc", healthy

Plugin 1: Filter unhealthy clusters
=> Decision: undecided
=> Remaining cluster: {Trino1, Trino2, Trino3, Trino5}

Plugin 2: Route by X-Trino-Routing-Group
=> Decision: undecided
=> Remaining cluster: {Trino1, Trino2, Trino3, Trino5}

Plugin 3: Route by User
[with config file: batch_job_account=>batch, adhoc_job_account=>adhoc]
=> Decision: undecided
=> Remaining cluster: {Trino1, Trino2}

Plugin 4: Random Pick one
=> Decision: decided
=> Remaining cluster: {Trino3}

Example 3: Load aware routing

Input:
Request: SELECT 1, X-Trino-User: batch_job_account, 
ClusterInfo:
Trino1: Routing group: "batch", healthy, 10 Running Query
Trino2: Routing group: "batch", healthy, 5 Running Query

(Skip some plugins)...

Plugin: Route by least load
=> Decision: undecided
=> Remaining cluster: {Trino2}

Other possible plugins:

• ML-based routing
• Route by query Type/table used/catalog used (using result from query parsing)
• Route by header (X-Trino-XXXXX)
• Route by cookie
• Route by query ID
• Route by RPC result from some external system
• A/B test (20% query to clusterA, )

Other random thoughts: ClusterStatsMonitor could grep more info from Trino, which enables more routing plugin to be implemented. eg.

• Catalog available in each Trino
• Worker count

Label based routing: Higher flexibility than the current routing group logic. Trino1: label={routing_group: batch, version: 440, env: prod} Trino2: label={routing_group: batch, version: 450, env: staging} Trino3: label={routing_group: adhoc, version: 440, env: test} Trino4: label={routing_group: admin, version: 440, env: prod}

Weight based routing: Weight cluster and distribute queries according to the weights. Weight cluster and select the highest weight cluster.

Other things that might need to be aware of:

• Query retry in other cluster
• Query result caching

Ref:

• https://kubernetes.io/docs/concepts/scheduling-eviction/scheduling-framework/
• https://github.com/kubernetes/kubernetes/blob/master/pkg/scheduler/framework/interface.go