Experimental. WIP: Multi-tenant import support in Vitess

[rohit-nayak-ps]

Description

While importing from a multi-tenant database, using MoveTables, it is possible one wants to only import some of the tenants. This PR allows the user to specify an additional filter like tenant_id=1. This filter is applied to both the forward and reverse workflows.

It also adds keyspace routing rules, which allow entire keyspaces to be routed to the source or target keyspace for MoveTables workflows, depending on whether are in progress or switched, respectively.

For applications that use a database qualifier for their per-tenant queries and the qualifiers do not match the source keyspace defined in Vitess, an option is added to also route this database to the same route pointed to by the source keyspace.

MoveTables changes

There are three new options to MoveTables

• --use-keyspace-routing-rules : uses keyspace routing rules instead of table routing rules
• --source-keyspace-elias : also add provided name as alias while mapping the source keyspace to the source or target keyspace, depending on whether traffic has been switched or not
• --additional-filter: adds the provided sql where clause predicate to the forward and reverse workflow binlogsource filters.

TBD:

• [ ] Unit tests
• [ ] Improve e2e test
• [ ] Test reverse traffic
• [ ] Compress rules if there are large number of tenants?

Related Issue(s)

Checklist

• [ ] "Backport to:" labels have been added if this change should be back-ported to release branches
• [ ] If this change is to be back-ported to previous releases, a justification is included in the PR description
• [ ] Tests were added or are not required
• [ ] Did the new or modified tests pass consistently locally and on CI?
• [ ] Documentation was added or is not required

[vitess-bot[bot]]

Review Checklist

Hello reviewers! :wave: Please follow this checklist when reviewing this Pull Request.

General

• [ ] Ensure that the Pull Request has a descriptive title.
• [ ] Ensure there is a link to an issue (except for internal cleanup and flaky test fixes), new features should have an RFC that documents use cases and test cases.

Tests

• [ ] Bug fixes should have at least one unit or end-to-end test, enhancement and new features should have a sufficient number of tests.

Documentation

• [ ] Apply the release notes (needs details) label if users need to know about this change.
• [ ] New features should be documented.
• [ ] There should be some code comments as to why things are implemented the way they are.
• [ ] There should be a comment at the top of each new or modified test to explain what the test does.

New flags

• [ ] Is this flag really necessary?
• [ ] Flag names must be clear and intuitive, use dashes (-), and have a clear help text.

If a workflow is added or modified:

• [ ] Each item in Jobs should be named in order to mark it as required.
• [ ] If the workflow needs to be marked as required, the maintainer team must be notified.

Backward compatibility

• [ ] Protobuf changes should be wire-compatible.
• [ ] Changes to _vt tables and RPCs need to be backward compatible.
• [ ] RPC changes should be compatible with vitess-operator
• [ ] If a flag is removed, then it should also be removed from vitess-operator and arewefastyet, if used there.
• [ ] vtctl command output order should be stable and awk-able.

[codecov[bot]]

Codecov Report

Attention: Patch coverage is 41.32653% with 230 lines in your changes are missing coverage. Please review.

Project coverage is 65.61%. Comparing base (7a2af6f) to head (6ab42cc). Report is 9 commits behind head on main.

Files	Patch %	Lines
...cmd/vtctldclient/command/keyspace_routing_rules.go	13.33%	52 Missing :warning:
go/vt/vtctl/workflow/server.go	56.07%	47 Missing :warning:
go/vt/vtctl/workflow/traffic_switcher.go	0.00%	33 Missing :warning:
go/vt/vtctl/grpcvtctldserver/server.go	0.00%	23 Missing :warning:
...ldclient/command/vreplication/movetables/create.go	0.00%	18 Missing :warning:
go/vt/vtctl/workflow/utils.go	60.52%	15 Missing :warning:
go/vt/vtctl/workflow/materializer.go	58.33%	10 Missing :warning:
go/vt/vtgate/vindexes/vschema.go	41.17%	10 Missing :warning:
go/vt/vtctl/grpcvtctldclient/client_gen.go	0.00%	8 Missing :warning:
go/vt/topo/vschema.go	71.42%	6 Missing :warning:
... and 4 more

Additional details and impacted files

@@            Coverage Diff             @@
##             main   #15128      +/-   ##
==========================================
+ Coverage   65.44%   65.61%   +0.17%     
==========================================
  Files        1562     1563       +1     
  Lines      193922   194239     +317     
==========================================
+ Hits       126905   127448     +543     
+ Misses      67017    66791     -226     

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

[rohit-nayak-ps]

Replaced with https://github.com/vitessio/vitess/pull/15503