apisix icon indicating copy to clipboard operation
apisix copied to clipboard

Apisix with multiple K8s clusters as a source for service discovery

Open afbm08 opened this issue 3 years ago • 8 comments

Description

As a user, I need to use multiple K8s clusters as a source for service discovery.

In forum, Zeping Bai told me that at this time it's not possible to create a multiple similar service discovery.

It's possible develop that feature?

afbm08 avatar Jul 29 '22 10:07 afbm08

I plan to complete this feature, but The configuration form may be worth discussing.

1: Use a new discovery name such as multi-kubernetes, but only supports multi-cluster

2: The current kubernetes discovery supports both single and multiple clusters, but the compatibility cost may be high

3: Bread update kubernetes discovery, supports both single and multi-cluster

Maybe someone can suggest a new direction

@afbm08 @tao12345666333 @tokers @tzssangglass @spacewander @liangyawang

zhixiongdu027 avatar Jul 30 '22 11:07 zhixiongdu027

Maybe it's not just the needs of a specific module

tao12345666333 avatar Jul 30 '22 12:07 tao12345666333

Well, I don't see we need to add another service discovery module as its core capability is same to the existing one.

tokers avatar Jul 31 '22 11:07 tokers

As for the compatibility, just adding another field: clusters, which is an array is enough. One can configure it via configuring a bunch of kubernetes clusters via the clusters array or using the existing one. Sub fields in the clusters are same to the flatten fields.

tokers avatar Jul 31 '22 11:07 tokers

As for the compatibility, just adding another field: clusters, which is an array is enough. One can configure it via configuring a bunch of kubernetes clusters via the clusters array or using the existing one. Sub fields in the clusters are same to the flatten fields.

+1

spacewander avatar Jul 31 '22 12:07 spacewander

Okay, Let's move on

zhixiongdu027 avatar Jul 31 '22 14:07 zhixiongdu027

Hello, every one:

This work is continuing, and I hope you can continue to discuss the configuration issue

At present, the demo I have completed supports the configuration in the following two modes:

Singleton Mode

compatible with current configuration, except for add shared_size field

config.yaml :

kubernets:
    schema:  
      ....   
    services:
      ....   
    client:
      ....  
    shared_size: 1m 

and rendered out nginx.conf as follows

nginx.conf:

http {

  ...
  
  lua_shared_dict kubernetes 1m;

  ....
}

Multiple Mode

configuration same as the singleton pattern except that they under an arrays

config.yaml

kubernets:
  - services: ...
    client: ...
    id: "release"
    shared_size: 2m
  - services: ...
    client: ...
    id: "debug"
    shared_size: 1m

the biggest difference from a singleton is that multiple lua_shared_dicts are rendered

nginx.conf

http {
  
  ...

  lua_shared_dict kubernetes-release 2m;

  lua_shared_dict kubernetes-debug 1m;

  ...
}

I think the configuration of these two mode is very easy to understand and has the same design as some another of the APISix configuration

If everyone thinks it's okay, I'll start the next step.

@spacewander @tokers @tzssangglass @afbm08 @liangyawang

zhixiongdu027 avatar Aug 29 '22 13:08 zhixiongdu027

LGTM

spacewander avatar Aug 30 '22 07:08 spacewander

fixed by https://github.com/apache/apisix/pull/7895

moonming avatar Jul 26 '23 03:07 moonming