kitex
kitex copied to clipboard
cloudwego
Feature Proposal: optionloader for Kitex client/server
Currently Kitex users have to manually intialize a client/server with options, for example:
cli, err := somService.NewClient(svcName, client.WithHostPorts(addr))
If there's a need to change the options, one has to modify the code and recompile it, which is no doubt tedious work.
In this proposal, we suggest implementing a library optionloader for loading options from certain source (e.g. some local yaml file), which is capable of loading frequently used options out-of-the-box, and is also extensible by allowing users to register config-to-option translators without modify the library itself.
With this library, users will be able to initialize a client (or server) this way:
loader := optionloader.NewClientOptionLoader()
loader.RegisterTranslator("timeout", func (config map[string]interface{}) client.Option {
// custom implementation
})
reader := optionloader.NewFileReader(someFile)
loader.SetSource(reader)
options, err := optionloader.Load()
cli, err := somService.NewClient(svcName, options...)
Note: the above is just pseudo code to show the core concept, and is not strict stardard for final design. For example, some user may prefer a default value (other than the default value in Kitex) when a config item does not appear in the yaml file.
To make this library handier, it's preferably better to support more config sources like etcd/consul, and also the translation for callopt.Option, streamclient.Option and streamcall.Option.
This library shall be released under https://github.com/kitex-contrib .
If you are interested in implementing this feature, please kindly prepare a detailed tech plan and reply with your lark id for us.
@felix021 When implementing an options loader, we can follow these steps:
Define the core interface:
-
- Define the core interface of the option loader, including registering the translator, setting the configuration source, loading options, etc.
-
- Implement the option loader: Implement the option loader to load options according to the registered translator and configuration source.
-
- Register configurations and translators: Allows users to register configurations and custom translators to map configurations to Kitex options.
-
- Support different configuration sources: implement different configuration sources, such as file readers, etcd readers, etc.
The following is the sample code
package optionloader
import (
"io/ioutil"
"gopkg.in/yaml.v2"
"github.com/kitex-contrib/kitex-client/client"
)
// 选项加载器的接口
type OptionLoader interface {
RegisterTranslator(name string, translator Translator)
SetSource(source ConfigSource)
Load() ([]client.Option, error)
}
// 翻译器的接口
type Translator func(config map[string]interface{}) client.Option
// 配置源的接口
type ConfigSource interface {
ReadConfig() (map[string]interface{}, error)
}
// 文件配置源的实现
type FileConfigSource struct {
Filepath string
}
// 从文件中读取配置
func (f *FileConfigSource) ReadConfig() (map[string]interface{}, error) {
data, err := ioutil.ReadFile(f.Filepath)
if err != nil {
return nil, err
}
var config map[string]interface{}
err = yaml.Unmarshal(data, &config)
if err != nil {
return nil, err
}
return config, nil
}
// OptionLoader 接口的实现
type OptionLoaderImpl struct {
translators map[string]Translator
source ConfigSource
}
// 创建一个新的选项加载器
func NewOptionLoader() OptionLoader {
return &OptionLoaderImpl{
translators: make(map[string]Translator),
}
}
// 注册翻译器
func (loader *OptionLoaderImpl) RegisterTranslator(name string, translator Translator) {
loader.translators[name] = translator
}
// 设置配置源
func (loader *OptionLoaderImpl) SetSource(source ConfigSource) {
loader.source = source
}
// 加载选项
func (loader *OptionLoaderImpl) Load() ([]client.Option, error) {
config, err := loader.source.ReadConfig()
if err != nil {
return nil, err
}
var options []client.Option
for key, value := range config {
translator, ok := loader.translators[key]
if !ok {
continue
}
option := translator(value.(map[string]interface{}))
options = append(options, option)
}
return options, nil
}
// 创建一个新的客户端选项加载器
func NewClientOptionLoader() ClientOptionLoader {
return &OptionLoaderImpl{
translators: make(map[string]Translator),
}
}
func NewClientOptionLoader(translators map[string]Translator) OptionLoader {
loader := NewOptionLoader()
for name, translator := range translators {
loader.RegisterTranslator(name, translator)
}
return loader
}
// 创建一个新的文件读取器
func NewFileReader(filepath string) ConfigSource {
return &FileConfigSource{
Filepath: filepath,
}
}
func main(){
//方式1
loader := NewClientOptionLoader()
// 注册 timeout 翻译器
loader.RegisterTranslator("timeout", func(config map[string]interface{}) client.Option {
timeout := config["timeout"].(int) // 假设 timeout 是一个整数
return client.WithTimeout(timeout)
});
//方式2
translators := map[string]Translator{
"timeout": func(config map[string]interface{}) client.Option {
timeout := config["timeout"].(int) // 假设 timeout 是一个整数
return client.WithTimeout(timeout)
},
// 可以注册更多的选项翻译器...
}
loader := NewClientOptionLoader(translators)
}
The above is just a simple interface abstraction. Is there anything else you need to add?
@felix021 When implementing an options loader, we can follow these steps:
Define the core interface:
- Define the core interface of the option loader, including registering the translator, setting the configuration source, loading options, etc.
- Implement the option loader: Implement the option loader to load options according to the registered translator and configuration source.
- Register configurations and translators: Allows users to register configurations and custom translators to map configurations to Kitex options.
- Support different configuration sources: implement different configuration sources, such as file readers, etcd readers, etc.
The following is the sample code
package optionloader import ( "io/ioutil" "gopkg.in/yaml.v2" "github.com/kitex-contrib/kitex-client/client" ) // 选项加载器的接口 type OptionLoader interface { RegisterTranslator(name string, translator Translator) SetSource(source ConfigSource) Load() ([]client.Option, error) } // 翻译器的接口 type Translator func(config map[string]interface{}) client.Option // 配置源的接口 type ConfigSource interface { ReadConfig() (map[string]interface{}, error) } // 文件配置源的实现 type FileConfigSource struct { Filepath string } // 从文件中读取配置 func (f *FileConfigSource) ReadConfig() (map[string]interface{}, error) { data, err := ioutil.ReadFile(f.Filepath) if err != nil { return nil, err } var config map[string]interface{} err = yaml.Unmarshal(data, &config) if err != nil { return nil, err } return config, nil } // OptionLoader 接口的实现 type OptionLoaderImpl struct { translators map[string]Translator source ConfigSource } // 创建一个新的选项加载器 func NewOptionLoader() OptionLoader { return &OptionLoaderImpl{ translators: make(map[string]Translator), } } // 注册翻译器 func (loader *OptionLoaderImpl) RegisterTranslator(name string, translator Translator) { loader.translators[name] = translator } // 设置配置源 func (loader *OptionLoaderImpl) SetSource(source ConfigSource) { loader.source = source } // 加载选项 func (loader *OptionLoaderImpl) Load() ([]client.Option, error) { config, err := loader.source.ReadConfig() if err != nil { return nil, err } var options []client.Option for key, value := range config { translator, ok := loader.translators[key] if !ok { continue } option := translator(value.(map[string]interface{})) options = append(options, option) } return options, nil } // 创建一个新的客户端选项加载器 func NewClientOptionLoader() ClientOptionLoader { return &OptionLoaderImpl{ translators: make(map[string]Translator), } } func NewClientOptionLoader(translators map[string]Translator) OptionLoader { loader := NewOptionLoader() for name, translator := range translators { loader.RegisterTranslator(name, translator) } return loader } // 创建一个新的文件读取器 func NewFileReader(filepath string) ConfigSource { return &FileConfigSource{ Filepath: filepath, } } func main(){ //方式1 loader := NewClientOptionLoader() // 注册 timeout 翻译器 loader.RegisterTranslator("timeout", func(config map[string]interface{}) client.Option { timeout := config["timeout"].(int) // 假设 timeout 是一个整数 return client.WithTimeout(timeout) }); //方式2 translators := map[string]Translator{ "timeout": func(config map[string]interface{}) client.Option { timeout := config["timeout"].(int) // 假设 timeout 是一个整数 return client.WithTimeout(timeout) }, // 可以注册更多的选项翻译器... } loader := NewClientOptionLoader(translators) }The above is just a simple interface abstraction. Is there anything else you need to add?
Sorry for the late reply. This proposal is already claimed by @Printemps417 and I forgot to update the asignee.
I have been following this issue for some time. How is it going? May I join it?
@BaiZe1998 The previous student's implementation has merit, but still needs more complete details. We can discuss it together.
I have been following this issue for some time. How is it going? May I join it?
Hi, we have completed most implementations of the optionloader for yml, etcd, and consul, which is being developed in this repository: https://github.com/Printemps417/optionloader. I'm glad we can exchange ideas and learn from each other.