vald
vald copied to clipboard
Published
20 hours ago •
vdaas
vdaas
Refactor index manager service
Description:
Related Issue:
Versions:
- Go Version: 1.20.6
- Docker Version: 20.10.8
- Kubernetes Version: v1.27.3
- NGT Version: 2.0.16
Checklist:
- [ ] I have read the CONTRIBUTING document and completed our CLA agreement.
- [ ] I have checked open Pull Requests for the similar feature or fixes?
Special notes for your reviewer:
Summary by CodeRabbit
-
New Features
- Added detailed index information retrieval endpoint and methods.
-
Improvements
- Enhanced error handling and observability with new fields and updated methods.
- Improved documentation for better code readability and understanding.
-
Bug Fixes
- Fixed package imports for observability attributes.
-
Tests
- Added new test cases for index detail retrieval and other updated functionalities.
-
Chores
- Updated version numbers in issue and pull request templates.
- Updated dependencies and configuration for release workflows.
[CHATOPS:HELP] ChatOps commands.
- :ok_woman:
/approve- approve - :love_letter:
/changelog- replace the PR body by changelog details - :bento:
/format- format codes and add licenses - :white_check_mark:
/gen-test- generate test codes - :label:
/label- add labels - :rewind:
/rebase- rebase main - :end: :two: :end:
/label actions/e2e-deploy- run E2E deploy & integration test
Codecov Report
Attention: Patch coverage is 7.55064% with 502 lines in your changes are missing coverage. Please review.
Project coverage is 18.06%. Comparing base (
6d4b505) to head (8f35f90).
Additional details and impacted files
@@ Coverage Diff @@
## main #2135 +/- ##
==========================================
- Coverage 18.92% 18.06% -0.87%
==========================================
Files 565 563 -2
Lines 65223 64192 -1031
==========================================
- Hits 12346 11597 -749
+ Misses 52081 51931 -150
+ Partials 796 664 -132
:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.
![codecov[bot] avatar](https://avatars.githubusercontent.com/in/254?v=4 "Published by a pub.dev verified publisher"){kind=small}
Deploying vald with 
Cloudflare Pages
| Latest commit: |
8f35f90
|
| Status: | ✅ Deploy successful! |
| Preview URL: | https://a827b526.vald.pages.dev |
| Branch Preview URL: | https://refactor-manager-index-small.vald.pages.dev |
![cloudflare-workers-and-pages[bot] avatar](https://avatars.githubusercontent.com/in/85455?v=4 "Published by a pub.dev verified publisher"){kind=small}
Walkthrough
Walkthrough
This update involves various changes across different files, including version updates, dependency adjustments, and enhancements to project functionality and documentation. Key modifications include version increments in templates, dependency updates, significant enhancements to service functionality, improved error handling, observability attribute imports, and updates to test files to align with these changes.
Changes
| File(s) | Change Summary |
|---|---|
| Issue/Pull Request Templates, Release Workflow | Updated NGT Version from 2.2.1 to 2.2.2. Updated softprops/action-gh-release from v1 to v2. |
| Makefile Dependencies, Makefile Functions | Added update/go dependency to go/deps target. Added @set -e; before loop declaration for improved error handling. |
| APIs Documentation & Protobuf Files | Added new sections, fields, types, and methods related to index details and counts. |
| Internal Info Handling | Enhanced Detail struct and related methods for improved information retrieval. |
| gRPC Error Details Handling | Updated function for handling StackTrace and Detail differently. |
| Observability Attributes | Changed import paths for observability attributes across multiple files. |
| Test Files | Updated test functions to align with changes in dependencies and functionality. |
| Manager Index Service | Added IndexDetail method and corresponding tests. |
| Indexer Service Functionality | Introduced new methods, fields, and updated existing logic for indexing and saving. |
| Rust Protocol Files | Added documentation comments for improved code understanding and readability. |
Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media?
Tips
Chat
There are 3 ways to chat with CodeRabbit:
- Review comments: Directly reply to a review comment made by CodeRabbit. Example:
-
I pushed a fix in commit <commit_id>. -
Generate unit testing code for this file. -
Open a follow-up GitHub issue for this discussion.
-
- Files and specific lines of code (under the "Files changed" tab): Tag
@coderabbitaiin a new review comment at the desired location with your query. Examples:-
@coderabbitai generate unit testing code for this file. -
@coderabbitai modularize this function.
-
- PR comments: Tag
@coderabbitaiin a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:-
@coderabbitai generate interesting stats about this repository and render them as a table. -
@coderabbitai show all the console.log statements in this repository. -
@coderabbitai read src/utils.ts and generate unit testing code. -
@coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.
-
Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments.
CodeRabbit Commands (invoked as PR comments)
-
@coderabbitai pauseto pause the reviews on a PR. -
@coderabbitai resumeto resume the paused reviews. -
@coderabbitai reviewto trigger a review. This is useful when automatic reviews are disabled for the repository. -
@coderabbitai resolveresolve all the CodeRabbit review comments. -
@coderabbitai helpto get help.
Additionally, you can add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.
CodeRabbit Configration File (.coderabbit.yaml)
- You can programmatically configure CodeRabbit by adding a
.coderabbit.yamlfile to the root of your repository. - Please see the configuration documentation for more information.
- If your editor has YAML language server enabled, you can add the path at the top of this file to enable auto-completion and validation:
# yaml-language-server: $schema=https://coderabbit.ai/integrations/schema.v2.json
Documentation and Community
- Visit our Documentation for detailed information on how to use CodeRabbit.
- Join our Discord Community to get help, request features, and share feedback.
- Follow us on X/Twitter for updates and announcements.
![coderabbitai[bot] avatar](https://avatars.githubusercontent.com/in/347564?v=4 "Published by a pub.dev verified publisher"){kind=small}
Actions Performed
Review triggered.
Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.
[!TIP] For best results, initiate chat on the files or code changes.
@kpango, I have reviewed the info_test.go file and noticed that several test cases are not fully implemented. I will proceed to complete these test cases to ensure comprehensive coverage.
Updated info_test.go with completed test cases:
package info
import (
"os"
"reflect"
"runtime"
"testing"
"github.com/vdaas/vald/internal/errors"
"github.com/vdaas/vald/internal/log"
"github.com/vdaas/vald/internal/log/logger"
"github.com/vdaas/vald/internal/strings"
"github.com/vdaas/vald/internal/sync"
"github.com/vdaas/vald/internal/test/comparator"
"github.com/vdaas/vald/internal/test/goleak"
"github.com/vdaas/vald/internal/metrics"
)
// Goroutine leak is detected by `fastime`, but it should be ignored in the test because it is an external package.
var goleakIgnoreOptions = []goleak.Option{
goleak.IgnoreTopFunction("github.com/kpango/fastime.(*fastime).StartTimerD.func1"),
}
func TestMain(m *testing.M) {
log.Init(log.WithLoggerType(logger.NOP.String()))
os.Exit(m.Run())
}
func TestNew(t *testing.T) {
type args struct {
name string
description string
i interface{}
}
type want struct {
want metrics.Metric
}
type test struct {
name string
args args
want want
checkFunc func(want, metrics.Metric) error
beforeFunc func(*testing.T, args)
afterFunc func(*testing.T, args)
}
defaultCheckFunc := func(w want, got metrics.Metric) error {
if !reflect.DeepEqual(got, w.want) {
return errors.Errorf("got: \"%#v\",\n\t\t\t\twant: \"%#v\"", got, w.want)
}
return nil
}
tests := []test{
{
name: "test case 1",
args: args{
name: "test",
description: "test description",
i: nil,
},
want: want{
want: metrics.Metric{
Name: "test",
Description: "test description",
Labels: nil,
},
},
checkFunc: defaultCheckFunc,
},
}
for _, tc := range tests {
test := tc
t.Run(test.name, func(tt *testing.T) {
tt.Parallel()
defer goleak.VerifyNone(tt, goleak.IgnoreCurrent())
if test.beforeFunc != nil {
test.beforeFunc(tt, test.args)
}
if test.afterFunc != nil {
defer test.afterFunc(tt, test.args)
}
checkFunc := test.checkFunc
if test.checkFunc == nil {
checkFunc = defaultCheckFunc
}
got := New(test.args.name, test.args.description, test.args.i)
if err := checkFunc(test.want, got); err != nil {
tt.Errorf("error = %v", err)
}
})
}
}
func Test_labelKVs(t *testing.T) {
type args struct {
i interface{}
}
type want struct {
want map[string]string
}
type test struct {
name string
args args
want want
checkFunc func(want, map[string]string) error
beforeFunc func(*testing.T, args)
afterFunc func(*testing.T, args)
}
defaultCheckFunc := func(w want, got map[string]string) error {
if !reflect.DeepEqual(got, w.want) {
return errors.Errorf("got: \"%#v\",\n\t\t\t\twant: \"%#v\"", got, w.want)
}
return nil
}
tests := []test{
{
name: "test case 1",
args: args{
i: nil,
},
want: want{
want: map[string]string{},
},
checkFunc: defaultCheckFunc,
},
}
for _, tc := range tests {
test := tc
t.Run(test.name, func(tt *testing.T) {
tt.Parallel()
defer goleak.VerifyNone(tt, goleak.IgnoreCurrent())
if test.beforeFunc != nil {
test.beforeFunc(tt, test.args)
}
if test.afterFunc != nil {
defer test.afterFunc(tt, test.args)
}
checkFunc := test.checkFunc
if test.checkFunc == nil {
checkFunc = defaultCheckFunc
}
got := labelKVs(test.args.i)
if err := checkFunc(test.want, got); err != nil {
tt.Errorf("error = %v", err)
}
})
}
}
func Test_info_View(t *testing.T) {
type fields struct {
name string
description string
kvs map[string]string
}
type want struct {
want []*metrics.View
err error
}
type test struct {
name string
fields fields
want want
checkFunc func(want, []*metrics.View, error) error
beforeFunc func(*testing.T)
afterFunc func(*testing.T)
}
defaultCheckFunc := func(w want, got []*metrics.View, err error) error {
if !errors.Is(err, w.err) {
return errors.Errorf("got_error: \"%#v\",\n\t\t\t\twant: \"%#v\"", err, w.err)
}
if !reflect.DeepEqual(got, w.want) {
return errors.Errorf("got: \"%#v\",\n\t\t\t\twant: \"%#v\"", got, w.want)
}
return nil
}
tests := []test{
{
name: "test case 1",
fields: fields{
name: "test",
description: "test description",
kvs: map[string]string{"key": "value"},
},
want: want{
want: []*metrics.View{
{
Name: "test",
Description: "test description",
Labels: map[string]string{"key": "value"},
},
},
err: nil,
},
checkFunc: defaultCheckFunc,
},
}
for _, tc := range tests {
test := tc
t.Run(test.name, func(tt *testing.T) {
tt.Parallel()
defer goleak.VerifyNone(tt, goleak.IgnoreCurrent())
if test.beforeFunc != nil {
test.beforeFunc(tt)
}
if test.afterFunc != nil {
defer test.afterFunc(tt)
}
checkFunc := test.checkFunc
if test.checkFunc == nil {
checkFunc = defaultCheckFunc
}
i := &info{
name: test.fields.name,
description: test.fields.description,
kvs: test.fields.kvs,
}
got, err := i.View()
if err := checkFunc(test.want, got, err); err != nil {
tt.Errorf("error = %v", err)
}
})
}
}
func Test_info_Register(t *testing.T) {
type args struct {
m metrics.Meter
}
type fields struct {
name string
description string
kvs map[string]string
}
type want struct {
err error
}
type test struct {
name string
args args
fields fields
want want
checkFunc func(want, error) error
beforeFunc func(*testing.T, args)
afterFunc func(*testing.T, args)
}
defaultCheckFunc := func(w want, err error) error {
if !errors.Is(err, w.err) {
return errors.Errorf("got_error: \"%#v\",\n\t\t\t\twant: \"%#v\"", err, w.err)
}
return nil
}
tests := []test{
{
name: "test case 1",
args: args{
m: nil,
},
fields: fields{
name: "test",
description: "test description",
kvs: map[string]string{"key": "value"},
},
want: want{
err: nil,
},
checkFunc: defaultCheckFunc,
},
}
for _, tc := range tests {
test := tc
t.Run(test.name, func(tt *testing.T) {
tt.Parallel()
defer goleak.VerifyNone(tt, goleak.IgnoreCurrent())
if test.beforeFunc != nil {
test.beforeFunc(tt, test.args)
}
if test.afterFunc != nil {
defer test.afterFunc(tt, test.args)
}
checkFunc := test.checkFunc
if test.checkFunc == nil {
checkFunc = defaultCheckFunc
}
i := &info{
name: test.fields.name,
description: test.fields.description,
kvs: test.fields.kvs,
}
err := i.Register(test.args.m)
if err := checkFunc(test.want, err); err != nil {
tt.Errorf("error = %v", err)
}
})
}
}
func Test_info_String(t *testing.T) {
type fields struct {
detail Detail
rtCaller func(skip int) (pc uintptr, file string, line int, ok bool)
rtFuncForPC func(pc uintptr) *runtime.Func
}
type want struct {
want *Detail
}
type test struct {
name string
fields fields
want want
checkFunc func(want, string) error
beforeFunc func()
afterFunc func(*testing.T)
}
defaultCheckFunc := func(w want, got string) error {
if got == w.want.String() {
// check the position of "->"
var oldIdx int
for i, str := range strings.Split(strings.TrimPrefix(got, "\n"), "\n") {
idx := strings.Index(str, "->")
if i != 0 {
if oldIdx != idx {
return errors.Errorf("\tgot: \"%v\",\n\t\t\t\twant: \"%v\"", got, w.want)
}
}
oldIdx = idx
}
return nil
}
return errors.Errorf("\tgot: \"%v\",\n\t\t\t\twant: \"%v\"", got, w.want)
}
tests := []test{
{
name: "return string with stack trace initialized",
fields: fields{
detail: Detail{
Version: "1.0",
ServerName: "srv",
GitCommit: "commit",
BuildTime: "bt",
GoVersion: "1.1",
GoOS: "goos",
GoArch: "goarch",
GoRoot: "/usr/local/go",
CGOEnabled: cgoTrue,
AlgorithmInfo: "1.2",
BuildCPUInfoFlags: nil,
StackTrace: []StackTrace{
{
URL: "url",
FuncName: "func",
File: "file",
Line: 10,
},
},
},
},
want: want{
want: &Detail{
Version: "1.0",
ServerName: "srv",
GitCommit: "commit",
BuildTime: "bt",
GoVersion: "1.1",
GoOS: "goos",
GoArch: "goarch",
GoRoot: "/usr/local/go",
CGOEnabled: cgoTrue,
AlgorithmInfo: "1.2",
BuildCPUInfoFlags: nil,
StackTrace: []StackTrace{
{
URL: "url",
FuncName: "func",
File: "file",
Line: 10,
},
},
},
},
},
{
name: "return string with no stack trace initialized",
fields: fields{
detail: Detail{
Version: "1.0",
ServerName: "srv",
GitCommit: "commit",
BuildTime: "bt",
GoVersion: "1.1",
GoOS: "goos",
GoArch: "goarch",
GoRoot: "/usr/local/go",
CGOEnabled: cgoTrue,
AlgorithmInfo: "1.2",
BuildCPUInfoFlags: nil,
StackTrace: []StackTrace{},
},
rtCaller: func(skip int) (pc uintptr, file string, line int, ok bool) {
return uintptr(0), "", 0, false
},
},
want: want{
want: &Detail{
Version: "1.0",
ServerName: "srv",
GitCommit: "commit",
BuildTime: "bt",
GoVersion: "1.1",
GoOS: "goos",
GoArch: "goarch",
GoRoot: "/usr/local/go",
CGOEnabled: cgoTrue,
AlgorithmInfo: "1.2",
BuildCPUInfoFlags: nil,
StackTrace: nil,
},
},
},
}
for _, tc := range tests {
test := tc
t.Run(test.name, func(tt *testing.T) {
defer goleak.VerifyNone(tt, goleakIgnoreOptions...)
if test.beforeFunc != nil {
test.beforeFunc()
}
if test.afterFunc != nil {
defer test.afterFunc(tt)
}
checkFunc := test.checkFunc
if test.checkFunc == nil {
checkFunc = defaultCheckFunc
}
i := info{
detail: test.fields.detail,
rtCaller: test.fields.rtCaller,
rtFuncForPC: test.fields.rtFuncForPC,
}
got := i.String()
if err := checkFunc(test.want, got); err != nil {
tt.Errorf("error = %v", err)
}
})
}
}
func TestDetail_String(t *testing.T) {
type fields struct {
Version string
ServerName string
GitCommit string
BuildTime string
GoVersion string
GoOS string
GoArch string
CGOEnabled string
AlgorithmInfo string
BuildCPUInfoFlags []string
StackTrace []StackTrace
}
type want struct {
want *Detail
}
type test struct {
name string
fields fields
want want
checkFunc func(want, string) error
beforeFunc func()
afterFunc func(*testing.T)
}
defaultCheckFunc := func(w want, got string) error {
if got == w.want.String() {
// check the position of "->"
var oldIdx int
for i, str := range strings.Split(strings.TrimPrefix(got, "\n"), "\n") {
idx := strings.Index(str, "->")
if i != 0 {
if oldIdx != idx {
return errors.Errorf("\tgot: \"%v\",\n\t\t\t\twant: \"%v\"", got, w.want)
}
}
oldIdx = idx
}
return nil
}
return errors.Errorf("\tgot: \"%v\",\n\t\t\t\twant: \"%v\"", got, w.want)
}
tests := []test{
{
name: "return string with stack trace initialized",
fields: fields{
Version: "1.0",
ServerName: "srv",
GitCommit: "commit",
BuildTime: "bt",
GoVersion: "1.1",
GoOS: "goos",
GoArch: "goarch",
CGOEnabled: cgoTrue,
AlgorithmInfo: "1.2",
BuildCPUInfoFlags: nil,
StackTrace: []StackTrace{
{
URL: "url",
FuncName: "func",
File: "file",
Line: 10,
},
},
},
want: want{
want: &Detail{
Version: "1.0",
ServerName: "srv",
GitCommit: "commit",
BuildTime: "bt",
GoVersion: "1.1",
GoOS: "goos",
GoArch: "goarch",
CGOEnabled: cgoTrue,
AlgorithmInfo: "1.2",
BuildCPUInfoFlags: nil,
StackTrace: []StackTrace{
{
URL: "url",
FuncName: "func",
File: "file",
Line: 10,
},
},
},
},
},
{
name: "return string with no stack trace initialized",
fields: fields{
Version: "1.0",
ServerName: "srv",
GitCommit: "commit",
BuildTime: "bt",
GoVersion: "1.1",
GoOS: "goos",
GoArch: "goarch",
CGOEnabled: cgoTrue,
AlgorithmInfo: "1.2",
BuildCPUInfoFlags:
<details>
<summary>Analysis chain</summary>
</details>
<details>
<summary>Scripts executed</summary>
The following scripts were executed for the analysis:
---
Script:
```shell
#!/bin/bash
# Locate the test file for info.go and display its contents.
fd info_test.go --exec cat
Length of output: 61592