layotto
layotto copied to clipboard
mosn
feat: implement sequencer API with postgresql
Thanks! you can
make formatto fix the CI errors
对对 我知道，我现在可以自己跑ci了，之前那个pr我关了，重新提了一个
关于 CI 里的 linter 报错，你在github 上可以看到报错提示
这个报错说的是 空引用建议统一在 main里引用，如果不得不在其他代码里用空引用，需要在上面一行加注释。比如这样：
// I have to blank-import this package because xxxxxxx
_ "github.com/lib/pq"
Codecov Report
Merging #640 (ba4ad06) into main (e3d78d6) will decrease coverage by
0.68%. The diff coverage is26.71%.
:exclamation: Current head ba4ad06 differs from pull request most recent head 2ec75e5. Consider uploading reports for the commit 2ec75e5 to get more accurate results
@@ Coverage Diff @@
## main #640 +/- ##
==========================================
- Coverage 60.87% 60.19% -0.69%
==========================================
Files 120 122 +2
Lines 6388 6534 +146
==========================================
+ Hits 3889 3933 +44
- Misses 2125 2215 +90
- Partials 374 386 +12
| Impacted Files | Coverage Δ | |
|---|---|---|
| components/pkg/utils/postgresql.go | 0.00% <0.00%> (ø) |
|
| sdk/go-sdk/client/invoke.go | 78.66% <ø> (ø) |
|
| sdk/go-sdk/client/state.go | 75.49% <ø> (ø) |
|
| ...nents/sequencer/postgresql/postgresql_sequencer.go | 49.36% <49.36%> (ø) |
|
| components/lock/zookeeper/zookeeper_lock.go | 64.28% <0.00%> (+8.92%) |
:arrow_up: |
Continue to review full report at Codecov.
Legend - Click here to learn more
Δ = absolute <relative> (impact),ø = not affected,? = missing dataPowered by Codecov. Last update 9a31451...2ec75e5. Read the comment docs.
![codecov[bot] avatar](https://avatars.githubusercontent.com/in/254?v=4 "Published by a pub.dev verified publisher"){kind=small}
@ZLBer Hi, could u help review this PR? Since it's similar with https://github.com/mosn/layotto/pull/605
As for those weird empty commits, we can help revert them before merging this PR
action: 调研一下要不要接存储 调研一下时钟回拨问题
@ZLBer @seeflood
感觉好像避免不了接入外部存储，那可不可以这样，一次存储一个范围，
Thanks! I solve it later
| | 11 | | @.*** | 签名由网易邮箱大师定制
On 07/3/2022 @.***> wrote：
@ZLBer commented on this pull request.
In components/sequencer/postgresql/postgresql_sequencer.go:
+func NewPostgresqlSequencer(logger log.ErrorLogger) *PostgresqlSequencer {
- p := &PostgresqlSequencer{
-
logger: logger, - }
- return p +}
+func (p *PostgresqlSequencer) Init(config sequencer.Configuration) error {
- meta, err := utils.ParsePostgresqlMetaData(config.Properties)
- if err != nil {
-
p.logger.Errorf("init properties error: %v", err) -
return err - }
- p.metadata = meta
- p.db = utils.NewPostgresqlCli(meta)
NewPostgresqlCli may return nil
In components/sequencer/postgresql/postgresql_sequencer.go:
var model PostgresqlModel
-
queryParams := fmt.Sprintf(`select id, value_id, biz_tag, create_time, update_time from %s where biz_tag = $1`, p.metadata.TableName) -
err := p.db.QueryRow(queryParams, key).Scan(&model.ID, &model.ValueId, &model.BizTag, &model.CreateTime, &model.UpdateTime) -
if err != nil { -
p.logger.Errorf("get nextId error, biz_tag: %s, err: %v", key, err) -
return err -
} -
if model.ID < value { -
return fmt.Errorf("postgresql sequenccer error: can not satisfy biggerThan guarantee.key: %s,key in postgres: %s", key, p.metadata.TableName) -
} - }
- return nil +}
+func (p *PostgresqlSequencer) GetNextId(req *sequencer.GetNextIdRequest) (*sequencer.GetNextIdResponse, error) {
- lock.Lock()
lock doesn't work, layotto may distributed deployment We can solve it with optimistic lock, you can refer to mysql sequencer
In components/sequencer/postgresql/postgresql_sequencer.go:
- if rowsId == 0 {
-
return nil, errors.New("no update") - }
- if err != nil {
-
p.logger.Errorf("get nextId error, biz_tag: %s, err: %v", req.Key, err) -
return nil, err - }
- return &sequencer.GetNextIdResponse{
-
NextId: model.ValueId, - }, nil +}
+func (p *PostgresqlSequencer) GetSegment(req *sequencer.GetSegmentRequest) (bool, *sequencer.GetSegmentResponse, error) {
- lock.Lock()
same above
In components/sequencer/postgresql/postgresql_sequencer_test.go:
t.Errorf("mock db error: %v", err)
- }
- defer db.Close()
- rows := sqlmock.NewRows([]string{"id", "value_id", "biz_tag", "create_time", "update_time"}).AddRow([]driver.Value{1, 10, "test11", 111111, 111111}...)
- mock.ExpectQuery("select").WillReturnRows(rows)
- mock.ExpectExec("update").WillReturnResult(sqlmock.NewResult(1, 1))
- req := &sequencer.GetSegmentRequest{Size: 10, Key: p.metadata.BizTag, Options: sequencer.SequencerOptions{AutoIncrement: sequencer.STRONG}, Metadata: initMap()}
- p.db = db
- _, _, err = p.GetSegment(req)
- assert.NoError(t, err) +}
+// +//func TestLocalNextId(t *testing.T) {
Remove not work code
— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you modified the open/close state.Message ID: @.***>
This pull request has been automatically marked as stale because it has not had activity in the last 30 days. It will be closed in 7 days if no further activity occurs. Please feel free to give a status update now, ping for review, or re-open when it's ready. Thank you for your contributions!
![github-actions[bot] avatar](https://avatars.githubusercontent.com/in/15368?v=4 "Published by a pub.dev verified publisher"){kind=small}
@azhsmesos Hi, are u still working on it? Could u resolve the review issues ?
This pull request has been automatically marked as stale because it has not had activity in the last 30 days. It will be closed in 7 days if no further activity occurs. Please feel free to give a status update now, ping for review, or re-open when it's ready. Thank you for your contributions!
This pull request has been automatically closed because it has not had activity in the last 37 days. Please feel free to give a status update now, ping for review, or re-open when it's ready. Thank you for your contributions!