driver-go icon indicating copy to clipboard operation
driver-go copied to clipboard

af接口锁释放问题

Open duyingsoft opened this issue 1 year ago • 4 comments

使用af的stmt接口插入数据,发送如果BindParam报错,一定次数调用失败后,会卡住 看af源码的话(https://github.com/taosdata/driver-go/blob/3.0/af/insertstmt/stmt.go)

发现这边加锁和释放锁的写法是不是有问题,不仅仅是下面这个方法,好像整个文件的锁的使用都有问题

func (stmt *InsertStmt) BindParam(params []*param.Param, bindType *param.ColumnType) error {
	data := make([][]driver.Value, len(params))
	for columnIndex, columnData := range params {
		value := columnData.GetValues()
		data[columnIndex] = value
	}
	columnTypes, err := bindType.GetValue()
	if err != nil {
		return err
	}
	locker.Lock()
	code := wrapper.TaosStmtBindParamBatch(stmt.stmt, data, columnTypes)
	locker.Unlock()
	if code != 0 {
		errStr := wrapper.TaosStmtErrStr(stmt.stmt)
		return taosError.NewError(code, errStr)
	}
	return nil
}

可以看到,如果这一行code := wrapper.TaosStmtBindParamBatch(stmt.stmt, data, columnTypes)异常了,好像是panic, 这样锁其实是没有释放,这边锁的写法是不是有问题 一般正常使用应该是lock()完,立马defer Unlock,再去执行逻辑,除非能保证执行的代码不会panic

duyingsoft avatar Mar 23 '23 11:03 duyingsoft

@duyingsoft TaosStmtBindParamBatch 这个接口期望是返回 error code 而不是 panic,此接口内包含 cgo 实现,如果 C 代码崩溃使用 recover 也无法捕获,即使使用 defer unlock 也不会执行。请提供 panic 信息和简单复现代码我们分析一下

huskar-t avatar Mar 23 '23 12:03 huskar-t

但是我defer是能捕获到panic,理论上,应该也是能defer unlock的吧@huskar-t

(1) log_repository_test.go是我的测试代码,这是捕获到panic信息,你可以看一下

Fail to InsertLog, error:interface conversion: driver.Value is types.TaosNchar, not types.TaosBinary
log_repository_test.go:142 (0x7ff603648e49)
C:/Program Files/Go/src/runtime/panic.go:1038 (0x7ff6034f9994)
C:/Program Files/Go/src/runtime/iface.go:261 (0x7ff6034cbd54)

/go/pkg/mod/github.com/taosdata/driver-go/[email protected]/wrapper/stmt.go:513 (0x7ff603643e5d)
/go/pkg/mod/github.com/taosdata/driver-go/[email protected]/af/insertstmt/stmt.go:95 (0x7ff60364621c)
/log_repository_test.go:172 (0x7ff603648a0a)
/log_repository_test.go:134 (0x7ff603648534)
C:/Program Files/Go/src/testing/testing.go:1259 (0x7ff603594b01)
C:/Program Files/Go/src/runtime/asm_amd64.s:1581 (0x7ff60352a320)

(2)简单测试代码 只要for循环的数量大于cpu核数,执行的循环次数大于cpu核数,就会卡住执行不了,我看代码逻辑,lock最大并发数是本地cpu核数,应该是锁没有释放

InsertLog()是故意将params和paramTypes 设置成不一致,stmt.BindParam就会panic

func TestMultiLogs(t *testing.T) {
	fmt.Println(runtime.NumCPU())
	for i := 0; i < 20; i++ {
		fmt.Printf("TestMultiLogs:%d\n", i)
		InsertLog()
	}
}

func InsertLog() error {
	defer func() {
		if panicErr := recover(); panicErr != nil {
			fmt.Printf("Fail to InsertLog, error:%v\n", errorUtil.PrintStackTrace(panicErr))
		}
	}()
	conn, err := af.Open("xxx", "root", "taosdata", "ddd", 6030)
	if err != nil {
		fmt.Printf("Fail to open connection, err:%v\n", err)
	}
	defer conn.Close()
	// create stmt
	stmt := conn.InsertStmt()
	defer stmt.Close()
	err = stmt.Prepare("insert into log_table values(?,?,?)")
	if err != nil {
		fmt.Printf("Fail to Prepare, err:%v\n", err)
		return err
	}
	params := make([]*param.Param, 3)
	params[0] = param.NewParam(1).AddTimestamp(time.Now(), common.PrecisionMilliSecond)
	params[1] = param.NewParam(1).AddNchar("nchar")
	params[2] = param.NewParam(1).AddNchar("nchar")

	paramTypes := param.NewColumnType(14).
		AddTimestamp().
		AddNchar(256).
		AddBinary(256)

	err = stmt.BindParam(params,paramTypes)
	if err != nil {
		fmt.Printf("Fail to BindParam, err:%v\n", err)
		return err
	}
	err = stmt.AddBatch()
	if err != nil {
		fmt.Printf("Fail to AddBatch, err:%v\n", err)
		return err
	}
	err = stmt.Execute()
	if err != nil {
		fmt.Printf("Fail to Execute insert batch, err:%v\n", err)
		return err
	}
	return nil
}

duyingsoft avatar Mar 23 '23 13:03 duyingsoft

这种情况属于参数错误,需要调用者保证传入参数正确

huskar-t avatar Mar 23 '23 13:03 huskar-t

因为你这个锁是全局的,如果某个新增的功能不小心写错了,很容易导致程序卡死,导致其他功能写入tdengine也阻塞住,影响比较大。 建议是不是尽量defer释放锁,控制影响范围

duyingsoft avatar Mar 23 '23 13:03 duyingsoft