go-zero icon indicating copy to clipboard operation
go-zero copied to clipboard

About the bug of [func Parse(r *http.Request, v interface{}) error]

Open george012 opened this issue 3 years ago • 10 comments

func Parse(r *http.Request, v interface{}) error {
	if err := ParsePath(r, v); err != nil {
		return err
	}

	if err := ParseForm(r, v); err != nil {
		return err
	}

	if err := ParseHeaders(r, v); err != nil {
		return err
	}

	return ParseJsonBody(r, v)
}

上述方法中 追溯 [ return ParseJsonBody(r, v) ] 方法进入 [ rest.httpx.requests.go ] 中 第[85]行使用的是 [ reader := io.LimitReader(r.Body, maxBodyLen) ] 会在CGO 混编工程、或者使用unsafe.Pointer() 方法使用过后 报 [ string: ``, error: EOF ] 错误

尝试修改[reader := io.LimitReader(r.Body, maxBodyLen)] 为 [reader,_ := io.ReadAll(r.Body) ] 后错误解决!! 尝试修改[reader := io.LimitReader(r.Body, maxBodyLen)] 为 [reader,_ := io.ReadAll(r.Body) ] 后错误解决!! 尝试修改[reader := io.LimitReader(r.Body, maxBodyLen)] 为 [reader,_ := io.ReadAll(r.Body) ] 后错误解决!!

george012 avatar Nov 23 '22 22:11 george012

Please provide scenarios that can be reproduced

wsx864321 avatar Nov 24 '22 04:11 wsx864321

①: Use [global middleware], use CGO library to operate data image ②: Data parsing error in [hadler] subsequent processing image Error message here: "string: ``, error: EOF" image

george012 avatar Nov 24 '22 09:11 george012

Bot detected the issue body's language is not English, translate it automatically. 👯👭🏻🧑‍🤝‍🧑👫🧑🏿‍🤝‍🧑🏻👩🏾‍🤝‍👨🏿👬🏿


①: Use [global middleware], use CGO library to operate data image ②: Data parsing error in [hadler] subsequent processing image Error message here: "string: ``, error: EOF" image

Issues-translate-bot avatar Nov 24 '22 09:11 Issues-translate-bot

msg

george012 avatar Nov 25 '22 14:11 george012

Same question about this error. Any update?

browningweb avatar Dec 21 '22 11:12 browningweb

I have solved this question. Idea from https://stackoverflow.com/questions/23070876/reading-body-of-http-request-without-modifying-request-state

Here is the code.

func (rp *RequestParse) Parse(r *http.Request) (map[string]interface{}, error) {

	vars := pathvar.Vars(r)
	params, err := httpx.GetFormValues(r)
	if err != nil {
		return nil, err
	}

	for k, v := range vars {
		params[k] = v
	}

	body, ok := getBody(r)
	if !ok {
		return params, nil
	}

	m := make(map[string]interface{})
	if err := json.NewDecoder(body).Decode(&m); err != nil {
		return nil, err
	}

	for k, v := range params {
		m[k] = v
	}

	return m, nil
}

func getBody(r *http.Request) (io.Reader, bool) {
	if r.Body == nil {
		return nil, false
	}

	if r.ContentLength == 0 {
		return nil, false
	}

	buf, _ := io.ReadAll(r.Body)

	rdr1 := io.NopCloser(bytes.NewBuffer(buf))
	rdr2 := io.NopCloser(bytes.NewBuffer(buf))

	r.Body = rdr1

	return rdr2, true
}

browningweb avatar Dec 21 '22 13:12 browningweb

I have solved this question. Idea from https://stackoverflow.com/questions/23070876/reading-body-of-http-request-without-modifying-request-state

Here is the code.

func (rp *RequestParse) Parse(r *http.Request) (map[string]interface{}, error) {

	vars := pathvar.Vars(r)
	params, err := httpx.GetFormValues(r)
	if err != nil {
		return nil, err
	}

	for k, v := range vars {
		params[k] = v
	}

	body, ok := getBody(r)
	if !ok {
		return params, nil
	}

	m := make(map[string]interface{})
	if err := json.NewDecoder(body).Decode(&m); err != nil {
		return nil, err
	}

	for k, v := range params {
		m[k] = v
	}

	return m, nil
}

func getBody(r *http.Request) (io.Reader, bool) {
	if r.Body == nil {
		return nil, false
	}

	if r.ContentLength == 0 {
		return nil, false
	}

	buf, _ := io.ReadAll(r.Body)

	rdr1 := io.NopCloser(bytes.NewBuffer(buf))
	rdr2 := io.NopCloser(bytes.NewBuffer(buf))

	r.Body = rdr1

	return rdr2, true
}

This case is because you have read the body, actually, it is not a bug.

wsx864321 avatar Dec 22 '22 04:12 wsx864321

同样是httpx.Parse报错"EOF" ` // go-zero版本 github.com/zeromicro/go-zero v1.5.5

// server配置了MaxBytes: 536870912 Name: img Host: 0.0.0.0 Port: 8888 MaxBytes: 536870912 Timeout: 600000

// api配置了maxBytes @server ( prefix: /v1 timeout: 300s // 对当前语法块下的所有路由添加请求体大小控制,512MB,单位为 byte maxBytes: 536870912 ) ` post请求,body数据超过8MB(8388608)就会报错,调试发现是这里报错 (调试时请求参数,ContentLength=11042730) image

把body的ContentLength降到8MB以下能正常请求

是bug吗?

@kevwan

yiGmMk avatar Sep 28 '23 09:09 yiGmMk

Bot detected the issue body's language is not English, translate it automatically. 👯👭🏻🧑‍🤝‍🧑👫🧑🏿‍🤝‍🧑🏻👩🏾‍🤝‍👨🏿👬🏿


The same httpx.Parse error "EOF"

Issues-translate-bot avatar Sep 28 '23 09:09 Issues-translate-bot

This issue is stale because it has been open for 30 days with no activity.

github-actions[bot] avatar Oct 01 '24 02:10 github-actions[bot]

This issue was closed because it has been inactive for 14 days since being marked as stale.

github-actions[bot] avatar Jan 03 '25 02:01 github-actions[bot]