go-zero icon indicating copy to clipboard operation
go-zero copied to clipboard

Cannot convert `%26` to `&` in http url when using `httpx.Parse`

Open jacksonjim opened this issue 1 year ago • 3 comments

Describe the bug A clear and concise description of what the bug is. 这是测试链接 http://127.0.0.1/api/v2/dev/test?dev=se205%5fy1205%5fj109%26verRelease=v01%26iid1=863494061186673%26iid2=863494061186681%26mcc=636%26mnc=1

请求后返回400错误,无法解析form中的参数 {"@timestamp":"2024-01-16T11:12:50.728+08:00","caller":"handler/loghandler.go:147","content":"[HTTP] 400

根踪了相关源码目前发现是调用这部分代码, if err := httpx.Parse(r, &req); err != nil { httpx.ErrorCtx(r.Context(), w, err) return } 深度追查,发现在是这里引起 func GetFormValues(r *http.Request) (map[string]any, error) { if err := r.ParseForm(); err != nil { return nil, err } ... }

目前能想到的解决方案是在调用这个转换参数函数 httpx.Parse()前调用 enEscapeUrl, _ := url.QueryUnescape(r.RequestURI) enRawQuery, _ := url.QueryUnescape(r.URL.RawQuery) r.RequestURI = enEscapeUrl r.URL.RawQuery = enRawQuery 这些处理下相关代码才可以正常识别出form的参数,所以希望能在官网库统一处理下,或是有更好的方式是在http url中增加处理这个转义字符识别处理

To Reproduce Steps to reproduce the behavior, if applicable:

  1. The code is 修改验证的
    func TestHandler(svcCtx *svc.ServiceContext) http.HandlerFunc {
     return func(w http.ResponseWriter, r *http.Request) {
     	var req types.SendActivationReq
     	enEscapeUrl, _ := url.QueryUnescape(r.RequestURI)
     	enRawQuery, _ := url.QueryUnescape(r.URL.RawQuery)
     	r.RequestURI = enEscapeUrl
     	r.URL.RawQuery = enRawQuery
     	if err := httpx.Parse(r, &req); err != nil {
     		httpx.ErrorCtx(r.Context(), w, err)
     		return
     	}
    
     	l := device.NewSendActivationLogic(r.Context(), svcCtx)
     	resp, err := l.SendActivation(&req)
     	if err != nil {
     		httpx.ErrorCtx(r.Context(), w, err)
     	} else {
     		httpx.OkJsonCtx(r.Context(), w, resp)
     	}
     }
    

}


2. The error is
原始来生成的代码
func TestHandler(svcCtx *svc.ServiceContext) http.HandlerFunc {
return func(w http.ResponseWriter, r *http.Request) {
	var req types.SendActivationReq
	if err := httpx.Parse(r, &req); err != nil {
		httpx.ErrorCtx(r.Context(), w, err)
		return
	}

	l := device.NewSendActivationLogic(r.Context(), svcCtx)
	resp, err := l.SendActivation(&req)
	if err != nil {
		httpx.ErrorCtx(r.Context(), w, err)
	} else {
		httpx.OkJsonCtx(r.Context(), w, resp)
	}
}

}


**Expected behavior**
A clear and concise description of what you expected to happen.

**Screenshots**
If applicable, add screenshots to help explain your problem.

**Environments (please complete the following information):**
- OS: window & linux
- go-zero version v1.6.0
- goctl version 1.2.5

**More description**
Add any other context about the problem here.

jacksonjim avatar Jan 16 '24 03:01 jacksonjim

我也遇到了类似的问题,我在用 go 写一个兼容 nacos client 的注册中心+配置中心,在做配置监听时,nacos client 是将 如下字符串以 post 的形式提交,参数放在 body 中,参数类型:application/x-www-form-urlencoded Listening-Configs=dataId%02group%02contentMD5%02tenant%01 经过 debug 发现,这似乎并不是 go-zero 的问题,当然,最初是这么以为的,在关闭 gzip 的情况下,可拿到 urldecode 后的结果(乱码),然后我重新 encode 了下 好了。开启 gzip 时,请求 会 400

ijustyce avatar Jan 20 '24 11:01 ijustyce

Check #3888

kevwan avatar Feb 02 '24 17:02 kevwan

Bot detected the issue body's language is not English, translate it automatically. 👯👭🏻🧑‍🤝‍🧑👫🧑🏿‍🤝‍🧑🏻👩🏾‍🤝‍👨🏿👬🏿


Check #3888

Issues-translate-bot avatar Feb 02 '24 17:02 Issues-translate-bot

测试下最新的版本v1.6.2 这个问题点还是存在,不知道什么时候会修复掉这小问题,可能部分场景会出现类似的情况

jacksonjim avatar Feb 21 '24 02:02 jacksonjim

Bot detected the issue body's language is not English, translate it automatically. 👯👭🏻🧑‍🤝‍🧑👫🧑🏿‍🤝‍🧑🏻👩🏾‍🤝‍👨🏿👬🏿


After testing the latest version v1.6.2, this problem still exists. I don’t know when this small problem will be fixed. Similar situations may occur in some scenarios.

Issues-translate-bot avatar Feb 21 '24 02:02 Issues-translate-bot