gorequest icon indicating copy to clipboard operation
gorequest copied to clipboard

I guess AddCookie/AddCookies doesnt work?

Open MikhailKlemin opened this issue 8 years ago • 5 comments

Hi!

I just tried to use it vs sites which use cookies to confirm language/currency, and it's just do not work, I am not sure why... I can only confirm that same sites with standard approach (cookiejar and http.Client(jar: cookieJar)) work just fine.

MikhailKlemin avatar Nov 17 '16 19:11 MikhailKlemin

Hi @MikhailKlemin I just have the same issue, but it works well if i put the AddCookies back the post/get.

han2015 avatar Jun 14 '18 12:06 han2015

Hi i just have the same issue, i have try many method, but i have not find the way, so i need you help@han2015, this is mine code , func GetGoRequest() { request := gorequest.New() //loginCookie := GetLoginCookie() // //fmt.Println("go request logincookie",loginCookie) cookie := &http.Cookie{Name: "PHPSESSID", Value: "t2cdimadecpoel6i8rcdomk1m6"} fmt.Println("go request cookie--->", cookie) _, _, err := request. Get("https://test-xxxxxx.cn/api/index.php?r=detail/brokerage-flows&token=cdkqqf1407307954&recommend_id=39ee6603-e648-49b1-12f5-e803efbf383a&devopenid=88374"). Send("r":"detail/recommend-flows","token":"cdkqqf1407307954","recommend_id":"39ee6603-e648-49b1-12f5-e803efbf383a"). AddCookie(cookie). End() if err != nil{ fmt.Printf("%v \n", err) } fmt.Println(request.Header) // this is always map[] } please, give me same help, thank you

kervinson avatar Jun 16 '19 09:06 kervinson

@kervinson correct your Send sentence refer doc(https://github.com/parnurzeal/gorequest/blame/develop/README.md#L120).

go request cookie---> PHPSESSID=t2cdimadecpoel6i8rcdomk1m6
++++++++++++++1
++++++++++++++TypeJSON
++++++++++++++5
[Get https://test-xxxxxx.cn/api/index.php?devopenid=88374&r=detail%2Fbrokerage-flows&recommend_id=39ee6603-e648-49b1-12f5-e803efbf383a&token=cdkqqf1407307954: dial tcp: lookup test-xxxxxx.cn: no such host]
[PHPSESSID=t2cdimadecpoel6i8rcdomk1m6]

han2015 avatar Jun 17 '19 00:06 han2015

I met the same problem, my code is bvelow

env : window 10, go 1.15, gorequest 1.6.0

package main


import (
	"github.com/parnurzeal/gorequest"
	"net/http"
	"strings"
	"log"
)

func buildCookie(cookie string) []*http.Cookie {
	cookieSlice := make([]*http.Cookie, 0)
	params := strings.Split(cookie, ";")
	for _, line := range params {
		res := strings.Split(line, "=")
		cookieSlice = append(cookieSlice, &http.Cookie{Name: res[0], Value: res[1]})
	}
	return cookieSlice
}

func main() {
	cookies := []string {"cookie1", "cookie2", "cookie3", "cookie4"..}
		
	r := gorequest.New()
	r = r.Set("Accept","text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8").
			Set("Accept-Encoding","gzip, deflate, br").
			Set("Accept-Language","zh-CN,zh;q=0.8,zh-TW;q=0.7,zh-HK;q=0.5,en-US;q=0.3,en;q=0.2").
			Set("Host","www.tianyancha.com").
			// Set("Referer","https://www.tianyancha.com/search/ocD?base=taiyuan").
			Set("User-Agent","Mozilla/5.0 (Windows NT 10.0; WOW64; rv:68.0) Gecko/20100101 Firefox/68.0")
	index := 0


	req, _ := http.NewRequest("GET", "https://www.tianyancha.com/company/3192219802", nil)
	req.Header.Set("Accept","text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8")
	req.Header.Set("Accept-Encoding","gzip, deflate, br")
	req.Header.Set("Accept-Language","zh-CN,zh;q=0.8,zh-TW;q=0.7,zh-HK;q=0.5,en-US;q=0.3,en;q=0.2")
	req.Header.Set("Host","www.tianyancha.com")
	// Set("Referer","https://www.tianyancha.com/search/ocD?base=taiyuan").
	req.Header.Set("User-Agent","Mozilla/5.0 (Windows NT 10.0; WOW64; rv:68.0) Gecko/20100101 Firefox/68.0")
	for i := 0; i<3; i ++ {
		// cookies[index%len(cookies)])
		r.AddCookies(buildCookie(cookies[index%len(cookies)]))
		// log.Println(r.Header)
		// r = r.Set("Cookie", cookies[index%len(cookies)])
		response, _, _ := r.Get("https://www.tianyancha.com/company/3192219802").End()
		log.Println("url, ", response.Request.URL.String())
		
		req.Header.Set("Cookie", cookies[index%len(cookies)])

		client := &http.Client{}
		resp, _ := client.Do(req)
		log.Println(resp.Request.URL.String())
		index ++
	}
}

output

2020/11/11 16:20:42 url,  https://www.tianyancha.com/login?from=https%3A%2F%2Fwww.tianyancha.com%2Fcompany%2F3192219802
2020/11/11 16:20:43 https://www.tianyancha.com/company/3192219802
2020/11/11 16:20:43 url,  https://www.tianyancha.com/login?from=https%3A%2F%2Fwww.tianyancha.com%2Fcompany%2F3192219802
2020/11/11 16:20:43 https://www.tianyancha.com/company/3192219802
2020/11/11 16:20:44 url,  https://www.tianyancha.com/login?from=https%3A%2F%2Fwww.tianyancha.com%2Fcompany%2F3192219802
2020/11/11 16:20:44 https://www.tianyancha.com/company/3192219802

anyway, i cannot get it through via gorequest~~~

evinstar avatar Nov 11 '20 08:11 evinstar

Now I've solved this problem. Its really a misuse of cookie.

When I read source code of gorequest, I found it would clear the agent stat every time before a new request if DoNotClearSuperAgent is default to false.

So the solution is add this code between you set cookie and start a new request.

agent := gorequest.New()
agent.SetDoNotClearSuperAgent(true)

evinstar avatar Nov 16 '20 09:11 evinstar