unshort.link is not respecting cookies set by inbetween requests

[nastyagrifon]

Russian popular social media has its own URL shortening service, which is used to generate a short link any time user shares a post/link/photo/etc. You can also manually create a link using vk.cc website. Well, unshort.link doesn't process those links correctly and instead of the target URL it redirects to login/register form. I believe you have to be logged in to create a link, but anyone can use the shortened link

Steps to reproduce: 1) Click the target link Target URL: https://vk.cc/aAGENC (as an example) Expected behavior: unshort.link tab opens up and suggests to redirect to https://github.com/simonfrey/unshort.link Actual behavior: unshort.link tab opens up and suggests to redirect to https://vk.com/login?to=YWxfZmVlZC5waHA-&u=2 Affected system: Win 10 LTSC x64, Chrome 85.0.4183.121 (Official Build) (64-bit)

Screenshot

[simonfrey]

Thanks for the info!

The inbetween urll seems to be: https://vk.com/away.php?cc_key=aAGENC&to=https%3A%2F%2Fgithub.com%2Fsimonfrey%2Funshort.link

Apparently that endpoint is setting a Cookie, which is not persistent for the next call by unshort.link

GET /away.php?cc_key=aAGENC&to=https%3A%2F%2Fgithub.com%2Fsimonfrey%2Funshort.link HTTP/1.1
Accept: */*
Accept-Encoding: gzip, deflate
Connection: keep-alive
Host: vk.com
User-Agent: HTTPie/2.2.0



HTTP/1.1 302 Found
Access-Control-Expose-Headers: X-Frontend
Cache-control: no-store
Connection: keep-alive
Content-Encoding: gzip
Content-Length: 20
Content-Type: text/html; charset=windows-1251
Date: Sun, 25 Oct 2020 15:43:01 GMT
Location: https://away.vk.com/away.php
Server: kittenx
Set-Cookie: remixir=DELETED; expires=Thu, 01 Jan 1970 00:00:01 GMT; path=/; domain=.vk.com; secure; HttpOnly
Set-Cookie: remixlang=6; expires=Tue, 02 Nov 2021 12:13:23 GMT; path=/; domain=.vk.com
Set-Cookie: remixsec_redir=https%3A%2F%2Fgithub.com%2Fsimonfrey%2Funshort.link; path=/; domain=.vk.com
Set-Cookie: remixua=-1%7C-1%7C-1%7C1436123317; expires=Fri, 29 Oct 2021 03:52:49 GMT; path=/; domain=.vk.com; secure
Strict-Transport-Security: max-age=15768000
X-Frame-Options: DENY
X-Frontend: front605111
X-Powered-By: KPHP/7.4.26843

As the root cause for the problem is the missing cookie, I will rename the issue to that.

[simonfrey]

I assume the way the vk link shortener is implemented unshort.link will not be able to support it :/

The problem is that vk sets a cookie and then redirects again. This cookie is required for redirecting and even if unshort.link handles the cookie, the final url given to the user will be useless as the user does not have the cookie. Kinda nice idea they did build there, as with this measure all users opening a vk link will be tracked permanently via the cookie and no unshort service is able to prevent that. Touché

[nastyagrifon]

Can't we temporarily store the needed cookie to determine the destination URL, then discard it afterwards?

[simonfrey]

Yeah actually that works. Added a cookie jar and adapted the regex in order for this to work. Should work now. Please try with a new shortlink (as the old one has the old info cached)

[nastyagrifon]

"webextension" compiles the old version 1.5.5 and obviously doesn't fix much. Compiling "server" fails on me both on Windows and Debian. Could you provide the crx or zip for testing?

[simonfrey]

Yeah the webextension did not change. Tried the build on debian and arch and both worked, what error do you get?

Otherwise you could try it on https://unshort.link

[nastyagrifon]

Otherwise you could try it on https://unshort.link Doesn't work for me, still redirects me to the login page.

Prob smth wrong with my Debian setup, I'll check on that and let you know asap

[simonfrey]

Can I consider this as fixed?

[nastyagrifon]

So my go lang setup seems to be borked. Building server and extension returns an error

https://unshort.link/ still returns https://vk.com/login?to=YWxfZmVlZC5waHA-&u=2 and so does extension.

Server "make build" output

nastyagrifon@laptop:~/Desktop/unshort.link/server$ make build
Generating assets...
go: downloading github.com/pkg/errors v0.9.1
go: downloading github.com/mattn/go-sqlite3 v2.0.2+incompatible
go: downloading github.com/jmoiron/sqlx v1.2.0
go: downloading golang.org/x/net v0.0.0-20190620200207-3b0461eec859
go: downloading github.com/sergi/go-diff v1.1.0
go: downloading github.com/sirupsen/logrus v1.4.2
go: downloading golang.org/x/sys v0.0.0-20190804053845-51ab0e2deafa
go: downloading github.com/programmfabrik/esc v0.2.5
go: github.com/programmfabrik/esc upgrade => v0.2.5
go: downloading golang.org/x/tools v0.0.0-20190925020647-22afafe3322a
go: github.com/pkg/errors upgrade => v0.9.1
go: golang.org/x/tools upgrade => v0.0.0-20201111224557-41a3a589386c
go: downloading golang.org/x/tools v0.0.0-20201111224557-41a3a589386c
go: downloading golang.org/x/mod v0.3.0
go: downloading golang.org/x/xerrors v0.0.0-20200804184101-5ec99f83aff1
go: golang.org/x/xerrors upgrade => v0.0.0-20200804184101-5ec99f83aff1
main.go:15: running "esc": exec: "esc": executable file not found in $PATH
go: github.com/pkg/errors upgrade => v0.9.1
go: golang.org/x/xerrors upgrade => v0.0.0-20200804184101-5ec99f83aff1
go: golang.org/x/tools upgrade => v0.0.0-20201111224557-41a3a589386c
db/db.go:18: running "esc": exec: "esc": executable file not found in $PATH
make: *** [Makefile:8: generate] Error 1 ```

[simonfrey]

Did you try it with a new link? (as old redirects are cached and thereby the old link will still redirect to the errored page)

[simonfrey]

The build problem is on me. Forgot to add go get github.com/programmfabrik/esc to the makefile. Should work with the new makefile :D

[nastyagrifon]

Didn't seem to change anything

Server "make build" output

nastyagrifon@laptop:~/Desktop/unshort.link/server$ make build Go get esc... Got esc Generating assets... go: downloading golang.org/x/sys v0.0.0-20200930185726-fdedc70b468f go: downloading golang.org/x/net v0.0.0-20201021035429-f5854403a974 go: github.com/pkg/errors upgrade => v0.9.1 go: golang.org/x/xerrors upgrade => v0.0.0-20200804184101-5ec99f83aff1 go: golang.org/x/tools upgrade => v0.0.0-20201125231158-b5590deeca9b main.go:15: running "esc": exec: "esc": executable file not found in $PATH go: github.com/pkg/errors upgrade => v0.9.1 go: golang.org/x/xerrors upgrade => v0.0.0-20200804184101-5ec99f83aff1 go: golang.org/x/tools upgrade => v0.0.0-20201125231158-b5590deeca9b db/db.go:18: running "esc": exec: "esc": executable file not found in $PATH make: *** [Makefile:11: generate] Error 1

[simonfrey]

Could you manually go get github.com/programmfabrik/esc to see if that helps?

[ghost]

FYI, vk.cc is included in Hexxium Creations Threat List and displayed accordingly in its blocklist.