pixivpy icon indicating copy to clipboard operation
pixivpy copied to clipboard

pixiv seems not allowing username/password login now

Open roytam1 opened this issue 3 years ago • 37 comments

server returning JSON:

{"has_error":true,"errors":{"system":{"message":"The grant type is unauthorized for this client_id","code":1508}},"error":"invalid_grant"}

roytam1 avatar Feb 09 '21 06:02 roytam1

background information:

In pixiv android Version 5.0.234 in 2021-02-08, there is a line:

  • From the beginning of February 2021, you'll have to update your app to the latest version if you're running ver.5.0.219 or earlier.

and for testing, I tried to install 5.0.220.1 apk and tried to login, and it pops out custom browser window and jumps to pixiv standard login form.

roytam1 avatar Feb 09 '21 07:02 roytam1

a cross reference is found: https://github.com/upbit/pixivpy/issues/140#issuecomment-708976828

roytam1 avatar Feb 09 '21 07:02 roytam1

I encountered the same problem

HTTP 400: {"has_error":true,"errors":{"system":{"message":"Invalid grant_type parameter or parameter missing","code":1508}},"error":"invalid_grant"}

I want to use set_auth, but I don't know which is access token, there're to many pairs in cookie

kilosonc avatar Feb 09 '21 10:02 kilosonc

I want to use set_auth, but I don't know which is access token, there're to many pairs in cookie

they're not exist in cookies or IndexedDB or Local Storage.

roytam1 avatar Feb 09 '21 10:02 roytam1

In my project I had to revert back to using the Ajax API. The problem is that the grant_type=password authentication method is no longer supported. The Pixiv app now logs in through https://accounts.pixiv.net/login, but this page is protected by Google reCAPTCHA, which seems impossible to circumvent.

I think that as long as you have a refresh token, you can keep using it, but getting one is tricky since you have to login with the mobile app and sniff traffic to get it.

https://github.com/danbooru/danbooru/blob/39cc3ed5cf913499093d2f641d70d7682a14fa42/app/logical/pixiv_ajax_client.rb

evazion avatar Feb 10 '21 00:02 evazion

If password authentication is disabled, is there no choice but to scrape with BeautifulSoup from now on? Unbelievable

emesh0620 avatar Feb 10 '21 00:02 emesh0620

Screenshot_20210210_155236 经测试该账号和密码可以pixiv官网正常登录

Mouwoo avatar Feb 10 '21 07:02 Mouwoo

Last year, you said that you are trying not to register with a leaked password, but you are also considering methods such as multi-factor authentication, so it is time to think about some fundamental measures. It may be. (Link to Japanese)

https://inside.pixiv.blog/2020/01/22/180000

emesh0620 avatar Feb 10 '21 08:02 emesh0620

Pixiv was changed grant_type password to authorization_code. But it's so hard to login because pixiv using login with OAuth 2.0 image

mrwan200 avatar Feb 11 '21 01:02 mrwan200

Pixiv now follows RFC7636. We won't be able to use (old) password login again, but there is a way to automate token retrieval without traffic sniffing. I've implemented a simple script to retrieve access/refresh tokens. Right now it requires single manual step inspecting dev console, but in general, entire flow could be fully automated using browser extensions: you can add request handler and extract code param from the url.

ZipFile avatar Feb 11 '21 21:02 ZipFile

Not being able to login with password is really bad news.. Thank you all for these new methods. My current network is relatively poor, and the login method will be updated as soon as possible after verification.

Happy Chinese New Year!

upbit avatar Feb 12 '21 11:02 upbit

pixiv seems not allowing username/password login now #158😢😢 It's really sad..😢 Is there any solutions yet?

farmerswalker avatar Feb 12 '21 15:02 farmerswalker

群友看不到涩图了,上issues一看果然有蹊跷 根据我这边程序的推断,Pixiv登录方式具体的更换时间应该是(UTC+8)2021/2/8 22:30 - (UTC+8)2021/2/9 22:30之间 还是希望尽快恢复罢

MFWT avatar Feb 13 '21 02:02 MFWT

Pixiv now follows RFC7636. We won't be able to use (old) password login again, but there is a way to automate token retrieval without traffic sniffing. I've implemented a simple script to retrieve access/refresh tokens. Right now it requires single manual step inspecting dev console, but in general, entire flow could be fully automated using browser extensions: you can add request handler and extract code param from the url.

这个方法有效耶, 可以用pyside2中内嵌的浏览器自动登录web版自动获取code, 登录后的phpssid保存起来下次用, 作者什么时候更新登录方法啊, 过年想改改pixiv爬虫代码结果登录都不行. @ZipFile

Akise avatar Feb 14 '21 08:02 Akise

Pixiv now follows RFC7636. We won't be able to use (old) password login again, but there is a way to automate token retrieval without traffic sniffing. I've implemented a simple script to retrieve access/refresh tokens. Right now it requires single manual step inspecting dev console, but in general, entire flow could be fully automated using browser extensions: you can add request handler and extract code param from the url.

这个方法有效耶, 可以用pyside2中内嵌的浏览器自动登录web版自动获取code, 登录后的phpssid保存起来下次用, 作者什么时候更新登录方法啊, 过年想改改pixiv爬虫代码结果登录都不行. @ZipFile

@Akise 不需要作者更新登录方法啊,本来就支持用refresh_token登录。你人工获取一次后存在本地,够用很长时间。api.auth(refresh_token='refresh_token')

Xdynix avatar Feb 14 '21 08:02 Xdynix

Pixiv now follows RFC7636. We won't be able to use (old) password login again, but there is a way to automate token retrieval without traffic sniffing. I've implemented a simple script to retrieve access/refresh tokens. Right now it requires single manual step inspecting dev console, but in general, entire flow could be fully automated using browser extensions: you can add request handler and extract code param from the url.

这个方法有效耶, 可以用pyside2中内嵌的浏览器自动登录web版自动获取code, 登录后的phpssid保存起来下次用, 作者什么时候更新登录方法啊, 过年想改改pixiv爬虫代码结果登录都不行. @ZipFile

@Akise 不需要作者更新登录方法啊,本来就支持用refresh_token登录。你人工获取一次后存在本地,够用很长时间。api.auth(refresh_token='refresh_token')

refresh_token要怎样获取呢?群友们没有涩图看饿到嗷嗷叫了

MFWT avatar Feb 14 '21 08:02 MFWT

Pixiv now follows RFC7636. We won't be able to use (old) password login again, but there is a way to automate token retrieval without traffic sniffing. I've implemented a simple script to retrieve access/refresh tokens. Right now it requires single manual step inspecting dev console, but in general, entire flow could be fully automated using browser extensions: you can add request handler and extract code param from the url.

这个方法有效耶, 可以用pyside2中内嵌的浏览器自动登录web版自动获取code, 登录后的phpssid保存起来下次用, 作者什么时候更新登录方法啊, 过年想改改pixiv爬虫代码结果登录都不行. @ZipFile

@Akise 不需要作者更新登录方法啊,本来就支持用refresh_token登录。你人工获取一次后存在本地,够用很长时间。api.auth(refresh_token='refresh_token')

你有几个账号? 都是人工获取? refresh_token能存活多久? 为什么不做成自动化?

一个账号,偶尔才用。目前观察到的来看存活至少一个月,上个月获取的还能用。ZipFile大佬的脚本都说了目前还需要人工介入。而操作浏览器获取refresh token的内容我认为有点超出pixiv api库的职能范畴了。

Xdynix avatar Feb 14 '21 09:02 Xdynix

Pixiv now follows RFC7636. We won't be able to use (old) password login again, but there is a way to automate token retrieval without traffic sniffing. I've implemented a simple script to retrieve access/refresh tokens. Right now it requires single manual step inspecting dev console, but in general, entire flow could be fully automated using browser extensions: you can add request handler and extract code param from the url.

这个方法有效耶, 可以用pyside2中内嵌的浏览器自动登录web版自动获取code, 登录后的phpssid保存起来下次用, 作者什么时候更新登录方法啊, 过年想改改pixiv爬虫代码结果登录都不行. @ZipFile

@Akise 不需要作者更新登录方法啊,本来就支持用refresh_token登录。你人工获取一次后存在本地,够用很长时间。api.auth(refresh_token='refresh_token')

refresh_token要怎样获取呢?群友们没有涩图看饿到嗷嗷叫了

@MFWT 楼上ZipFile大佬的脚本https://github.com/upbit/pixivpy/issues/158#issuecomment-777815440

Xdynix avatar Feb 14 '21 09:02 Xdynix

现在pixiv app端登录会打开个浏览器登录, 会遇到谷歌reCAPTCHA, 我一直使用pyside2内嵌的浏览器模拟登录pixiv, 小概率触发验证码. 或者用pyppeteer . @ZipFile @upbit @Xdynix @evazion

Akise avatar Feb 14 '21 09:02 Akise

At present, the login method of the previous app is no longer available 😢 As @Xdynix said, integrating OAuth2 login in pixivpy is not reasonable, but we can consider using third-party libraries to simplify token acquisition.

In the short term, you can use the method provided by @ZipFile to manually obtain the token through the browser. In addition, using the refresh(refresh_token) method, you can retrieve the access_token to avoid expiration (default 3600 seconds), this may also be a solution to keep login.


目前,之前App的登录方法都不再可用😢 正如 @Xdynix 所说,在pixivpy中集成OAuth2登录是不合理的,不过我们可以考虑使用第三方库来简化access_token的获取。

在短期内,您可以使用 @ZipFile 提供的方法通过浏览器手动获取tokens。 另外,使用refresh(refresh_token)方法,可以重新获取access_token以避免过期(默认3600秒),这也许可以作为保持登录的另一个解决方案。


临时解决方法:

  1. 根据 @ZipFile 的方法取的refresh_token
  2. 采用 api.auth(refresh_token=_REFRESH_TOKEN) 方式登录 (refresh_token目前测试中未变化过,推测可以保存后长时间使用;如果失效可以用1的方法再获取一次)
    # api.login(_USERNAME, _PASSWORD)
    api.auth(refresh_token=_REFRESH_TOKEN)

upbit avatar Feb 15 '21 03:02 upbit

不过我们可以考虑使用第三方库来简化access_token的获取。

什么方法? 半自动+手动? 我想找个全自动登录的思路. @upbit

Do you have a fully automatic login scheme? @ZipFile

Akise avatar Feb 15 '21 06:02 Akise

不过我们可以考虑使用第三方库来简化access_token的获取。

什么方法? 半自动+手动? 我想找个全自动登录的思路. @upbit

Do you have a fully automatic login scheme? @ZipFile

全自动估计比较复杂(至少想到的得用headless的浏览器模拟)。半自动是指,可以把callback转向本地监听的server,从而自动获取token(避免过期),不过目前还没找到可行的库

upbit avatar Feb 15 '21 06:02 upbit

I don't have much time to implement fully automated solution, so I shared bare minimum implementation so that everyone could recover ASAP.

Algorithm itself is not something unique, as it is described by aforementioned RFC. Important parts are urls and params used in Pixiv auth flow. OAuth PKCE works well on mobile because it relies on OS implementation of deep links. On desktop it's pain in the ass to deal with, so we have to use embedded/debug browser (QtWebEngine, Puppeteer, etc...) or extension with elevated permissions to extract the code.

ZipFile avatar Feb 15 '21 07:02 ZipFile

Many of you may be worried that you'll have to log in and set it up again, but you'll still be able to log in after several attempts, so even if you use Auth, it will last quite a while. But I can't say for sure because we don't examine it every few hours or days.

emesh0620 avatar Feb 16 '21 13:02 emesh0620

I tested the login method of refresh_token, and I can get access_token successfully (and refresh_token not changed):

  1. Within 3600 seconds, success
  2. About 1-2 hours, success
  3. About 3-4 hours, success
  4. More than 24 hours, success

Temporarily use the following method, you should be able to keep logged in:

api.auth(refresh_token=_REFRESH_TOKEN)

upbit avatar Feb 16 '21 13:02 upbit

楼上有人问RefreshToken管不管用,我在这说一句,管用 我回复一条issue的时候,是2/14 image 然后一直到现在,都能用(注意看ranking.csv修改时间) image

MFWT avatar Feb 16 '21 13:02 MFWT

Update the semi-automatic script, get the code through chromedriver


更新了半自动脚本,通过chromedriver获取code

https://gist.github.com/upbit/6edda27cb1644e94183291109b8a5fde

upbit avatar Feb 17 '21 03:02 upbit

我这几天自己研究后把我的脚本更新了,果然这边也有问题。

有需要的可以参考我的做法。 https://github.com/Mapaler/PixivUserBatchDownload/wiki/%E8%8E%B7%E5%8F%96APP%E7%99%BB%E9%99%86%E9%93%BE%E6%8E%A5

也是第一次用 pivix 私有链接登陆后保存auth数据,之后就每次只刷新token了。

具体网络请求代码可以参考 https://github.com/Mapaler/PixivUserBatchDownload/blob/f1d7286ce033a242d0278feac0fad99380c30a51/PixivUserBatchDownload.user.js#L379

Mapaler avatar Feb 20 '21 18:02 Mapaler

Update the semi-automatic script, get the code through chromedriver

更新了半自动脚本,通过chromedriver获取code

https://gist.github.com/upbit/6edda27cb1644e94183291109b8a5fde

请问手机端能拿到吗 pc 的试了拿到code后就过不去了 开了全局

yzwik avatar Feb 24 '21 01:02 yzwik

Update the semi-automatic script, get the code through chromedriver 更新了半自动脚本,通过chromedriver获取code https://gist.github.com/upbit/6edda27cb1644e94183291109b8a5fde

请问手机端能拿到吗 pc 的试了拿到code后就过不去了 开了全局

如果能输出code,可以参考requests的代理配置,对95行的post请求增加代理

upbit avatar Feb 24 '21 01:02 upbit