py-linkedin-jobs-scraper icon indicating copy to clipboard operation
py-linkedin-jobs-scraper copied to clipboard

li_at token expires after 15 minutes

Open kalmanlukelin opened this issue 1 year ago • 6 comments

Hi, it seems that li_at token would expire after 15 minutes. Although it says that it would expire in a year in the official doc. Any suggestion to extend the lifetime of the cookie instead of refreshing it manually? https://www.linkedin.com/legal/l/cookie-table

kalmanlukelin avatar Dec 19 '23 12:12 kalmanlukelin

Hi, maybe you could listen to the invalid session event https://github.com/spinlud/py-linkedin-jobs-scraper/blob/master/linkedin_jobs_scraper/events/events.py#L11 and try a programmatic login (e.g. using Selenium) to Linkedin to obtain a new session cookie.

spinlud avatar Jan 16 '24 10:01 spinlud

On my end, it's expiring even sooner. No matter how slow I set the slow_mo value too. I started with 5 now I am at 100. By the way, I am authenticated

calvinomiguel avatar Feb 04 '24 23:02 calvinomiguel

This may be related to the random UA string being used. After replacing the list of UA strings with the exact one from my browser my session remains valid ~2 hours in.

phillips-systems avatar May 13 '24 17:05 phillips-systems

I have removed the random user agent override as suggested by @phillips-systems, let me know if this helps a bit. Unfortunately it seems that Linkedin expires earlier sessions that appear 'suspect' (according to some criteria that are unclear). Using the same browser to obtain the cookie and run the scraper could probably help...

The same issue is mentioned here https://www.reddit.com/r/webscraping/comments/18t2xao/issues_with_linkedin_li_at_cookie_expiration/

spinlud avatar Jun 02 '24 20:06 spinlud

If I had to guess, they are validating cookie values against UA strings as a method to prevent account takeover.

phillips-systems avatar Jun 02 '24 21:06 phillips-systems

On my end, it's expiring even sooner. No matter how slow I set the slow_mo value too. I started with 5 now I am at 100. By the way, I am authenticated

how did you get authenticated, I cant seem to get there either

rshah2399 avatar Jun 05 '24 11:06 rshah2399