TikTokPy icon indicating copy to clipboard operation
TikTokPy copied to clipboard

[BUG] TikTok seems to have changed things

Open bobcogneato opened this issue 2 years ago • 2 comments

Describe the bug TikTok got rid of #SIGI_STATE and replaced it with #__UNIVERSAL_DATA_FOR_REHYDRATION__, now the data exists in ex /__DEFAULT_SCOPE__/webapp.video-detail/itemInfo and has different keys so the mapper doesn't work anymore.

To Reproduce Steps to reproduce the behavior:

  1. Scrape any page. Timeout occurs because SIGI_STATE isn't attached.

Expected behavior Well, you know...

Version Information Please include what versions of pydantic, playwright, and tiktokapipy you have installed (can be found with pip freeze).

playwright==1.30.0
playwright-stealth==1.0.5
pydantic==2.5.2
pydantic_core==2.14.5
tiktokapipy==0.2.4.post2

System Information Please provide information about the system you're running the library on. What's the OS? Is it a desktop, server, VM, or container? Various linux (ubuntu/debian) machines Region Information Where in the world are you located? If you're not in the US, it's likely that TikTok treats your region differently, which may present some challenges. US Additional context Add any other context about the problem here.

bobcogneato avatar Dec 10 '23 14:12 bobcogneato

As of today, i only see static and user data in the new state object, eg __$UNIVERSAL_DATA$__.__DEFAULT_SCOPE__["webapp.user-detail"] but no webapp.video-detail. Video data should be gathered through the DeferredItemListIterator(self._api, "post", self.sec_uid) magic anyways, so shmaybe not a big change needed?

RobertModus avatar Dec 14 '23 13:12 RobertModus

I'll need some time to address this. At the moment, I'm pretty busy with research and various applications, but I'll try to get to this soon.

Russell-Newton avatar Dec 14 '23 19:12 Russell-Newton