ArchivedUltimaScraper icon indicating copy to clipboard operation
ArchivedUltimaScraper copied to clipboard

KeyError: 340600631

Open god43 opened this issue 1 year ago • 8 comments

i am getting a weird error on one model, i think its because of the messages. the old 7.6.1 version will scrape the profile, but does not get all the ppv. thanks.

Processing Scraped Messages 0%| | 0/73 [00:00<?, ?it/s]Traceback (most recent call last): File "E:\OnlyFans-ultima\start_us.py", line 62, in asyncio.run(main()) File "C:\Users\user\AppData\Local\Programs\Python\Python310\lib\asyncio\runners.py", line 44, in run return loop.run_until_complete(main) File "C:\Users\user\AppData\Local\Programs\Python\Python310\lib\asyncio\base_events.py", line 649, in run_until_complete return future.result() File "E:\OnlyFans-ultima\start_us.py", line 44, in main _api = await USR.start( File "E:\OnlyFans-ultima\ultima_scraper\ultima_scraper.py", line 50, in start await self.start_datascraper(datascraper) File "E:\OnlyFans-ultima\ultima_scraper\ultima_scraper.py", line 137, in start_datascraper await datascraper.datascraper.api.job_manager.process_jobs() File "C:\Users\user\AppData\Local\pypoetry\Cache\virtualenvs\ultima-scraper-dOa14nUa-py3.10\lib\site-packages\ultima_scraper_api\managers\job_manager\job_manager.py", line 45, in process_jobs await asyncio.create_task(self.__worker()) File "C:\Users\user\AppData\Local\pypoetry\Cache\virtualenvs\ultima-scraper-dOa14nUa-py3.10\lib\site-packages\ultima_scraper_api\managers\job_manager\job_manager.py", line 53, in __worker await job.task File "C:\Users\user\AppData\Local\pypoetry\Cache\virtualenvs\ultima-scraper-dOa14nUa-py3.10\lib\site-packages\ultima_scraper_collection\modules\module_streamliner.py", line 202, in prepare_scraper await self.process_scraped_content( File "C:\Users\user\AppData\Local\pypoetry\Cache\virtualenvs\ultima-scraper-dOa14nUa-py3.10\lib\site-packages\ultima_scraper_collection\modules\module_streamliner.py", line 237, in process_scraped_content unrefined_set: list[dict[str, Any]] = await tqdm_asyncio.gather( File "C:\Users\user\AppData\Local\pypoetry\Cache\virtualenvs\ultima-scraper-dOa14nUa-py3.10\lib\site-packages\tqdm\asyncio.py", line 79, in gather res = [await f for f in cls.as_completed(ifs, loop=loop, timeout=timeout, File "C:\Users\user\AppData\Local\pypoetry\Cache\virtualenvs\ultima-scraper-dOa14nUa-py3.10\lib\site-packages\tqdm\asyncio.py", line 79, in res = [await f for f in cls.as_completed(ifs, loop=loop, timeout=timeout, File "C:\Users\user\AppData\Local\Programs\Python\Python310\lib\asyncio\tasks.py", line 571, in _wait_for_one return f.result() # May raise f.exception(). File "C:\Users\user\AppData\Local\pypoetry\Cache\virtualenvs\ultima-scraper-dOa14nUa-py3.10\lib\site-packages\tqdm\asyncio.py", line 76, in wrap_awaitable return i, await f File "C:\Users\user\AppData\Local\pypoetry\Cache\virtualenvs\ultima-scraper-dOa14nUa-py3.10\lib\site-packages\ultima_scraper_collection\managers\datascraper_manager\datascrapers\onlyfans.py", line 61, in media_scraper reformat_item = reformat_manager.prepare_reformat(asset) File "C:\Users\user\AppData\Local\pypoetry\Cache\virtualenvs\ultima-scraper-dOa14nUa-py3.10\lib\site-packages\ultima_scraper_renamer\reformat.py", line 372, in prepare_reformat download_path = self.filesystem_manager.get_directory_manager( File "C:\Users\user\AppData\Local\pypoetry\Cache\virtualenvs\ultima-scraper-dOa14nUa-py3.10\lib\site-packages\ultima_scraper_collection\managers\filesystem_manager.py", line 55, in get_directory_manager return self.directory_manager_users[user_id] KeyError: 340600631 53%|███████████████████████████████████████████▎ | 39/73 [00:00<00:00, 279.42it/s]

god43 avatar Jun 23 '23 17:06 god43

I was getting a very similar (possibly the exact same) error today, it is definitely the messages which is causing it (I was able to download everything else by selecting the API individually). Here's my traceback (the [PROJECTFOLDER] and [AUTHID] inserted by me to anonymize ... the number after the KeyError for me seemed to be the same as the 'auth_id' contained in the cookie entry for the auth.json file).

Processing Scraped Messages 0%| | 0/571 [00:00<?, ?it/s] Traceback (most recent call last): File "[PROJECTFOLDER]\Scraper-7.6.1\start_us.py", line 62, in asyncio.run(main()) File "C:\Program Files\Python311\Lib\asyncio\runners.py", line 190, in run return runner.run(main) ^^^^^^^^^^^^^^^^ File "C:\Program Files\Python311\Lib\asyncio\runners.py", line 118, in run return self._loop.run_until_complete(task) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Program Files\Python311\Lib\asyncio\base_events.py", line 653, in run_until_complete return future.result() ^^^^^^^^^^^^^^^ File "[PROJECTFOLDER]\Scraper-7.6.1\start_us.py", line 44, in main _api = await USR.start( ^^^^^^^^^^^^^^^^ File "[PROJECTFOLDER]\Scraper-7.6.1\ultima_scraper\ultima_scraper.py", line 50, in start await self.start_datascraper(datascraper) File "[PROJECTFOLDER]\Scraper-7.6.1\ultima_scraper\ultima_scraper.py", line 137, in start_datascraper await datascraper.datascraper.api.job_manager.process_jobs() File "[PROJECTFOLDER]\pypoetry\Cache\virtualenvs\ultima-scraper-5zq_mRi5-py3.11\Lib\site-packages\ultima_scraper_api\managers\job_manager\job_manager.py", line 45, in process_jobs await asyncio.create_task(self.__worker()) File "[PROJECTFOLDER]\pypoetry\Cache\virtualenvs\ultima-scraper-5zq_mRi5-py3.11\Lib\site-packages\ultima_scraper_api\managers\job_manager\job_manager.py", line 53, in __worker await job.task File "[PROJECTFOLDER]\pypoetry\Cache\virtualenvs\ultima-scraper-5zq_mRi5-py3.11\Lib\site-packages\ultima_scraper_collection\modules\module_streamliner.py", line 202, in prepare_scraper await self.process_scraped_content( File "[PROJECTFOLDER]\pypoetry\Cache\virtualenvs\ultima-scraper-5zq_mRi5-py3.11\Lib\site-packages\ultima_scraper_collection\modules\module_streamliner.py", line 237, in process_scraped_content unrefined_set: list[dict[str, Any]] = await tqdm_asyncio.gather( ^^^^^^^^^^^^^^^^^^^^^^^^^^ File "[PROJECTFOLDER]\pypoetry\Cache\virtualenvs\ultima-scraper-5zq_mRi5-py3.11\Lib\site-packages\tqdm\asyncio.py", line 79, in gather res = [await f for f in cls.as_completed(ifs, loop=loop, timeout=timeout, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "[PROJECTFOLDER]\pypoetry\Cache\virtualenvs\ultima-scraper-5zq_mRi5-py3.11\Lib\site-packages\tqdm\asyncio.py", line 79, in res = [await f for f in cls.as_completed(ifs, loop=loop, timeout=timeout, ^^^^^^^ File "C:\Program Files\Python311\Lib\asyncio\tasks.py", line 605, in _wait_for_one return f.result() # May raise f.exception(). ^^^^^^^^^^ File "[PROJECTFOLDER]\pypoetry\Cache\virtualenvs\ultima-scraper-5zq_mRi5-py3.11\Lib\site-packages\tqdm\asyncio.py", line 76, in wrap_awaitable return i, await f ^^^^^^^ File "[PROJECTFOLDER]\pypoetry\Cache\virtualenvs\ultima-scraper-5zq_mRi5-py3.11\Lib\site-packages\ultima_scraper_collection\managers\datascraper_manager\datascrapers\onlyfans.py", line 61, in media_scraper reformat_item = reformat_manager.prepare_reformat(asset) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "[PROJECTFOLDER]\pypoetry\Cache\virtualenvs\ultima-scraper-5zq_mRi5-py3.11\Lib\site-packages\ultima_scraper_renamer\reformat.py", line 372, in prepare_reformat download_path = self.filesystem_manager.get_directory_manager( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "[PROJECTFOLDER]\pypoetry\Cache\virtualenvs\ultima-scraper-5zq_mRi5-py3.11\Lib\site-packages\ultima_scraper_collection\managers\filesystem_manager.py", line 55, in get_directory_manager return self.directory_manager_users[user_id] ~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^ KeyError: [AUTHID] 9%|███████▍ | 52/571 [00:00<00:05, 90.04it/s]

billmonti2346 avatar Jun 23 '23 20:06 billmonti2346

I'm receiving very similar errors:

image

Could anybody help us?

Thx.

djmarioka avatar Jun 24 '23 09:06 djmarioka

I'm getting this too

Edit: I had to turn off paid content and it stops giving me error but it doesn't download all vids, some models it does and some it doesn't

TheMissingPort avatar Jun 24 '23 23:06 TheMissingPort

https://github.com/DIGITALCRIMINALS/UltimaScraper/issues/1004

The same problem, I also tried a pure reset, not a legacy of the 7.6.1 version of the problem

xdf000 avatar Jun 25 '23 06:06 xdf000

I think I have the same error?

Traceback (most recent call last): File "C:\Users\Matt\Desktop\UltimaScraper-master\start_us.py", line 62, in asyncio.run(main()) File "C:\Program Files\Python310\lib\asyncio\runners.py", line 44, in run return loop.run_until_complete(main) File "C:\Program Files\Python310\lib\asyncio\base_events.py", line 649, in run_until_complete return future.result() File "C:\Users\Matt\Desktop\UltimaScraper-master\start_us.py", line 44, in main _api = await USR.start( File "C:\Users\Matt\Desktop\UltimaScraper-master\ultima_scraper\ultima_scraper.py", line 50, in start await self.start_datascraper(datascraper) File "C:\Users\Matt\Desktop\UltimaScraper-master\ultima_scraper\ultima_scraper.py", line 137, in start_datascraper await datascraper.datascraper.api.job_manager.process_jobs() File "C:\Users\Matt\AppData\Local\pypoetry\Cache\virtualenvs\ultima-scraper-yfGR4bq4-py3.10\lib\site-packages\ultima_scraper_api\managers\job_manager\job_manager.py", line 45, in process_jobs await asyncio.create_task(self.__worker()) File "C:\Users\Matt\AppData\Local\pypoetry\Cache\virtualenvs\ultima-scraper-yfGR4bq4-py3.10\lib\site-packages\ultima_scraper_api\managers\job_manager\job_manager.py", line 53, in __worker await job.task File "C:\Users\Matt\AppData\Local\pypoetry\Cache\virtualenvs\ultima-scraper-yfGR4bq4-py3.10\lib\site-packages\ultima_scraper_collection\modules\module_streamliner.py", line 202, in prepare_scraper await self.process_scraped_content( File "C:\Users\Matt\AppData\Local\pypoetry\Cache\virtualenvs\ultima-scraper-yfGR4bq4-py3.10\lib\site-packages\ultima_scraper_collection\modules\module_streamliner.py", line 237, in process_scraped_content unrefined_set: list[dict[str, Any]] = await tqdm_asyncio.gather( File "C:\Users\Matt\AppData\Local\pypoetry\Cache\virtualenvs\ultima-scraper-yfGR4bq4-py3.10\lib\site-packages\tqdm\asyncio.py", line 79, in gather res = [await f for f in cls.as_completed(ifs, loop=loop, timeout=timeout, File "C:\Users\Matt\AppData\Local\pypoetry\Cache\virtualenvs\ultima-scraper-yfGR4bq4-py3.10\lib\site-packages\tqdm\asyncio.py", line 79, in res = [await f for f in cls.as_completed(ifs, loop=loop, timeout=timeout, File "C:\Program Files\Python310\lib\asyncio\tasks.py", line 571, in _wait_for_one return f.result() # May raise f.exception(). File "C:\Users\Matt\AppData\Local\pypoetry\Cache\virtualenvs\ultima-scraper-yfGR4bq4-py3.10\lib\site-packages\tqdm\asyncio.py", line 76, in wrap_awaitable return i, await f File "C:\Users\Matt\AppData\Local\pypoetry\Cache\virtualenvs\ultima-scraper-yfGR4bq4-py3.10\lib\site-packages\ultima_scraper_collection\managers\datascraper_manager\datascrapers\onlyfans.py", line 61, in media_scraper reformat_item = reformat_manager.prepare_reformat(asset) File "C:\Users\Matt\AppData\Local\pypoetry\Cache\virtualenvs\ultima-scraper-yfGR4bq4-py3.10\lib\site-packages\ultima_scraper_renamer\reformat.py", line 372, in prepare_reformat download_path = self.filesystem_manager.get_directory_manager( File "C:\Users\Matt\AppData\Local\pypoetry\Cache\virtualenvs\ultima-scraper-yfGR4bq4-py3.10\lib\site-packages\ultima_scraper_collection\managers\filesystem_manager.py", line 55, in get_directory_manager return self.directory_manager_users[user_id] Keyerror: <My Auth ID>

Prophaniti86 avatar Jun 27 '23 04:06 Prophaniti86

I turned off paid content and message and it seems to work now. I think it misses some items though

EDIT: Seems to crash on a very specific profile now.

Prophaniti86 avatar Jun 27 '23 06:06 Prophaniti86

same here

twist3dimages avatar Jun 30 '23 17:06 twist3dimages

Well ok, but I want to download my paid content and messages!

I turned off paid content and message and it seems to work now. I think it misses some items though

Are you saying that paid content and/or messages are working generally now for you, just not for a specific profile?

EDIT: Seems to crash on a very specific profile now.

billmonti2346 avatar Jul 04 '23 21:07 billmonti2346