UltimaScraper icon indicating copy to clipboard operation
UltimaScraper copied to clipboard

"TypeError: object list can't be used in 'await' expression" in asyncio.py

Open yannick00237 opened this issue 4 years ago • 25 comments

I keep receiving the same error on every run:

Scrape Processing
Name: yaya
Scrape: 0 = All | 1 = Images | 2 = Videos | 3 = Audios | 4 = Texts
0
Type: Profile
Type: Stories
No Stories Found.
Type: Posts
Scrape Attempt: 1/100
Type: Archived Posts
Scrape Attempt: 1/100
Processing Scraped Posts
  0%|                                                                                            | 0/1 [00:00<?, ?it/s]Traceback (most recent call last):
  File "E:\Data\Downloads\OnlyFans-7.4.1\OnlyFans-7.4.1\start_ofd.py", line 60, in <module>
    loop.run_until_complete(main())
  File "C:\Python39\lib\asyncio\base_events.py", line 642, in run_until_complete
    return future.result()
  File "E:\Data\Downloads\OnlyFans-7.4.1\OnlyFans-7.4.1\start_ofd.py", line 44, in main
    api = await main_datascraper.start_datascraper(
  File "E:\Data\Downloads\OnlyFans-7.4.1\OnlyFans-7.4.1\datascraper\main_datascraper.py", line 93, in start_datascraper
    await main_helper.process_names(
  File "E:\Data\Downloads\OnlyFans-7.4.1\OnlyFans-7.4.1\helpers\main_helper.py", line 939, in process_names
    result = await module.start_datascraper(authed, username, site_name)
  File "E:\Data\Downloads\OnlyFans-7.4.1\OnlyFans-7.4.1\modules\onlyfans.py", line 160, in start_datascraper
    await prepare_scraper(authed, site_name, item)
  File "E:\Data\Downloads\OnlyFans-7.4.1\OnlyFans-7.4.1\modules\onlyfans.py", line 823, in prepare_scraper
    unrefined_set = await tqdm.gather(tasks, **settings)
  File "C:\Python39\lib\site-packages\tqdm\asyncio.py", line 79, in gather
    res = [await f for f in cls.as_completed(ifs, loop=loop, timeout=timeout,
  File "C:\Python39\lib\site-packages\tqdm\asyncio.py", line 79, in <listcomp>
    res = [await f for f in cls.as_completed(ifs, loop=loop, timeout=timeout,
  File "C:\Python39\lib\asyncio\tasks.py", line 614, in _wait_for_one
    return f.result()  # May raise f.exception().
  File "C:\Python39\lib\site-packages\tqdm\asyncio.py", line 76, in wrap_awaitable
    return i, await f
TypeError: object list can't be used in 'await' expression

Can someone help me please?

yannick00237 avatar Aug 10 '21 16:08 yannick00237

Having the same issue as well :/

Lateralusularetal avatar Aug 10 '21 22:08 Lateralusularetal

Make sure you have updated tqdm to version 4.62.0 or greater.

LunarPenguin avatar Aug 11 '21 11:08 LunarPenguin

Make sure you have updated tqdm to version 4.62.0 or greater.

Shouldn't tqdm be updated if you use the pip install option? I am having the same issue with this.

7uAip6 avatar Aug 12 '21 14:08 7uAip6

Shouldn't tqdm be updated if you use the pip install option?

yes it should but if you update the script and forget to update the library things can break. I was just making sure that was not the problem

LunarPenguin avatar Aug 12 '21 14:08 LunarPenguin

Shouldn't tqdm be updated if you use the pip install option?

yes it should but if you update the script and forget to update the library things can break. I was just making sure that was not the problem

OK, so there's still no real fix yet?

7uAip6 avatar Aug 12 '21 16:08 7uAip6

OK, so there's still no real fix yet?

no, not yet

LunarPenguin avatar Aug 12 '21 19:08 LunarPenguin

I'm with the same issue.

Isbbrt avatar Aug 12 '21 23:08 Isbbrt

same, same (my tqdm version is 4.62.0) . Scraper worked for me as recently July 29th :(

horatiorosa avatar Aug 13 '21 17:08 horatiorosa

I'm having the same issue, my tqdm is up to date:

  File "K:\OnlyFans-7.4.1\start_ofd.py", line 44, in main
    api = await main_datascraper.start_datascraper(
  File "K:\OnlyFans-7.4.1\datascraper\main_datascraper.py", line 93, in start_datascraper
    await main_helper.process_names(
  File "K:\OnlyFans-7.4.1\helpers\main_helper.py", line 939, in process_names
    result = await module.start_datascraper(authed, username, site_name)
  File "K:\OnlyFans-7.4.1\modules\onlyfans.py", line 160, in start_datascraper
    await prepare_scraper(authed, site_name, item)
  File "K:\OnlyFans-7.4.1\modules\onlyfans.py", line 823, in prepare_scraper
    unrefined_set = await tqdm.gather(tasks, **settings)
  File "C:\Users\p\AppData\Roaming\Python\Python39\site-packages\tqdm\asyncio.py", line 79, in gather
    res = [await f for f in cls.as_completed(ifs, loop=loop, timeout=timeout,
  File "C:\Users\p\AppData\Roaming\Python\Python39\site-packages\tqdm\asyncio.py", line 79, in <listcomp>
    res = [await f for f in cls.as_completed(ifs, loop=loop, timeout=timeout,
  File "C:\Users\p\AppData\Local\Programs\Python\Python39\lib\asyncio\tasks.py", line 614, in _wait_for_one
    return f.result()  # May raise f.exception().
  File "C:\Users\p\AppData\Roaming\Python\Python39\site-packages\tqdm\asyncio.py", line 76, in wrap_awaitable
    return i, await f
TypeError: object list can't be used in 'await' expression

McFork avatar Aug 14 '21 16:08 McFork

I found the solution in another thread:

  • In the requirements.txt, replace the line tqdm with tqdm==4.59.0 to use an older and more stable version
  • Run the command pip uninstall tqdm
  • Then rerun the command pip install --upgrade --user -r requirements.txt

It worked for me, it should work for you too.

McFork avatar Aug 14 '21 17:08 McFork

I found the solution in another thread:

  • In the requirements.txt, replace the line tqdm with tqdm==4.59.0 to use an older and more stable version
  • Run the command pip uninstall tqdm
  • Then rerun the command pip install --upgrade --user -r requirements.txt

It worked for me, it should work for you too.

Thanks...I was able to get farther, but now I'm having a new issue (probably unrelated):

Traceback (most recent call last):
  File "S:\OnlyFans\start_ofd.py", line 60, in <module>
    loop.run_until_complete(main())
  File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.9_3.9.1776.0_x64__qbz5n2kfra8p0\lib\asyncio\base_events.py", line 642, in run_until_complete
    return future.result()
  File "S:\OnlyFans\start_ofd.py", line 44, in main
    api = await main_datascraper.start_datascraper(
  File "S:\OnlyFans\datascraper\main_datascraper.py", line 93, in start_datascraper
    await main_helper.process_names(
  File "S:\OnlyFans\helpers\main_helper.py", line 939, in process_names
    result = await module.start_datascraper(authed, username, site_name)
  File "S:\OnlyFans\modules\onlyfans.py", line 160, in start_datascraper
    await prepare_scraper(authed, site_name, item)
  File "S:\OnlyFans\modules\onlyfans.py", line 791, in prepare_scraper
    master_set = await subscription.get_posts()
  File "S:\OnlyFans\apis\onlyfans\classes\create_user.py", line 301, in get_posts
    final_results = [create_post(x, self) for x in results]
  File "S:\OnlyFans\apis\onlyfans\classes\create_user.py", line 301, in <listcomp>
    final_results = [create_post(x, self) for x in results]
  File "S:\OnlyFans\apis\onlyfans\classes\create_post.py", line 13, in __init__
    self.author = create_user.create_user(option["author"])
KeyError: 'author'

7uAip6 avatar Aug 14 '21 18:08 7uAip6

  • pip install --upgrade --user -r requirements.txt

this worked thanks

astonat avatar Aug 14 '21 19:08 astonat

I found the solution in another thread:

just to be clear, you installed an earlier version first and after that uninstalled that older version and reinstalled the newest version and that fixed the problem?

if that is the case i will recommended that troubleshooting step in any new issues.

Thanks...I was able to get farther, but now I'm having a new issue (probably unrelated):

That indeed is unrelated, look here #1419 for a possible workaround

LunarPenguin avatar Aug 15 '21 11:08 LunarPenguin

just to be clear, you installed an earlier version first and after that uninstalled that older version and reinstalled the newest version and that fixed the problem?

I uninstalled the latest version (the one that's currently installed from requirements.txt), and then edited requirements.txt and ran pip on it again to install the older version. You need the older version to be installed for the script to work, it's not compatible with the latest tqdm unfortunately it seems.

I think you can just run pip install tqdm==4.59.0 after uninstalling the default version and it would work too, no need to edit requirements.txt.

McFork avatar Aug 15 '21 16:08 McFork

ok, i still find it strange that the newest version of tqdm is causing these issues because the maintainer of tqdm personally opened this #1521 Pull Request in which he bumps the required version of tqdm to tqdm>=4.62.0

LunarPenguin avatar Aug 15 '21 17:08 LunarPenguin

Hmm, that is strange indeed, hopefully we can get more information from @DIGITALCRIMINAL

McFork avatar Aug 15 '21 18:08 McFork

Thanks y'all. I just ran, as suggested, pip install tqdm==4.59.0 after uninstalling tqdm v4.62.0. I also modified my Dockerfile to reference pip3 in the RUN command on line 6 and changed the CMD on line 10 to reference python3. Scraper works good as new. :-)

horatiorosa avatar Aug 16 '21 17:08 horatiorosa

I've faced the same issue multiple times on 7.4.1. Only workaround was to downgrade tqdm.

SafadezaSemLimites avatar Aug 20 '21 04:08 SafadezaSemLimites

Is tqdm absolutely necessary? I got the same error without having it installed at all. I assumed that the requirements listed were included in the downloads of the major requirements, Python and Visual Studio.

expressodepresso avatar Aug 26 '21 04:08 expressodepresso

It is a very nice feature so that we know what is going on. This has been fixed. Install from this repo for now.

americanseeder1865 avatar Aug 26 '21 07:08 americanseeder1865

Is tqdm absolutely necessary?

Yes, otherwise you will get (more) errors eventually

I assumed that the requirements listed were included in the downloads of the major major requirements.

Some of the most used libraries sometimes become part of the libraries included with python but most of them are not and are not build not maintained by the python core developers and have to be installed using pip

LunarPenguin avatar Aug 26 '21 09:08 LunarPenguin

Thanks for answering! Guess now I just have to figure out how to use pip. I'm new to all this so any suggestions/directions are welcome. I thought I could just download, unzip, and run tqdm like most things, but I guess it doesn't work that way.

expressodepresso avatar Aug 27 '21 04:08 expressodepresso

Thanks for answering! Guess now I just have to figure out how to use pip. I'm new to all this so any suggestions/directions are welcome. I thought I could just download, unzip, and run tqdm like most things, but I guess it doesn't work that way.

My suggestion is to use docker. This helps avoid dependency hell when it comes to requirements.

americanseeder1865 avatar Aug 27 '21 05:08 americanseeder1865

I thought I could just download, unzip, and run tqdm like most things, but I guess it doesn't work that way.

Pip is installed alongside python and can be invoked by one of the following ways

  • py -3 -m pip
  • python3 -m pip
  • python -m pip
  • pip
  • pip3

It's good practice to update pip before you install or update any packages, you can do so like this: py -3 -m pip install -U pip using the prefix from above that works for you.

After that you can install a single package like this: py -3 -m pip install -U <packagename> replacing <packagename> with the name of your package.

You can install all requirements from a requirements.txt file by running py -3 -m pip install -U -r requirements.txt

These are the basics and should at least get you most of the way there

LunarPenguin avatar Aug 27 '21 18:08 LunarPenguin

I found the solution in another thread:

  • In the requirements.txt, replace the line tqdm with tqdm==4.59.0 to use an older and more stable version
  • Run the command pip uninstall tqdm
  • Then rerun the command pip install --upgrade --user -r requirements.txt

It worked for me, it should work for you too.

Thanks, this worked for me too.

nasty23 avatar Nov 08 '21 20:11 nasty23