UltimaScraper icon indicating copy to clipboard operation
UltimaScraper copied to clipboard

NOT WORKING IN DOCKER - traceback error 'values = input().strip().split(",")' in main_helper.py

Open KptnKMan opened this issue 3 years ago • 20 comments

Hi, I seem to be experiencing a strange traceback that I'm having trouble troubleshooting. I don't think it's an authentication issue, but I could be wrong. It appears to authenticate and list usernames. I am running in docker.

Apologies in advance if this is something simple I've missed. Can anyone help?

Full error:

Session IP: hidden

Auth (V1) Attempt 1/10
Welcome user | user
Scraping Paid Content
Scraping - profileA | 0 / 7
Scraping - profileB | 1 / 7
Scraping Subscriptions
Names: Username = username | 0 = All | 1 = profileA | 2 = profileB
Traceback (most recent call last):
File "/usr/src/app/./start_ofd.py", line 43, in <module>
apis = main_datascraper.start_datascraper(json_config, site_name_lower)
File "/usr/src/app/datascraper/main_datascraper.py", line 104, in start_datascraper
names = main_helper.process_names(
File "/usr/src/app/helpers/main_helper.py", line 622, in process_names
names = choose_option(
File "/usr/src/app/helpers/main_helper.py", line 559, in choose_option
values = input().strip().split(",")
EOFError: EOF when reading a line

Traceback (most recent call last):
File "/usr/src/app/./start_ofd.py", line 43, in <module>
apis = main_datascraper.start_datascraper(json_config, site_name_lower)
File "/usr/src/app/datascraper/main_datascraper.py", line 104, in start_datascraper
names = main_helper.process_names(
File "/usr/src/app/helpers/main_helper.py", line 622, in process_names
names = choose_option(
File "/usr/src/app/helpers/main_helper.py", line 559, in choose_option
values = input().strip().split(",")
EOFError: EOF when reading a line

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/usr/src/app/./start_ofd.py", line 55, in <module>
input()
EOFError: EOF when reading a line

KptnKMan avatar Apr 08 '21 11:04 KptnKMan

With some testing, I think I've managed to determine that the issue seems related to the /usr/src/app/datascraper/main_datascraper.py file, and the processing of names thein.

in the config.json file, it seems that the issue is related to the auto_profile_choice setting, and I'm unsure what needs to be placed here. I'm not able to find any reference to this setting anywhere and what should be there, apart from in other tickets and in errors.

in the config.json file, I get the originally reported error if I set a number:

  "supported": {
    "onlyfans": {
      "settings": {
        "auto_profile_choice": "0",
        "auto_scrape_names": false,
        "auto_choice": "a",
        "auto_scrape_apis": true,

response:

Session IP: hidden

Auth (V1) Attempt 1/10
Welcome username | username
Scraping Paid Content
Scraping - profileA | 0 / 7
Scraping - profileB | 1 / 7
Scraping Subscriptions
Names: Username = username | 0 = All | 1 = profileA | 2 = profileB
Traceback (most recent call last):
File "/usr/src/app/./start_ofd.py", line 43, in <module>
apis = main_datascraper.start_datascraper(json_config, site_name_lower)
File "/usr/src/app/datascraper/main_datascraper.py", line 104, in start_datascraper
names = main_helper.process_names(
File "/usr/src/app/helpers/main_helper.py", line 622, in process_names
names = choose_option(
File "/usr/src/app/helpers/main_helper.py", line 559, in choose_option
values = input().strip().split(",")
EOFError: EOF when reading a line

in the same config.json file, I get an error "There's nothing to scrape." if I set "All" or a profile name:

  "supported": {
    "onlyfans": {
      "settings": {
        "auto_profile_choice": "All",
        "auto_scrape_names": false,
        "auto_choice": "a",
        "auto_scrape_apis": true,

response:

Session IP: hidden

Scraping Paid Content
Scraping Subscriptions
There's nothing to scrape.
Archive Completed in 0.0 Minutes
Now exiting.

Interestingly, it seems that the if-chain of listing profiles, is not initiated, when setting a word.

There is currently an issue #855 that seems related to this, and references the same error and code.

From this, I conclude that a number should be set, but there is an issue parsing something. I'm trying to figure out what that is.

Has anyone else been able to replicate?

KptnKMan avatar Apr 08 '21 16:04 KptnKMan

Try using 0 instead of "0" for the auto_profile_choice

UltimaHoarder avatar Apr 08 '21 16:04 UltimaHoarder

@DIGITALCRIMINAL sorry I misread your response, I replaced string "0" (in quotes) with a int 0 (without quotes), and got some different results...

Also tried specifying 1...

response using 0:

Session IP: hidden

Choose Profile
Names: Username = username | 0 = All | 1 = default
Traceback (most recent call last):
File "/usr/src/app/./start_ofd.py", line 43, in <module>
apis = main_datascraper.start_datascraper(json_config, site_name_lower)
File "/usr/src/app/datascraper/main_datascraper.py", line 71, in start_datascraper
apis = choose_option(
File "/usr/src/app/helpers/main_helper.py", line 564, in choose_option
values = input().strip().split(",")
EOFError: EOF when reading a line

Traceback (most recent call last):
File "/usr/src/app/./start_ofd.py", line 43, in <module>
apis = main_datascraper.start_datascraper(json_config, site_name_lower)
File "/usr/src/app/datascraper/main_datascraper.py", line 71, in start_datascraper
apis = choose_option(
File "/usr/src/app/helpers/main_helper.py", line 564, in choose_option
values = input().strip().split(",")
EOFError: EOF when reading a line

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/usr/src/app/./start_ofd.py", line 55, in <module>
input()
EOFError: EOF when reading a line

reponse using 1:

Session IP: hidden

Traceback (most recent call last):
File "/usr/src/app/./start_ofd.py", line 43, in <module>
Traceback (most recent call last):
File "/usr/src/app/./start_ofd.py", line 43, in <module>
apis = main_datascraper.start_datascraper(json_config, site_name_lower)
File "/usr/src/app/datascraper/main_datascraper.py", line 71, in start_datascraper
apis = main_datascraper.start_datascraper(json_config, site_name_lower)
apis = choose_option(
File "/usr/src/app/helpers/main_helper.py", line 566, in choose_option
File "/usr/src/app/datascraper/main_datascraper.py", line 71, in start_datascraper
apis = choose_option(
File "/usr/src/app/helpers/main_helper.py", line 566, in choose_option
values = auto_scrape.split(",")
AttributeError: 'int' object has no attribute 'split'

values = auto_scrape.split(",")
AttributeError: 'int' object has no attribute 'split'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/usr/src/app/./start_ofd.py", line 55, in <module>
input()
EOFError: EOF when reading a line

KptnKMan avatar Apr 08 '21 16:04 KptnKMan

Looking closer at the choose_option function, I dont fully understand everything, but is there something missing with the new_names dict? It doesn't seem to be updated, but again, I'm not sure of everything happening.

https://github.com/DIGITALCRIMINAL/OnlyFans/blob/fdf65509c09d5e58f2fbe8138ac1edd3acdffa9b/helpers/main_helper.py#L548-L579

Also, something seems wrong with the line declaring:

values = input().strip().split(",")

KptnKMan avatar Apr 08 '21 16:04 KptnKMan

try replacing the "auto_profile_choice": "0" to "auto_profile_choice": "default"

LunarPenguin avatar Apr 12 '21 21:04 LunarPenguin

Thanks, I just tried, and get the same traceback from the first post:

Traceback (most recent call last):
File "/usr/src/app/./start_ofd.py", line 43, in <module>
apis = main_datascraper.start_datascraper(json_config, site_name_lower)
File "/usr/src/app/datascraper/main_datascraper.py", line 104, in start_datascraper
names = main_helper.process_names(
File "/usr/src/app/helpers/main_helper.py", line 622, in process_names
names = choose_option(
Traceback (most recent call last):
File "/usr/src/app/./start_ofd.py", line 43, in <module>
File "/usr/src/app/helpers/main_helper.py", line 559, in choose_option
values = input().strip().split(",")
EOFError: EOF when reading a line

apis = main_datascraper.start_datascraper(json_config, site_name_lower)
File "/usr/src/app/datascraper/main_datascraper.py", line 104, in start_datascraper
names = main_helper.process_names(
File "/usr/src/app/helpers/main_helper.py", line 622, in process_names
names = choose_option(
File "/usr/src/app/helpers/main_helper.py", line 559, in choose_option
values = input().strip().split(",")
EOFError: EOF when reading a line

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/usr/src/app/./start_ofd.py", line 55, in <module>
input()
EOFError: EOF when reading a line

KptnKMan avatar Apr 12 '21 22:04 KptnKMan

Updated to the latest version, and still getting the same error. I'm using a reproducible environment built directly off the root Dockerfile

Is anyone able to help?

KptnKMan avatar Apr 18 '21 22:04 KptnKMan

Does this problem still occur?

UltimaHoarder avatar May 03 '21 12:05 UltimaHoarder

Hi, yes still getting this error unfortunately.

I'm using the Chrome Cookie Helper to generate auth data, which seems to be working. I'm running an unmodified automated build from DockerHub.

I updated my docker image to the latest v7 changes, and still seeing the same below (taken today):

Session IP: Hidden

Auth (V1) Attempt 1/10
Welcome user | user
Scraping Paid Content
Scraping - ProfileA | 1 / 6
Scraping - ProfileB | 2 / 6
Scraping - ProfileC | 3 / 6
Scraping - ProfileD | 4 / 6
Scraping - ProfileE | 5 / 6
Scraping - ProfileF | 6 / 6
Scraping Subscriptions
Names: Username = username | 0 = All | 1 = ProfileA | 2 = ProfileB | 3 = ProfileC | 4 = ProfileD | 5 = ProfileE | 6 = ProfileF
Traceback (most recent call last):
File "/usr/src/app/./start_ofd.py", line 43, in <module>
apis = main_datascraper.start_datascraper(json_config, site_name_lower)
File "/usr/src/app/datascraper/main_datascraper.py", line 105, in start_datascraper
names = main_helper.process_names(
File "/usr/src/app/helpers/main_helper.py", line 618, in process_names
names = choose_option(
Traceback (most recent call last):
File "/usr/src/app/helpers/main_helper.py", line 557, in choose_option
File "/usr/src/app/./start_ofd.py", line 43, in <module>
values = input().strip().split(",")
EOFError: EOF when reading a line

apis = main_datascraper.start_datascraper(json_config, site_name_lower)
File "/usr/src/app/datascraper/main_datascraper.py", line 105, in start_datascraper
names = main_helper.process_names(
File "/usr/src/app/helpers/main_helper.py", line 618, in process_names
names = choose_option(
File "/usr/src/app/helpers/main_helper.py", line 557, in choose_option
values = input().strip().split(",")
EOFError: EOF when reading a line

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/usr/src/app/./start_ofd.py", line 55, in <module>
input()
EOFError: EOF when reading a line

Any help would be appreciated.

KptnKMan avatar May 03 '21 13:05 KptnKMan

Additionally, response today when setting 0 instead of "0" for the auto_profile_choice (Nothing else modified):

Session IP: Hidden

Choose Profile
Names: Username = username | 0 = All | 1 = default
Traceback (most recent call last):
File "/usr/src/app/./start_ofd.py", line 43, in <module>
apis = main_datascraper.start_datascraper(json_config, site_name_lower)
File "/usr/src/app/datascraper/main_datascraper.py", line 70, in start_datascraper
auths = choose_option(
File "/usr/src/app/helpers/main_helper.py", line 562, in choose_option
Traceback (most recent call last):
File "/usr/src/app/./start_ofd.py", line 43, in <module>
values = input().strip().split(",")
EOFError: EOF when reading a line

apis = main_datascraper.start_datascraper(json_config, site_name_lower)
File "/usr/src/app/datascraper/main_datascraper.py", line 70, in start_datascraper
auths = choose_option(
File "/usr/src/app/helpers/main_helper.py", line 562, in choose_option
values = input().strip().split(",")
EOFError: EOF when reading a line

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/usr/src/app/./start_ofd.py", line 55, in <module>
input()
EOFError: EOF when reading a line
Session IP: Hidden

Choose Profile
Names: Username = username | 0 = All | 1 = default
Traceback (most recent call last):
File "/usr/src/app/./start_ofd.py", line 43, in <module>
apis = main_datascraper.start_datascraper(json_config, site_name_lower)
File "/usr/src/app/datascraper/main_datascraper.py", line 70, in start_datascraper
auths = choose_option(
File "/usr/src/app/helpers/main_helper.py", line 562, in choose_option
values = input().strip().split(",")
EOFError: EOF when reading a line

Traceback (most recent call last):
File "/usr/src/app/./start_ofd.py", line 43, in <module>
apis = main_datascraper.start_datascraper(json_config, site_name_lower)
File "/usr/src/app/datascraper/main_datascraper.py", line 70, in start_datascraper
auths = choose_option(
File "/usr/src/app/helpers/main_helper.py", line 562, in choose_option
values = input().strip().split(",")
EOFError: EOF when reading a line

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/usr/src/app/./start_ofd.py", line 55, in <module>
input()
EOFError: EOF when reading a line

KptnKMan avatar May 03 '21 13:05 KptnKMan

It's probably something to do with your terminal.

UltimaHoarder avatar May 11 '21 03:05 UltimaHoarder

It's probably something to do with your terminal.

I highly doubt that because as I've said, I'm building this as a docker image direct from your Dockerfile as an automated build.

My terminal would be your terminal, not much I can say about that. All the logs I posted are from the docker.

@DIGITALCRIMINAL have you been able to run this successfully as a docker instance?

KptnKMan avatar May 11 '21 08:05 KptnKMan

It's probably something to do with your terminal.

I highly doubt that because as I've said, I'm building this as a docker image direct from your Dockerfile as an automated build.

My terminal would be your terminal, not much I can say about that. All the logs I posted are from the docker.

@DIGITALCRIMINAL have you been able to run this successfully as a docker instance?

I don't use docker, the tutorial was made by someone else who uses it. I just use Windows. Just make sure you're inputting the correct inputs.

UltimaHoarder avatar May 11 '21 08:05 UltimaHoarder

Oh ok, this makes a lot more sense then.

I'm sure I'm using the correct inputs, and can modify behaviour by changing the config parameters (As mentioned in this ticket/issue) but I've been taking looks at the code (When available time) and trying to understand further. I've been using the recommended cookie helper with good results, and I can see successful authentication (Also mentioned here).

I try to run services in Docker where possible, makes things cleaner and reproduceable. That being said, you could also use Docker for Windows and test your code/build, which would make this 1:1.

I would be happy to collaborate closer (If you're willing) and test automated docker builds, this might help a lot of people.

KptnKMan avatar May 11 '21 08:05 KptnKMan

I'm having the same (or very similar) issue. It occurs whenever Profile #1 is selected, be it by setting "auto_model_choice": true, or selecting 0 or 1 in the list of profiles.

Profile 1 happens to be my own profile; I assume this is standard?

Names: Username = username | 0 = All | 1 = Me | 2 = Model

Scraping [photo]. Should take less than a minute. Traceback (most recent call last): File "C:\Users\Me\Downloads\OnlyFans-master\start_ofd.py", line 44, in api = main_datascraper.start_datascraper(json_config, site_name_lower) File "C:\Users\Me\Downloads\OnlyFans-master\datascraper\main_datascraper.py", line 105, in start_datascraper names = main_helper.process_names( File "C:\Users\Me\Downloads\OnlyFans-master\helpers\main_helper.py", line 641, in process_names result = module.start_datascraper( File "C:\Users\Me\Downloads\OnlyFans-master\modules\onlyfans.py", line 159, in start_datascraper results = prepare_scraper(authed, site_name, item) File "C:\Users\Me\Downloads\OnlyFans-master\modules\onlyfans.py", line 778, in prepare_scraper unrefined_result = pool.starmap( File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.9_3.9.1520.0_x64__qbz5n2kfra8p0\lib\multiprocessing\pool.py", line 372, in starmap return self._map_async(func, iterable, starmapstar, chunksize).get() File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.9_3.9.1520.0_x64__qbz5n2kfra8p0\lib\multiprocessing\pool.py", line 771, in get raise self._value File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.9_3.9.1520.0_x64__qbz5n2kfra8p0\lib\multiprocessing\pool.py", line 125, in worker result = (True, func(*args, **kwds)) File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.9_3.9.1520.0_x64__qbz5n2kfra8p0\lib\multiprocessing\pool.py", line 51, in starmapstar return list(itertools.starmap(args[0], args[1])) File "C:\Users\Me\Downloads\OnlyFans-master\modules\onlyfans.py", line 1036, in media_scraper post_id = post_result.id AttributeError: 'list' object has no attribute 'id'

outsdr01 avatar May 31 '21 02:05 outsdr01

Oh ok, this makes a lot more sense then.

I'm sure I'm using the correct inputs, and can modify behaviour by changing the config parameters (As mentioned in this ticket/issue) but I've been taking looks at the code (When available time) and trying to understand further. I've been using the recommended cookie helper with good results, and I can see successful authentication (Also mentioned here).

I try to run services in Docker where possible, makes things cleaner and reproduceable. That being said, you could also use Docker for Windows and test your code/build, which would make this 1:1.

I would be happy to collaborate closer (If you're willing) and test automated docker builds, this might help a lot of people.

I can't, I'm a VMWare user

UltimaHoarder avatar Jun 13 '21 02:06 UltimaHoarder

I'm having the same (or very similar) issue. It occurs whenever Profile #1 is selected, be it by setting "auto_model_choice": true, or selecting 0 or 1 in the list of profiles.

Profile 1 happens to be my own profile; I assume this is standard?

Names: Username = username | 0 = All | 1 = Me | 2 = Model

Scraping [photo]. Should take less than a minute. Traceback (most recent call last): File "C:\Users\Me\Downloads\OnlyFans-master\start_ofd.py", line 44, in api = main_datascraper.start_datascraper(json_config, site_name_lower) File "C:\Users\Me\Downloads\OnlyFans-master\datascraper\main_datascraper.py", line 105, in start_datascraper names = main_helper.process_names( File "C:\Users\Me\Downloads\OnlyFans-master\helpers\main_helper.py", line 641, in process_names result = module.start_datascraper( File "C:\Users\Me\Downloads\OnlyFans-master\modules\onlyfans.py", line 159, in start_datascraper results = prepare_scraper(authed, site_name, item) File "C:\Users\Me\Downloads\OnlyFans-master\modules\onlyfans.py", line 778, in prepare_scraper unrefined_result = pool.starmap( File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.9_3.9.1520.0_x64__qbz5n2kfra8p0\lib\multiprocessing\pool.py", line 372, in starmap return self._map_async(func, iterable, starmapstar, chunksize).get() File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.9_3.9.1520.0_x64__qbz5n2kfra8p0\lib\multiprocessing\pool.py", line 771, in get raise self._value File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.9_3.9.1520.0_x64__qbz5n2kfra8p0\lib\multiprocessing\pool.py", line 125, in worker result = (True, func(*args, **kwds)) File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.9_3.9.1520.0_x64__qbz5n2kfra8p0\lib\multiprocessing\pool.py", line 51, in starmapstar return list(itertools.starmap(args[0], args[1])) File "C:\Users\Me\Downloads\OnlyFans-master\modules\onlyfans.py", line 1036, in media_scraper post_id = post_result.id AttributeError: 'list' object has no attribute 'id'

Latest commit would fix it

UltimaHoarder avatar Jun 13 '21 02:06 UltimaHoarder

@DIGITALCRIMINAL just saying, the comment from @outsdr01 that you closed this issue for, is not the issue that this issue/ticket was opened for. Not trying to invalidate his comment, but its not the same issue, from what I can see. If possible, it would be good to have this reopened, but I'm not going to hold hope for that happening.

Anyway, I updated to the latest 7.3 release and redeployed a docker, and now I have a different repeating error:

If you're not using >= v7 release, please download said release so the script can properly update your config.
If you're using >= v7 release or you don't care about your current config settings, press enter to continue. If script crashes, delete config.
Traceback (most recent call last):
File "/usr/src/app/./start_ofd.py", line 10, in <module>
main_test.check_config()
File "/usr/src/app/tests/main_test.py", line 24, in check_config
json_config, updated = main_helper.get_config(path)
File "/usr/src/app/helpers/main_helper.py", line 562, in get_config
json_config = make_settings.fix(json_config)
File "/usr/src/app/classes/make_settings.py", line 11, in fix
input()
EOFError: EOF when reading a line

I deleted my config, and when running the container, I can see that the .settings directory gets created, but alas no config file.

Also, lastly, I understand you're using VMWare, but Docker does work in VMWare. Anyhow, I've been trying to get this working since early April, with zero success so far. I'm not holding my breath that this is going to get resolved.

KptnKMan avatar Jun 15 '21 21:06 KptnKMan

Here is the docker command I'm using, if it matters:

root@localhost:# docker create `
--name='DIGITALCRIMINAL-onlyfans' `
--net='bridge' `
--privileged=true `
-e TZ="Europe/Berlin" `
-e HOST_OS="Unraid" `
-v '/mnt/user/appdata/DIGIALCRIMINAL-onlyfans/.settings':'/usr/src/app/.settings':'rw' `
-v '/mnt/user/appdata/DIGIALCRIMINAL-onlyfans/.profiles':'/usr/src/app/.profiles':'rw' `
-v '/mnt/user/appdata/DIGIALCRIMINAL-onlyfans/.sites':'/usr/src/app/.sites':'rw' `
'image/digitalcriminal-onlyfans'

KptnKMan avatar Jun 15 '21 21:06 KptnKMan

I keep trying to make this work, its been months. @DIGITALCRIMINAL @DIGITALCRIMINALS do you think I can get some assistance or should I give up on this?

KptnKMan avatar Nov 05 '21 00:11 KptnKMan