RatS Movielens - Connection Refused

Describe the bug As soon as I run : python3 transfer_ratings.py --source trakt --destination movielens --file 20211126132038_Trakt.json

it gives me this:

===== Trakt: performing login/home/hypatia/.local/lib/python3.9/site-packages/selenium/webdriver/remote/webelement.py:206: UserWarning: find_elements_by_* commands are deprecated. Please use find_elements() instead
  warnings.warn("find_elements_by_* commands are deprecated. Please use find_elements() instead")

===== loaded 1346 movies from /home/hypatia/RatS/RatS/exports/20211126132038_Trakt.json
===== Movielens: posting 1346 movies
===== saving movies to CSV

ERROR: Couldn't upload CSV.

===== ABORTING =====

===== Movielens: The file with 1346 movies was uploaded and will be process by the servers. You may check your Movielens account later.
Note, that this might not overwrite any existing ratings.
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/urllib3/connection.py", line 169, in _new_conn
    conn = connection.create_connection(
  File "/usr/lib/python3/dist-packages/urllib3/util/connection.py", line 96, in create_connection
    raise err
  File "/usr/lib/python3/dist-packages/urllib3/util/connection.py", line 86, in create_connection
    sock.connect(sa)
ConnectionRefusedError: [Errno 111] Connection refused

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 699, in urlopen
    httplib_response = self._make_request(
  File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 394, in _make_request
    conn.request(method, url, **httplib_request_kw)
  File "/usr/lib/python3/dist-packages/urllib3/connection.py", line 234, in request
    super(HTTPConnection, self).request(method, url, body=body, headers=headers)
  File "/usr/lib/python3.9/http/client.py", line 1253, in request
    self._send_request(method, url, body, headers, encode_chunked)
  File "/usr/lib/python3.9/http/client.py", line 1299, in _send_request
    self.endheaders(body, encode_chunked=encode_chunked)
  File "/usr/lib/python3.9/http/client.py", line 1248, in endheaders
    self._send_output(message_body, encode_chunked=encode_chunked)
  File "/usr/lib/python3.9/http/client.py", line 1008, in _send_output
    self.send(msg)
  File "/usr/lib/python3.9/http/client.py", line 948, in send
    self.connect()
  File "/usr/lib/python3/dist-packages/urllib3/connection.py", line 200, in connect
    conn = self._new_conn()
  File "/usr/lib/python3/dist-packages/urllib3/connection.py", line 181, in _new_conn
    raise NewConnectionError(
urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7f90cd0d9310>: Failed to establish a new connection: [Errno 111] Connection refused

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/hypatia/RatS/transfer_ratings.py", line 252, in insert_movie_ratings
    inserter.insert(movies, source)
  File "/home/hypatia/RatS/RatS/base/base_ratings_uploader.py", line 56, in insert
    self.site.browser_handler.kill()
  File "/home/hypatia/RatS/RatS/utils/browser_handler.py", line 86, in kill
    self.browser.close()
  File "/home/hypatia/.local/lib/python3.9/site-packages/selenium/webdriver/remote/webdriver.py", line 931, in close
    self.execute(Command.CLOSE)
  File "/home/hypatia/.local/lib/python3.9/site-packages/selenium/webdriver/remote/webdriver.py", line 416, in execute
    response = self.command_executor.execute(driver_command, params)
  File "/home/hypatia/.local/lib/python3.9/site-packages/selenium/webdriver/remote/remote_connection.py", line 398, in execute
    return self._request(command_info[0], url, body=data)
  File "/home/hypatia/.local/lib/python3.9/site-packages/selenium/webdriver/remote/remote_connection.py", line 420, in _request
    resp = self._conn.request(method, url, body=body, headers=headers)
  File "/usr/lib/python3/dist-packages/urllib3/request.py", line 74, in request
    return self.request_encode_url(
  File "/usr/lib/python3/dist-packages/urllib3/request.py", line 96, in request_encode_url
    return self.urlopen(method, url, **extra_kw)
  File "/usr/lib/python3/dist-packages/urllib3/poolmanager.py", line 375, in urlopen
    response = conn.urlopen(method, u.request_uri, **kw)
  File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 783, in urlopen
    return self.urlopen(
  File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 783, in urlopen
    return self.urlopen(
  File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 783, in urlopen
    return self.urlopen(
  File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 755, in urlopen
    retries = retries.increment(
  File "/usr/lib/python3/dist-packages/urllib3/util/retry.py", line 573, in increment
    raise MaxRetryError(_pool, url, error or ResponseError(cause))
urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='localhost', port=54107): Max retries exceeded with url: /session/aabed919-8c97-4f87-b34b-b952d422cd3b/window (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f90cd0d9310>: Failed to establish a new connection: [Errno 111] Connection refused'))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/urllib3/connection.py", line 169, in _new_conn
    conn = connection.create_connection(
  File "/usr/lib/python3/dist-packages/urllib3/util/connection.py", line 96, in create_connection
    raise err
  File "/usr/lib/python3/dist-packages/urllib3/util/connection.py", line 86, in create_connection
    sock.connect(sa)
ConnectionRefusedError: [Errno 111] Connection refused

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 699, in urlopen
    httplib_response = self._make_request(
  File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 394, in _make_request
    conn.request(method, url, **httplib_request_kw)
  File "/usr/lib/python3/dist-packages/urllib3/connection.py", line 234, in request
    super(HTTPConnection, self).request(method, url, body=body, headers=headers)
  File "/usr/lib/python3.9/http/client.py", line 1253, in request
    self._send_request(method, url, body, headers, encode_chunked)
  File "/usr/lib/python3.9/http/client.py", line 1299, in _send_request
    self.endheaders(body, encode_chunked=encode_chunked)
  File "/usr/lib/python3.9/http/client.py", line 1248, in endheaders
    self._send_output(message_body, encode_chunked=encode_chunked)
  File "/usr/lib/python3.9/http/client.py", line 1008, in _send_output
    self.send(msg)
  File "/usr/lib/python3.9/http/client.py", line 948, in send
    self.connect()
  File "/usr/lib/python3/dist-packages/urllib3/connection.py", line 200, in connect
    conn = self._new_conn()
  File "/usr/lib/python3/dist-packages/urllib3/connection.py", line 181, in _new_conn
    raise NewConnectionError(
urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7f90cd0d9d60>: Failed to establish a new connection: [Errno 111] Connection refused

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/hypatia/RatS/transfer_ratings.py", line 271, in <module>
    main()
  File "/home/hypatia/RatS/transfer_ratings.py", line 92, in main
    execute(args)
  File "/home/hypatia/RatS/transfer_ratings.py", line 170, in execute
    execute_inserting(args, movies, parser)
  File "/home/hypatia/RatS/transfer_ratings.py", line 193, in execute_inserting
    insert_movie_ratings(inserter, movies, type(parser.site).__name__)
  File "/home/hypatia/RatS/transfer_ratings.py", line 258, in insert_movie_ratings
    inserter.site.browser_handler.kill()
  File "/home/hypatia/RatS/RatS/utils/browser_handler.py", line 86, in kill
    self.browser.close()
  File "/home/hypatia/.local/lib/python3.9/site-packages/selenium/webdriver/remote/webdriver.py", line 931, in close
    self.execute(Command.CLOSE)
  File "/home/hypatia/.local/lib/python3.9/site-packages/selenium/webdriver/remote/webdriver.py", line 416, in execute
    response = self.command_executor.execute(driver_command, params)
  File "/home/hypatia/.local/lib/python3.9/site-packages/selenium/webdriver/remote/remote_connection.py", line 398, in execute
    return self._request(command_info[0], url, body=data)
  File "/home/hypatia/.local/lib/python3.9/site-packages/selenium/webdriver/remote/remote_connection.py", line 420, in _request
    resp = self._conn.request(method, url, body=body, headers=headers)
  File "/usr/lib/python3/dist-packages/urllib3/request.py", line 74, in request
    return self.request_encode_url(
  File "/usr/lib/python3/dist-packages/urllib3/request.py", line 96, in request_encode_url
    return self.urlopen(method, url, **extra_kw)
  File "/usr/lib/python3/dist-packages/urllib3/poolmanager.py", line 375, in urlopen
    response = conn.urlopen(method, u.request_uri, **kw)
  File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 783, in urlopen
    return self.urlopen(
  File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 783, in urlopen
    return self.urlopen(
  File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 783, in urlopen
    return self.urlopen(
  File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 755, in urlopen
    retries = retries.increment(
  File "/usr/lib/python3/dist-packages/urllib3/util/retry.py", line 573, in increment
    raise MaxRetryError(_pool, url, error or ResponseError(cause))
urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='localhost', port=54107): Max retries exceeded with url: /session/aabed919-8c97-4f87-b34b-b952d422cd3b/window (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f90cd0d9d60>: Failed to establish a new connection: [Errno 111] Connection refused'))

Desktop (please complete the following information):

Version: latest
Platform OS: pop os
Firefox version:
Geckodriver version: geckodriver 0.30.0
Python version: Python 3.9.5
PIP version: pip 20.3.4 from /usr/lib/python3/dist-packages/pip (python 3.9)
Running in a virtualenv?: no
Running in Docker?: no

Any help would be much appreciated.

Nov 26 '21 17:11 littlebirdiegr

Ok did some testing and this might help find the bug.

The converted_for_Movielens.csv file had about 1342 ratings. I split them in the 400s and manually uploaded them to movielens. When trying to upload all of the 1342 ratings , movielens showed a message There was an error trying to import the file. However when I uploaded just 400 it imported them fine. So it's probs something to do with the number of the ratings. Hope this helps.

Nov 26 '21 18:11 littlebirdiegr

Trying the command with -x shows that it get a json as response when expecting a csv. So the Parser needs a rewrite.

Dec 24 '23 18:12 StegSchreck

correction, this was actually caused by improperly closing the browser after the parsing

Dec 28 '23 20:12 StegSchreck

RatS RatS copied to clipboard

Movielens - Connection Refused

RatS
RatS copied to clipboard