Implement WebSocket connection backend

[njbooher]

Resolves #114

[denizkoekden]

I just wanted to throw my support behind this PR. I’m currently using @njboohers' wsproto branch by manually importing it, and it’s been working great for me. This library is used in a lot of Steam Python projects, many of which have stopped working due to the current connection method breaking down.

It would be awesome to see this merged ASAP since it fixes the issue and gets everything back on track. Big thanks to everyone involved in pushing this forward! Maybe @rossengeorgiev can help out here?

[njbooher]

IIRC, during my testing, the main issue I encountered was connections to some servers didn't work, and I think it might have been due to them sending messages larger than the 16kb this reads at the time. I can't do any work on this this week, but possibly next week.

[denizkoekden]

IIRC, during my testing, the main issue I encountered was connections to some servers didn't work, and I think it might have been due to them sending messages larger than the 16kb this reads at the time. I can't do any work on this this week, but possibly next week.

I am using your fork in my productive Environment I have seen some errors in the log when connected to ams1 CM but they resolve after restarting the application. I will provide Logs with all the errors so we can resolve the Issues. ATM I am testing the fixes from @revij and @BlankTMing (https://github.com/denizkoekden/steam/commit/bee32ca1e4d07d752179f42704741f9f0e34bb4c).

[denizkoekden]

Okay I use https://github.com/denizkoekden/steam/commit/bee32ca1e4d07d752179f42704741f9f0e34bb4c which is your (@njbooher) Fork + the changes @revij and @BlankTMing did. The errors seem to be exactly the same as without their fixes. I was just some trial and error nobrainer testing as the base functionallity is given it just takes a few more tries to get the desired result. I use the steam lib in the steamcmd-project which is basically an simple but helpful FastAPI Project to retrive informations about an given appid.

Here are the last 100 lines of the log:

level=INFO msg="Requested app info" app_id=2667530
level=INFO msg="App info succesfully retrieved from cache" app_id=2667530
level=INFO msg="Succesfully retrieved app info" app_id=2667530
level=INFO msg="Requested app info" app_id=1794810
level=INFO msg="App info could not be found in cache" app_id=1794810
level=INFO msg="Started requesting app info" app_id=1794810
level=INFO msg="Retrieving app info from steamclient" app_id=1794810 retry_count=0
level=INFO msg="Succesfully retrieved app info" app_id=2667530
level=INFO msg="Succesfully retrieved app info" app_id=1794810
level=INFO msg="Requested app info" app_id=349090
level=INFO msg="App info could not be found in cache" app_id=349090
level=INFO msg="Started requesting app info" app_id=349090
level=INFO msg="Retrieving app info from steamclient" app_id=349090 retry_count=0
level=INFO msg="Requested app info" app_id=1794810
level=INFO msg="App info succesfully retrieved from cache" app_id=1794810
level=INFO msg="Succesfully retrieved app info" app_id=1794810
level=INFO msg="Succesfully retrieved app info" app_id=1794810
level=INFO msg="Requested app info" app_id=349090
level=INFO msg="App info could not be found in cache" app_id=349090
level=INFO msg="Started requesting app info" app_id=349090
level=INFO msg="Retrieving app info from steamclient" app_id=349090 retry_count=0
level=INFO msg="Requested app info" app_id=349090
level=INFO msg="App info could not be found in cache" app_id=349090
level=INFO msg="Started requesting app info" app_id=349090
level=INFO msg="Retrieving app info from steamclient" app_id=349090 retry_count=0
Traceback (most recent call last):
  File "src/gevent/greenlet.py", line 906, in gevent._gevent_cgreenlet.Greenlet.run
  File "/usr/local/lib/python3.10/site-packages/steam/core/connection.py", line 216, in _reader_loop
    rlist, _, _ = gselect([self.socket], [], [])
  File "/usr/local/lib/python3.10/site-packages/gevent/select.py", line 181, in select
    sel_results = _original_select(rlist, wlist, xlist, 0)
ValueError: filedescriptor out of range in select()
2024-10-14T12:52:49Z <Greenlet at 0x7f3660773380: <bound method WebsocketConnection._reader_loop of <steam.core.connection.WebsocketConnection object at 0x7f3660798340>>> failed with ValueError

level=INFO msg="Succesfully retrieved app info" app_id=349090
level=INFO msg="Requested app info" app_id=2515600
level=INFO msg="App info could not be found in cache" app_id=2515600
level=INFO msg="Started requesting app info" app_id=2515600
level=INFO msg="Retrieving app info from steamclient" app_id=2515600 retry_count=0
Traceback (most recent call last):
  File "src/gevent/greenlet.py", line 906, in gevent._gevent_cgreenlet.Greenlet.run
  File "/usr/local/lib/python3.10/site-packages/steam/core/connection.py", line 216, in _reader_loop
    rlist, _, _ = gselect([self.socket], [], [])
  File "/usr/local/lib/python3.10/site-packages/gevent/select.py", line 181, in select
    sel_results = _original_select(rlist, wlist, xlist, 0)
ValueError: filedescriptor out of range in select()
2024-10-14T12:52:50Z <Greenlet at 0x7f3649624c20: <bound method WebsocketConnection._reader_loop of <steam.core.connection.WebsocketConnection object at 0x7f366436fd60>>> failed with ValueError

level=INFO msg="Succesfully retrieved app info" app_id=349090
level=INFO msg="Requested app info" app_id=2515600
level=INFO msg="App info could not be found in cache" app_id=2515600
level=INFO msg="Started requesting app info" app_id=2515600
level=INFO msg="Retrieving app info from steamclient" app_id=2515600 retry_count=0
level=WARNING msg="Encountered timeout when trying to connect to steam api. Retrying.."
level=ERROR msg="Max connect retries exceeded" connect_retries=2
level=ERROR msg="Failed in retrieving app info" app_id=2394010
level=ERROR msg="Max connect retries (2) exceeded" app_id=2394010
level=INFO msg="The SteamCMD backend returned no actual data and failed" app_id=2394010
level=INFO msg="Requested app info" app_id=2394010
level=INFO msg="App info could not be found in cache" app_id=2394010
level=INFO msg="Started requesting app info" app_id=2394010
level=INFO msg="Retrieving app info from steamclient" app_id=2394010 retry_count=0
level=INFO msg="Succesfully retrieved app info" app_id=2515600
level=INFO msg="Requested app info" app_id=2533070
level=INFO msg="App info could not be found in cache" app_id=2533070
level=INFO msg="Started requesting app info" app_id=2533070
level=INFO msg="Retrieving app info from steamclient" app_id=2533070 retry_count=0
Traceback (most recent call last):
  File "src/gevent/greenlet.py", line 906, in gevent._gevent_cgreenlet.Greenlet.run
  File "/usr/local/lib/python3.10/site-packages/steam/core/connection.py", line 216, in _reader_loop
    rlist, _, _ = gselect([self.socket], [], [])
  File "/usr/local/lib/python3.10/site-packages/gevent/select.py", line 181, in select
    sel_results = _original_select(rlist, wlist, xlist, 0)
ValueError: filedescriptor out of range in select()
2024-10-14T12:52:51Z <Greenlet at 0x7f36607734c0: <bound method WebsocketConnection._reader_loop of <steam.core.connection.WebsocketConnection object at 0x7f366088f0d0>>> failed with ValueError

Traceback (most recent call last):
  File "src/gevent/greenlet.py", line 906, in gevent._gevent_cgreenlet.Greenlet.run
  File "/usr/local/lib/python3.10/site-packages/steam/client/__init__.py", line 101, in _parse_message
    result = CMClient._parse_message(self, message)
  File "/usr/local/lib/python3.10/site-packages/steam/core/cm.py", line 289, in _parse_message
    msg.parse()
  File "/usr/local/lib/python3.10/site-packages/steam/core/msg/__init__.py", line 201, in parse
    self.body.ParseFromString(self.payload)
google.protobuf.message.DecodeError: Error parsing message with type 'CMsgMulti'
2024-10-14T12:52:51Z <Greenlet at 0x7f3660a43420: <bound method SteamClient._parse_message of <SteamClient(('cmp2-ams1.steamserver.net', 27018)) online>>(bytearray(b'\x01\x00\x00\x80\x00\x00\x00\x00\x08\x)> failed with DecodeError

Traceback (most recent call last):
  File "src/gevent/greenlet.py", line 906, in gevent._gevent_cgreenlet.Greenlet.run
  File "/usr/local/lib/python3.10/site-packages/steam/client/__init__.py", line 101, in _parse_message
    result = CMClient._parse_message(self, message)
  File "/usr/local/lib/python3.10/site-packages/steam/core/cm.py", line 259, in _parse_message
    emsg = EMsg(clear_proto_bit(emsg_id))
  File "/usr/local/lib/python3.10/enum.py", line 385, in __call__
    return cls.__new__(cls, value)
  File "/usr/local/lib/python3.10/enum.py", line 710, in __new__
    raise ve_exc
ValueError: 1003589537 is not a valid EMsg
2024-10-14T12:52:51Z <Greenlet at 0x7f3660a43420: <bound method SteamClient._parse_message of <SteamClient(('cmp2-ams1.steamserver.net', 27018)) online>>(bytearray(b'\xa1\x8f\xd1\xbb\xdc\xda\x1be\xee\x19\)> failed with ValueError

[BlankTMing]

Use TCPConnection instead of WebSocket @denizkoekden

[kostrykin]

@njbooher I was using the repo from your PR for a project. Any chance that you can bring it back? Is there any other fork that is working?

[Gobot1234]

I'd been holding off from posting anything for a while but I think it's been long enough, I don't have a fork but my library for steam has support for this https://github.com/Gobot1234/steam.py

[kostrykin]

Thanks @Gobot1234 I will consider migrating to your library!