StreaMonitor icon indicating copy to clipboard operation
StreaMonitor copied to clipboard

Camsoda (CS) Streams Showing as Unknown Error

Open maraudersmapx opened this issue 1 year ago • 19 comments

I recently noticed that Camsoda streams are showing as "Unknown error". I tested with a few different streams and observed issues with all. Issue seems to occur whether the model is currently streaming or not.

I tried gathering more information by setting DEBUG = True within parameters.py, but Downloader.py always said Debug mode: off when starting.

maraudersmapx avatar Dec 13 '24 16:12 maraudersmapx

This might be a DNS issue on my end. I've been using Cloudflare as my DNS provider and they provide protection for Camsoda. I'm wondering if using StreaMonitor is being blocked somehow. I'm able to access the site through a normal web browser.

maraudersmapx avatar Dec 16 '24 14:12 maraudersmapx

This might be a DNS issue on my end. I've been using Cloudflare as my DNS provider and they provide protection for Camsoda. I'm wondering if using StreaMonitor is being blocked somehow. I'm able to access the site through a normal web browser.

Nah it isn't on your end, I've been getting "unknown error" for at least a week now too, didn't actually realize it was an issue 'til tonight but now I see it failing to record a stream that I know is running. :\

ashamefuldog avatar Dec 17 '24 02:12 ashamefuldog

Oh thank goodness, I've been going crazy trying to figure this out! If I try to run nslookup camsoda.com or nslookup www.camsoda.com, I always get 0.0.0.0 as a response. I can still ping camsoda.com and access the website from an actual browser.

  • I've tried with Cloudflare DNS (using 1.1.1.1 and 1.1.1.2) and Quad9 (9.9.9.9), and Google DNS (8.8.8.8), but the result is always the same
  • I've found that modifying my hosts file doesn't appear to work either using 104.18.4.137 and 104.18.5.137 as found on https://www.nslookup.io/domains/camsoda.com/dns-records/
~ % cat /etc/hosts
##
# Host Database
#
# localhost is used to configure the loopback interface
# when the system is booting.  Do not change this entry.
##
127.0.0.1	localhost
255.255.255.255	broadcasthost
104.18.4.137    www.camsoda.com
104.18.4.137    camsoda.com
::1             localhost
~ % nslookup camsoda.com        
Server:		9.9.9.9
Address:	9.9.9.9#53

Name:	camsoda.com
Address: 0.0.0.0

~ % nslookup camsoda.com 1.1.1.1
Server:		1.1.1.1
Address:	1.1.1.1#53

Name:	camsoda.com
Address: 0.0.0.0

~ % nslookup camsoda.com 8.8.8.8
Server:		8.8.8.8
Address:	8.8.8.8#53

Name:	camsoda.com
Address: 0.0.0.0

maraudersmapx avatar Dec 18 '24 17:12 maraudersmapx

Additionally, it looks like attempts to access the API are getting refused. Here's an error that got posted to the Terminal:

HTTPSConnectionPool(host='www.camsoda.com', port=443): Max retries exceeded with url: /api/v1/video/vtoken/adalline (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x1045cfd90>: Failed to establish a new connection: [Errno 61] Connection refused'))
Traceback (most recent call last):
  File "/Users/maraudersmapx/PythonVirtualEnv/StreaMonitor/lib/python3.13/site-packages/urllib3/connection.py", line 199, in _new_conn
    sock = connection.create_connection(
        (self._dns_host, self.port),
    ...<2 lines>...
        socket_options=self.socket_options,
    )
  File "/Users/maraudersmapx/PythonVirtualEnv/StreaMonitor/lib/python3.13/site-packages/urllib3/util/connection.py", line 85, in create_connection
    raise err
  File "/Users/maraudersmapx/PythonVirtualEnv/StreaMonitor/lib/python3.13/site-packages/urllib3/util/connection.py", line 73, in create_connection
    sock.connect(sa)
    ~~~~~~~~~~~~^^^^
ConnectionRefusedError: [Errno 61] Connection refused

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/Users/maraudersmapx/PythonVirtualEnv/StreaMonitor/lib/python3.13/site-packages/urllib3/connectionpool.py", line 789, in urlopen
    response = self._make_request(
        conn,
    ...<10 lines>...
        **response_kw,
    )
  File "/Users/maraudersmapx/PythonVirtualEnv/StreaMonitor/lib/python3.13/site-packages/urllib3/connectionpool.py", line 490, in _make_request
    raise new_e
  File "/Users/maraudersmapx/PythonVirtualEnv/StreaMonitor/lib/python3.13/site-packages/urllib3/connectionpool.py", line 466, in _make_request
    self._validate_conn(conn)
    ~~~~~~~~~~~~~~~~~~~^^^^^^
  File "/Users/maraudersmapx/PythonVirtualEnv/StreaMonitor/lib/python3.13/site-packages/urllib3/connectionpool.py", line 1095, in _validate_conn
    conn.connect()
    ~~~~~~~~~~~~^^
  File "/Users/maraudersmapx/PythonVirtualEnv/StreaMonitor/lib/python3.13/site-packages/urllib3/connection.py", line 693, in connect
    self.sock = sock = self._new_conn()
                       ~~~~~~~~~~~~~~^^
  File "/Users/maraudersmapx/PythonVirtualEnv/StreaMonitor/lib/python3.13/site-packages/urllib3/connection.py", line 214, in _new_conn
    raise NewConnectionError(
        self, f"Failed to establish a new connection: {e}"
    ) from e
urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPSConnection object at 0x1045cfd90>: Failed to establish a new connection: [Errno 61] Connection refused

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/Users/maraudersmapx/PythonVirtualEnv/StreaMonitor/lib/python3.13/site-packages/requests/adapters.py", line 667, in send
    resp = conn.urlopen(
        method=request.method,
    ...<9 lines>...
        chunked=chunked,
    )
  File "/Users/maraudersmapx/PythonVirtualEnv/StreaMonitor/lib/python3.13/site-packages/urllib3/connectionpool.py", line 843, in urlopen
    retries = retries.increment(
        method, url, error=new_e, _pool=self, _stacktrace=sys.exc_info()[2]
    )
  File "/Users/maraudersmapx/PythonVirtualEnv/StreaMonitor/lib/python3.13/site-packages/urllib3/util/retry.py", line 519, in increment
    raise MaxRetryError(_pool, url, reason) from reason  # type: ignore[arg-type]
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='www.camsoda.com', port=443): Max retries exceeded with url: /api/v1/video/vtoken/adalline (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x1045cfd90>: Failed to establish a new connection: [Errno 61] Connection refused'))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/maraudersmapx/Desktop/StreaMonitor-master/streamonitor/bot.py", line 134, in run
    self.sc = self.getStatus()
              ~~~~~~~~~~~~~~^^
  File "/Users/maraudersmapx/Desktop/StreaMonitor-master/streamonitor/sites/camsoda.py", line 25, in getStatus
    r = requests.get('https://www.camsoda.com/api/v1/video/vtoken/' + self.username, headers=headers)
  File "/Users/maraudersmapx/PythonVirtualEnv/StreaMonitor/lib/python3.13/site-packages/requests/api.py", line 73, in get
    return request("get", url, params=params, **kwargs)
  File "/Users/maraudersmapx/PythonVirtualEnv/StreaMonitor/lib/python3.13/site-packages/requests/api.py", line 59, in request
    return session.request(method=method, url=url, **kwargs)
           ~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/maraudersmapx/PythonVirtualEnv/StreaMonitor/lib/python3.13/site-packages/requests/sessions.py", line 589, in request
    resp = self.send(prep, **send_kwargs)
  File "/Users/maraudersmapx/PythonVirtualEnv/StreaMonitor/lib/python3.13/site-packages/requests/sessions.py", line 703, in send
    r = adapter.send(request, **kwargs)
  File "/Users/maraudersmapx/PythonVirtualEnv/StreaMonitor/lib/python3.13/site-packages/requests/adapters.py", line 700, in send
    raise ConnectionError(e, request=request)
requests.exceptions.ConnectionError: HTTPSConnectionPool(host='www.camsoda.com', port=443): Max retries exceeded with url: /api/v1/video/vtoken/adalline (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x1045cfd90>: Failed to establish a new connection: [Errno 61] Connection refused'))

maraudersmapx avatar Dec 18 '24 17:12 maraudersmapx

It's possible that my IP has been blocked from using the API. I connected my laptop to my phone's hotspot, which gave me a new IP address. When I attempted to connect again, I got this:

PS /> invoke-webrequest www.camsoda.com/api/v1/video/vtoken/adalline
Invoke-WebRequest: Access Denied - Bot Restriction                                                                      

It appears that you are attempting to crawl this site. To help you access our content in the best way possible, please use our official feed:
https://feed.camsoda.com/api/v1/browse/online_embed

If you need additional access or have specific requirements, please contact us at:
[email protected]

PS /> invoke-webrequest https://feed.camsoda.com/api/v1/browse/online_embed
                                                                                                                        
StatusCode        : 200
StatusDescription : OK
Content           : {"results":[{"user_id":31022080,"username":"18perfecttits","name":"18perfecttits","status":"private","subjec
                    t":"it's been 2 weeks since the last squirt!!! help me squirt everywhere!!! #squirt on face …
RawContent        : HTTP/1.1 200 OK
                    Server: nginx
                    Date: Wed, 18 Dec 2024 17:24:19 GMT
                    Access-Control-Allow-Origin: *
                    Vary: Accept-Encoding
                    x-varnish: 185277135 184709760
                    Age: 37
                    Via: 1.1 varnish (Varnish/5.0)
                    X-Cache: HI…
Headers           : {[Server, System.String[]], [Date, System.String[]], [Access-Control-Allow-Origin, System.String[]], [Vary, 
                    System.String[]]…}
Images            : {}
InputFields       : {}
Links             : {}
RawContentLength  : 1316370
RelationLink      : {}

maraudersmapx avatar Dec 18 '24 17:12 maraudersmapx

I figured out the issue I was having with nslookup not resolving by disabling the ad blocking option on my router.

That doesn't appear to be related to StreamMonitor and the Unknown error response. I'm still getting the error.

maraudersmapx avatar Dec 18 '24 18:12 maraudersmapx

It is my understanding that the CS has upgraded its HTTP version from HTTP/2 to HTTP/3.

gh-1

using curl_cffi gh-2

streamonitor\sites\camsoda.py

from curl_cffi import requests
    from streamonitor.bot import Bot

    class CamSoda(Bot):
    site = "CamSoda"
    siteslug = "CS"

        def __init__(self, username):
            super(CamSoda, self).__init__(username)

        def getWebsiteURL(self):
            return "https://www.camsoda.com/" + self.username

        def getVideoUrl(self):
            v = (
                "https://"
                + self.lastInfo["edge_servers"][0]
                + "/"
                + self.lastInfo["stream_name"]
                + "_v1/index.m3u8?token="
                + self.lastInfo["token"]
            )
            return self.getWantedResolutionPlaylist(v)

        def getStatus(self):
            headers = self.headers | {
                "Accept": "text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8",
                "User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/131.0.0.0 Safari/537.36",
            }

            r = requests.get(
                "https://www.camsoda.com/api/v1/video/vtoken/" + self.username,
                headers=headers,
                impersonate="chrome",
            )

            if r.status_code != 200:
                return Bot.Status.UNKNOWN

            self.lastInfo = r.json()

            if (
                "message" in self.lastInfo
                and self.lastInfo["message"] == "No broadcaster found"
            ):
                return Bot.Status.NOTEXIST
            elif "edge_servers" in self.lastInfo and len(self.lastInfo["edge_servers"]) > 0:
                return Bot.Status.PUBLIC
            elif (
                "private_servers" in self.lastInfo
                and len(self.lastInfo["private_servers"]) > 0
            ):
                return Bot.Status.PRIVATE
            elif "token" in self.lastInfo:
                return Bot.Status.OFFLINE
            return Bot.Status.UNKNOWN

    Bot.loaded_sites.add(CamSoda)

quavoo avatar Dec 19 '24 18:12 quavoo

Thank you! I've confirmed that the updated bot has restored CS downloads!

maraudersmapx avatar Dec 20 '24 15:12 maraudersmapx

@maraudersmapx, you may want to re-open this issue for visibility until the fix is merged. I've tested and merged the fix in my own StreaMonitor fork. I can create a PR with this fix later today if @quavoo and no one else has opened a PR by then.

whimsical-c4lic0 avatar Dec 20 '24 18:12 whimsical-c4lic0

Great point! I didn’t realize it was on your own fork. Re-opening now

maraudersmapx avatar Dec 21 '24 15:12 maraudersmapx

Weird update: whimsical's merge was working flawlessly (thanks, btw!) until today, but now seems like it's causing an "unknown error" issue with MFC? Or at least, I was getting that error with all MFC streams this afternoon and confirmed there was at least one stream running but not being recorded; reverting to my backup copy of Streamonitor immediately fixed that issue. Though of course now CS is back to showing up as "unknown error" so... sort of a lateral move.

Anybody wanna check and make sure this isn't just me? Possible I goofed something up or it was a temporary hitch.

ashamefuldog avatar Dec 23 '24 21:12 ashamefuldog

Weird update: whimsical's merge was working flawlessly (thanks, btw!) until today, but now seems like it's causing an "unknown error" issue with MFC? Or at least, I was getting that error with all MFC streams this afternoon and confirmed there was at least one stream running but not being recorded; reverting to my backup copy of Streamonitor immediately fixed that issue. Though of course now CS is back to showing up as "unknown error" so... sort of a lateral move.

Anybody wanna check and make sure this isn't just me? Possible I goofed something up or it was a temporary hitch.

Were you using my fork or the open PR on this repository? The reason I ask is that my fork has diverged a bit from upstream (this repository). I don't track any MFC streams, so they may have been broken on my fork for some time if you were using that.

whimsical-c4lic0 avatar Dec 23 '24 22:12 whimsical-c4lic0

@whimsical-c4lic0 I tried to use your updated version but somehow the camsoda stream still missing audio. Did some testing and find out camsoda.py URL is different that URL passed to ffmpeg.

camsoda.py URL : https: //streaming-edge-front.livemediahost.com/edge11-ild/cam_obs/model_name_v1/index.m3u8?token=eyJpdiI6IkpJZGJGcHZwSlJZaHltenhhTXpJZ2c9PSIsInZhbHVlIjoiaFRPMzBuYkpuXC9uWmt1Mm90RGFPM1E9PSIsIm1hYyI6IjI3ZjgzYmU3ZWE0ZjRkMmY2Nzk3ZjNhNDgwZGYwMWE2OGZiY2U3ZWE0NzhhNjU2ZDFjZWUxNjdiMWUxNjI2NTEifQ==

ffmpeg used URL : https: //streaming-edge-front.livemediahost.com/edge11-ild/cam_obs/model_name_v1/tracks-v4/index.fmp4.m3u8?token=eyJpdiI6IkpJZGJGcHZwSlJZaHltenhhTXpJZ2c9PSIsInZhbHVlIjoiaFRPMzBuYkpuXC9uWmt1Mm90RGFPM1E9PSIsIm1hYyI6IjI3ZjgzYmU3ZWE0ZjRkMmY2Nzk3ZjNhNDgwZGYwMWE2OGZiY2U3ZWE0NzhhNjU2ZDFjZWUxNjdiMWUxNjI2NTEifQ%3D%3D

I need some help please.

padillageorge8989 avatar Dec 26 '24 18:12 padillageorge8989

I apologize for the issue in the previous update. I did not verify whether the audio was properly captured within the videos, as my focus was primarily on recording the video.

You can use the following code to combine the audio and video:


def getVideoUrl(self):
    audio = "multitrack=true&filter=tracks:v4v3v2v1a1a2"
    v = (
        "https://"
        + self.lastInfo["edge_servers"][0]
        + "/"
        + self.lastInfo["stream_name"]
        + "_v1/index.ll.m3u8?"
        + audio
        + "&token="
        + self.lastInfo["token"]
    )
    return self.getWantedResolutionPlaylist(v)

    # Rest of the code

Yes I have verified that the audio is being captured properly in the video.


Stream URL from the browser:-

Request URL:
https://streaming-edge-front.livemediahost.com/
edge5-cad/
cam_obs/
model_name_v1/
tracks-v4a2/
index.ll.m3u8?
multitrack=true&
filter.tracks=v4v3v2v1a1a2&
token=eyJpdiI6IkpJZGJGcHZwSlJZaHltenhhTXpJZ2c9PSIsInZhbHVlIjoiaFRPMzBuYkpuXC9uWmt1Mm90RGFPM1E9PSIsIm1hYyI6IjI3ZjgzYmU3ZWE0ZjRkMmY2Nzk3ZjNhNDgwZGYwMWE2OGZiY2U3ZWE0NzhhNjU2ZDFjZWUxNjdiMWUxNjI2NTEifQ==&
_HLS_msn=2652&
_HLS_part=0

Request Method: GET
Status Code: 200 OK

The new url from the stream that I have found while specting the request

index.ll.m3u8?multitrack=true&filter=tracks:v4v3v2v1a1a2

  • multitrack=true

    This parameter indicates that the content supports multiple tracks.

  • filter=tracks:v4v3v2v1a1a2

    The following parameter specifies which video (v4-v1) and audio (a1-a2) tracks to include, with v4 being the highest video quality and a1/a2 representing different audio streams.


The file to this issue for your reference.

  • streamonitor\sites\camsoda.py -> https://pastebin.com/raw/TVhX9Hsc

quavoo avatar Dec 26 '24 21:12 quavoo

Thanks @quavoo ! I should have tested it more thoroughly myself before pushing the code changes, so some (if not most) of the blame is on me.

@padillageorge8989 , both my PR and my fork has been updated with the audio fix.

whimsical-c4lic0 avatar Dec 27 '24 08:12 whimsical-c4lic0

Thanks for the sound fix, but it seems that the "Unknown error" problem appears again but in a different way. Initially everything works fine, but after running the app for some time it starts to show "Unknown error" on some models, even after recording starts. In the download folder there are multiple 1-2 minute files. Strange that at first everything is ok and degrades during use. I leave here a ss what the errors look like in the application: ->https://imgur.com/a/WX0ObIF

padillageorge8989 avatar Dec 28 '24 12:12 padillageorge8989

Thanks for the sound fix, but it seems that the "Unknown error" problem appears again but in a different way. Initially everything works fine, but after running the app for some time it starts to show "Unknown error" on some models, even after recording starts. In the download folder there are multiple 1-2 minute files. Strange that at first everything is ok and degrades during use. I leave here a ss what the errors look like in the application: ->https://imgur.com/a/WX0ObIF

I did some tests and found out that a temporary workaround is to reset the config and manually re-enter the models in the console. If I copy a config with the models already inserted I get "Unknown error" from the start, if I manually insert them into a new config it goes for a period of time that can vary from 1-2 days to 1-2 hours, very annoying. Of course I'm only talking about camsoda. Can someone who has the knowledge to look into this problem please? @quavoo , @whimsical-c4lic0 , @maraudersmapx someone?

padillageorge8989 avatar Jan 18 '25 14:01 padillageorge8989

I haven't personally seen this issue since updating the config files from your branch. CS recordings are working like a charm for me

maraudersmapx avatar Feb 04 '25 13:02 maraudersmapx

I haven't personally seen this issue since updating the config files from your branch. CS recordings are working like a charm for me

We are talking about this version? https://github.com/whimsical-c4lic0/StreaMonitor

Because for me it's works only if I did that workaround from above. it's frustrating when after a few hours of running I only see "Unknown error" in the console and nothing is recorded.

padillageorge8989 avatar Feb 05 '25 16:02 padillageorge8989