lossless1024
StripChat Streams Stopping and Starting Intermittently
I just started using this, but I noticed that the SC streams frequently stop and then immediately resume. This causes several videos to be created that can vary in length (as little as 8 seconds, but usually around 30-120 seconds). I've noticed this issue with all SC streams. Maybe this is something related to my setup, but I don't see this issue with at least Chaturbate streams.
This is what I see in the console (actual username removed):
2024-07-09 08:24:06,480 - INFO - [SC] username: Recording ended
2024-07-09 08:24:11,775 - INFO - [SC] username: Channel online
2024-07-09 08:24:12,638 - INFO - [SC] username: Selected 1280x720 28.0fps resolution
2024-07-09 08:24:12,638 - INFO - [SC] username: Started downloading show
2024-07-09 08:24:36,552 - INFO - [SC] username: Recording ended
2024-07-09 08:24:41,892 - INFO - [SC] username: Channel online
2024-07-09 08:24:42,740 - INFO - [SC] username: Selected 1280x720 28.0fps resolution
2024-07-09 08:24:42,740 - INFO - [SC] username: Started downloading show
Let me know if you need more information to replicate or identify the issue.
Thank you. I tried the latest snapshot build of ffmpeg for macOS (version 116208-gb6c43328ee), but the issue persisted. I also tried earlier releases before version 7, but still no luck.
I don't currently have the option to use a gigabit connection to reduce latency to the level that's needed.
Would using Docker instead of a local configuration help at all?
See #126, #135 and #141
Does this have anything to do with stripchat Ultra-low Latency option?
Does this have anything to do with stripchat Ultra-low Latency option?
No, it has to do with Cloudflare's anti-bot protection they've started using recently. It detects the bots and the CDN servers just stop responding for some time. It's unclear how this protection works 'cause nobody wants to debug the issue properly. It may be triggered by the number of streams or bandwidth utilized, I guess.
Does this have anything to do with stripchat Ultra-low Latency option?
Yes. It's related to HLS Low Latency features.
The problem is that ffmpeg has no capability to stretch it's behaviour when chunks getting invalidated very quickly. If ffmpeg sees too much invalidated pieces it stops working. The problem could be handled in last summer with good connections and a few tricks. So you were able to record an hour or so and then have a restart from time to time.
It might be possible that they now use Cloudflare. How Cloudflare works is clear. Some sites today use the heavy modes (IUAM). That requires JavaScript functionality up to logic that mimics user behaviour (in this modes they use also stuff like mouse tracking and so on to detect a real browser). Pages that use a Cloudflare feature protection can't be handled by StreaMonitor.
The problem is that ffmpeg has no capability to stretch it's behaviour when chunks getting invalidated very quickly.
Other sites also use LL HLS. Are there the same issue?
It might be possible that they now use Cloudflare.
Yes, they do. Just check the IP addresses );
That requires JavaScript functionality up to logic that mimics user behaviour
Do you know what CDNs are? They don't use JS or track anyone's behavior );
The problem is that ffmpeg has no capability to stretch it's behaviour when chunks getting invalidated very quickly.
Other sites also use LL HLS. Are there the same issue?
Yes. In general you will have the same problems when the HLS extensions for Low Latency are used. Especially at normal dial-up lines. Also it might depend on the ISP. It's different how efficient the target network can be reached.
You can see this very simple in the ffmpeg output. You will see appropiate info about segment lossing shortly before it stops.
It might be possible that they now use Cloudflare.
Yes, they do. Just check the IP addresses );
Yes they have Cloudflare IPs on the domain. This tells you nothing about how other systems like separate APIs, HLS Servers and so on are possibly protected. In this case they have Cloudflare also in front of the CDN.
That requires JavaScript functionality up to logic that mimics user behaviour
Do you know what CDNs are? They don't use JS or track anyone's behavior );
Yes I know what a Content Delivery Network is. And yes I know that Cloudflare also have such services. But you need the get the design of these things clear. The bot detection can be combined with a CDN. Today it's not rare that this is the case. Already to prevent web scraping.
Today normally the website and the APIs are protected by Cloudflare. Here you already need to use different protection modes. Because the more restrictive protection modes would harass the users very much mostly they use one of the moderate modes for the website which restrict stuff in some cases (much requests and so on). The API protection is mostly also in the moderate modes (JavaScript basic challanges from time to time). This just needs basic logic in the sites JavaScript to handle this.
Often the HLS Servers (in that case the CDN) have no direct protection through Cloudflare (or at least are not combined with bot protection). But they often want to see cookie sets or "auth tokens". Because of the fact that HLS is simply HTTP it's also possible to protect also these by Cloudflares HTTP Solutions. It's not a problem because the website just need JavaScript logic to handle this. Normally the protection level in front of the HLS servers is not that high to prevent problems with the user experience (possibly stopping video playing because of access cookie refresh and so on).
And Cloudflare uses things like user behaviour especially in the heavy modes like IUAM. The basic modes preventing nothing because it's more or less simple to solve the challenges. IUAM for example is active when a site prevents with things like "Wait a moment", "Click to proceed" and a Cloudflare Logo. Then everything is done to check that it's a real browser.
StreaMonitor can't capture any site that has Cloudflare in front of the API or HLS in a more or less restrictive mode. For StripChat that's not the case today. You can access the site without problems also from datacenter IPs. Also you can do successful API requests outside of a browser from a datacenter IP. That's a good indicator for the level of protection (mode). Just very low protection levels allow this. They will just give challanges or other stuff to notoriously behaving clients.
I did a few tests again: Dial-up lines fail within the first minute(s). Often really within the first minute. With a solid networking and I/O it's not a problem to record 1h+ in one take. StripChat interrupts from time to time heavily. The reaons seem to be:
- Also in a good situation it might happen that segment loosing stops the recording.
- They seem to interrupt very hardly when the model switches states. So also in that situation ffmpeg might fail instead of stopping recording cleanly.
- Toys that the models use from time to time also bring problems. They are connected to the Streaming-App the models use. It seems that e.g. changing the toy from time to time restarts streams completely from time to time. Which for ffmpeg in some cases also looks like a fail.
So nothing changed since last summer.
In general you will have the same problems when the HLS extensions for Low Latency are used.
The only problem I had with LL-HLS is that it was unsupported by everyone, so I had to write my own tiny script. But that was 5 or 6 years ago );
You can access the site without problems also from datacenter IPs. Also you can do successful API requests outside of a browser from a datacenter IP. That's a good indicator for the level of protection (mode). Just very low protection levels allow this.
That's the point! They protect the content, aka the CDNs. So, there's no sense to protect anything else );
I did a few tests again: Dial-up lines fail within the first minute(s).
Dial-up lines? What shit-hole country are you living in? Brazil );
StripChat interrupts from time to time heavily.
The main cause of interrupts are fake, aka virtual, privates );
In general you will have the same problems when the HLS extensions for Low Latency are used.
The only problem I had with LL-HLS is that it was unsupported by everyone, so I had to write my own tiny script. But that was 5 or 6 years ago );
Then it should be no problem for you to contribute a specification-compliant component that can handle this and generate usable videos.
You can access the site without problems also from datacenter IPs. Also you can do successful API requests outside of a browser from a datacenter IP. That's a good indicator for the level of protection (mode). Just very low protection levels allow this.
That's the point! They protect the content, aka the CDNs. So, there's no sense to protect anything else );
They could protect the content much better with DRM or similar. That would make it more complicated to steal content. The aim of Cloudflare protection is not content protection in this sense. At best to make collection more laborious.
I did a few tests again: Dial-up lines fail within the first minute(s).
Dial-up lines? What shit-hole country are you living in? Brazil );
Then call it DSL. Everyone must know for themselves whether such behavior is generally appropriate.
StripChat interrupts from time to time heavily.
The main cause of interrupts are fake, aka virtual, privates );
The interruptions could always be traced back to an event in the stream. At least when the network connection was good.
In this respect, it is not clear to me what a “fake aka. virtual private” is supposed to be.
However, it would be irrelevant even if the stream appeared to be set to a corresponding mode. Which still doesn't make sense to me. Because you could simply try to restart the recording. At some point, the entire logic of the website code would become too complicated if you played such games.
They could protect the content much better with DRM or similar.
It's so expensive that even Alphabet, aka Google, don't use their own Widevine to protect Youtube videos. And they don't give a shit about content creators. They even fool them so that they pay to protect their content. Like, the site owners can't do anything about it );
ATV have implemented encryption recently and the picture quality went to shit. So, everything has its cost );
Idk about the reasons or how to fix, but as i mentioned in #182 what I do is write my own scripts to combine them
model_name=model
site=SC
streamonitor_downloads=StreaMonitor/downloads
model_folder="${streamonitor_downloads}/${model_name} [${site}]"
if [ -d $model_folder ]; then
files=()
for f in ${model_folder}/*.mp4;
do
files+=$f;
echo "file '$f'" >> ${model_folder}/mylist.txt;
done
new_video_file_name=$model_name-$(date +'%Y-%m-%d_%H-%M-%S').mp4
ffmpeg -f concat -safe 0 -i mylist.txt -c copy $new_video_file_name
# for f in ${files[@]}
# do
# rm $f;
# done;
# rm mylist.txt
fi
I'm using ffmpeg 4.4.1 which does not exhibit this problem. I verified by switching between ffmpeg 4.4.1 and 4.4.5 at a time where the 4.4.5 recording gets interrupted every few minutes: the 4.4.1 has no problem at all recording in one go. So it seems the problem was introduced somewhere between these 2 versions. does anyone have an idea where the problem is located and whether it is known on the ffmpeg bug tracker? I had a look but didn't find anything although i may not have been using the right search terms..
我使用的是 ffmpeg 4.4.1，它没有出现这个问题。我通过在 ffmpeg 4.4.1 和 4.4.5 之间切换来验证，4.4.5 的录制每隔几分钟就会中断一次：4.4.1 的录制一次完全没有问题。所以问题似乎是在这两个版本之间出现的。有人知道问题出在哪里吗？ ffmpeg 的 bug tracker上是否有相关信息？我查看了一下，但什么也没找到，可能是我使用的搜索词不对。
can you share your ffmpeg 4.4.1 binary? I use this but I got error:
ffmpeg version 4.4.1-static https://johnvansickle.com/ffmpeg/ Copyright (c) 2000-2021 the FFmpeg developers
built with gcc 8 (Debian 8.3.0-6)
configuration: --enable-gpl --enable-version3 --enable-static --disable-debug --disable-ffplay --disable-indev=sndio --disable-outdev=sndio --cc=gcc --enable-fontconfig --enable-frei0r --enable-gnutls --enable-gmp --enable-libgme --enable-gray --enable-libaom --enable-libfribidi --enable-libass --enable-libvmaf --enable-libfreetype --enable-libmp3lame --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-librubberband --enable-libsoxr --enable-libspeex --enable-libsrt --enable-libvorbis --enable-libopus --enable-libtheora --enable-libvidstab --enable-libvo-amrwbenc --enable-libvpx --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxml2 --enable-libdav1d --enable-libxvid --enable-libzvbi --enable-libzimg
libavutil 56. 70.100 / 56. 70.100
libavcodec 58.134.100 / 58.134.100
libavformat 58. 76.100 / 58. 76.100
libavdevice 58. 13.100 / 58. 13.100
libavfilter 7.110.100 / 7.110.100
libswscale 5. 9.100 / 5. 9.100
libswresample 3. 9.100 / 3. 9.100
libpostproc 55. 9.100 / 55. 9.100
Unrecognized option 'readrate'.
Error splitting the argument list: Option not found
