KGrabber
KGrabber copied to clipboard
Downloading HydraX Streams
This script is amazing and after adding the idm export feature it became even more amazing. but since kissanime changed from rapidvideo to hydrax, i cant download any anime. I searched a lot but couldn't find a way to generate links from hydrax stream. It seems to be impossible so i had to use nova server and solve all the captcha to get the download links. If anybody knows a way to generate links from hydrax stream that would be a great help.
I tried (and failed) to do this a while ago. You can find my attempts on the hydrax branch. I got to the point of getting all the data they use to assemble the stream, but i wasn't able to assemble it or even play their assembled streams outside of their website. Perhaps someone more knowledgeable about HLS can crack this, in which case I'd be happy to write my findings down in detail.
Hydrax seems to have been removed. Re-open when it is spotted again.
Hey it is back...
Indeed it is. I've re-added it to the script, although the links don't even work outside of an iframe anymore.
Hydrax got The Alliance for creativity (ACE) protection which was created on 2017 they are anti-piracy Organization. They was even shutdown Openload and streaming at 31 October 2019. I guess Hydrax thought better form alliance get protected then shutdown as well.
@thorio I spent some time messing around with HydraX, and so far this is what I have been able to reverse engineer:
If the embed URL is https://playhydrax.com/?v=VMG9wjJtM
To download the video, first you need to make a POST request to https://ping.idocdn.com/
.
curl "https://ping.idocdn.com/" -d "slug=VMG9wjJtM"
Response:
{
"status": true,
"url": "m0weXd1eG1pdS54eXo=a",
"sources": ["sd", "hd"],
"isCdn": false,
"isHls": false,
"ads": "https://unreshiramor.com/4/3239564/"
}
Then, you can start downloading the video.
curl "https://www.4vs5osstge.buzz/" -H "Referer: https://playhydrax.com/?v=VMG9wjJtM" -o "video.mp4"
Notes
- All the videos I found were mp4, so I don't know if this will work for HLS files. I can try it out if I get an HLS link.
- I have no idea how the quality selection works. Right now, this will download files in the highest quality available.
I haven't use IDM a lot, so I don't know if this method can be used with it.
PowerShell does have an in-built Invoke-WebRequest
command, and MacOS and Linux ship with curl
so we can generate download scripts.
It appears that the "www." in the URL actually determines it's quality. Ex. https://www.4vs5osstge.buzz/ would download 720p and https://4vs5osstge.buzz/ would download 360p
~~I still have no idea how it deciphers the link from "m0weXd1eG1pdS54eXo=a"~~ EDIT: I actually did end up figuring out how to "decode" the URL provided by ping.idocdn.com All you have to do is move the last character in the string to the beginning and then decode it in base64.
am0weXd1eG1pdS54eXo=
decodes to jm0ywuxmiu.xyz
, which is different from 4vs5osstge.buzz
but still works. Maybe the url doesn't actually matter? When you open the dev console on the kissanime page it tells you it's a google drive proxy, so the servers might all be identical.
@KartikSoneji I don't think IDM supports Headers outside of the browser extensions.
@JambDev
It appears that the "www." in the URL actually determines it's quality.
Hmm, quite clever.
EDIT: I actually did end up figuring out how to "decode" the URL provided by ping.idocdn.com All you have to do is move the last character in the string to the beginning and then decode it in base64.
Great! How did you figure it out? I would never have thought that a company would use something like this. Looks like HydraX is going for security through obscurity.
Still haven't been able to test HSL though.
@thorio
Interestingly, you don't have to use the url returned from https://ping.idocdn.com
.
For example, I used 4vs5osstge.buzz
for all requests, and it worked for (all of my) ~140 videos.
If you resolve 4vs5osstge.buzz
and jm0ywuxmiu.xyz
with nslookup
, you will find that they point to different ip addresses.
Maybe the multiple urls are a form of load-balancing?
But it is good that we can decode them now.
@KartikSoneji I don't think IDM supports Headers outside of the browser extensions.
I was afraid of that. There has been some discussion about allowing custom headers in VLC for remote playback, but it does not seem to be implemented currently.
How about generating a PowerShell/cmd/shell script to download the videos?
I still need to work out some details (like hd
/sd
selection and fixing the base64) but this is what I have so far:
# downloadHydraXVideo <slug> <output file>
downloadHydraXVideo(){
url64=$(curl "https://ping.idocdn.com/" -d "slug=$1" | grep -Po '(?<="url":")[a-zA-Z0-9=+-]*?(?=",)');
# TODO: Fix the base64 string by moving the last character to the front.
curl "$(echo $url64 | base64 -d)" -# -C - -H "Referer: https://playhydrax.com/?v=$1" -o "$2";
}
downloadHydraXVideo "VMG9wjJtM" "Video 1.mp4"
downloadHydraXVideo "TiDe3FPHK" "Video 2.mp4"
#...
@KartikSoneji I believe aria2c supports custom headers, as well as multithreaded downloads. I looked into it a while ago, I don't remember why I didn't pursue it further. We should be able to export a file with all the urls and headers required.
@thorio Agree, but that would be an external dependency.
curl
is installed on almost all systems, and Invoke-WebRequest
ships with all modern versions of Windows.
But it can be good as an additonal option though.
There are already aria2c and IDM exporters in the script. I don't believe curl can be multithreaded on the same download?
I'll go ahead and add an action to ping idocdn
and get the url, as well as a new aria2c exporter for this.
Feel free to open a PR for the curl version.
I don't believe curl can be multithreaded on the same download?
Unfortunately, no :(
I'll go ahead and add a step to ping idocdn and get the url
If there is a large delay (> 5-6 hours) between pinging idocdn and downloading the video then you might get an Access Denied
error.
The only way that worked reliably is to ping idocnd right before trying to download the video.
I don't know if that is possible in aria2c
without some shell scripting.
Feel free to open a PR for the curl version.
OK, will do that.
I pushed the action to feature/hydrax-download. You can copy one of the existing exporters and edit it accordingly.
@thorio Thanks, will take a look.
I wrote up a quick bash script that you can provide the stub to and it will retrieve the video. It assumes 720p for the video resolution. ~~I don't know what would happen if that wasn't available and wasn't tested. If you were grabbing a series that only had a 360p version you can remove the www. in the url that it accesses in order to get that one. I might implement a check that handles it. Not sure. For now just remove the www. in the url if you know you can't get a 720p version.~~ It will automatically end up grabbing the 360p version of a 720p version isn't availabele. Resulting JSON says SD where it would normally have SD/HD as the available formats and the download is SD only even with the www. on the front.
--insecure is needed on the second curl statement as it seems that at least one of the domains to download the video from doesn't have a valid SSL certificate so curl aborted on it.
EDIT: If you couldn't tell the default filename is $STUB.mp4 as in if the stub is 18IX9abCy it translates to 18IX9abCy.mp4. This is assuming you don't give it a filename in the first place. Wrap whatever filename you want to give in quotes if it has spaces.
You can replace the curl near the end with aria2c by using this line.
aria2c --check-certificate=false "https://www.$URL/" --referer="https://playhydrax.com/?v=$STUB" -o "$OUT"
#!/usr/bin/env bash
# Video stub
STUB=$1
# Output filename. If none set in command then default to stub.mp4
OUT=$2
if [ -z "$OUT" ]; then OUT="$STUB.mp4"; fi
# Grab JSON data from server
JSON=`curl -s "https://ping.idocdn.com/" -d "slug=$STUB"`
# Cut away and eventually get domain to download video from. -- Load balancing?
# A bit messy here. First cut away excess data and remove enclosing quotes.
# After use a bit of possible bashism to rotate string so that the last character is in front of first character.
# Finally store the URL for the retrieval.
URLB64=`echo $JSON | cut -d, -f2 | cut -d: -f2`
URLB64=`sed -e 's/^"//' -e 's/"$//' <<<"$URLB64"`
LURL=${#URLB64}
URLB64=${URLB64:$LURL-1:1}${URLB64:0:$LURL-1}
URL=`echo $URLB64 | base64 -d`
# Testing to see if HD exists. If it doesn't warn to user that SD is being downloaded instead.
if ! [[ "`echo $JSON | cut -d[ -f2 | cut -d] -f1`" =~ .*"hd".* ]]; then echo -e "\e[31m[WARN]\e[39m No HD version. Downloading SD instead."; fi
curl --insecure "https://www.$URL/" --referer "https://playhydrax.com/?v=$STUB" -o "$OUT"
I've noticed that the requests HydraX's player is making don't directly fetch the mp4 anymore. At the time of writing this, the current method (in the replies to this issue) still works. I am afraid that it might break though with this new web socket approach they are using
This is no longer needed. KissAnime.ru shut down permanently a few minutes ago.
welp
@JambDev When I originally tried this (before this issue was opened) they were using HLS, so they might be changing things around a lot.
@shinji257 There is still kissanime.nz (#39), which also has a hydrax option (Hserver). They also don't seem to have captchas? I will look at adding support for it again when I get the chance, but any help is appreciated.
@thorio Ahh. I was unaware that it also used hydrax.
@shinji257 Yes, and if you look closely you can still find the familiar ping to idocdn
in the network tab.
So it's just a matter of figuring out how they get the link to the iframe.
@thorio @shinji257
kissanime.nz
is down as well, so maybe the outage is temporary?
KissAnime.nz is still working here.
On Sat, Aug 15, 2020, 9:18 AM Kartik Soneji [email protected] wrote:
@thorio https://github.com/thorio @shinji257 https://github.com/shinji257
kissanime.nz is down as well, so maybe the outage is temporary?
[image: image] https://user-images.githubusercontent.com/25801283/90313154-da746080-df27-11ea-99ed-853907388ede.png
— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/thorio/KGrabber/issues/35#issuecomment-674395493, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAB7XDNJTQ3IE5SDSRIYFC3SA2DKDANCNFSM4JTINIAA .
Yay, kissanime.nz
works for me now!
I looked at https://kisscartoon.nz/
and https://kimcartoon.si/
(#46)
For https://kisscartoon.nz/Cartoon/Angry-Birds.88861/Movie-HD?id=71028
# server = hserver | xserver | ptserver | oserver
curl "https://kisscartoon.nz/ajax/anime/load_episodes_v2?s=<server>" -d "episode_id=71028"
Using hserver
:
curl "https://kisscartoon.nz/ajax/anime/load_episodes_v2?s=hserver" -d "episode_id=71028"
{
"status": true,
"value": "<iframe src=\"https:\/\/embed.streamx.me?k=756ecb2dfb240248235934ab3ac1b943&li=71028&tham=1597515866<=hydrax&check_hot=1&qlt=720p&spq=p&prv=bWVkaWEvdGh1bWIvMTcwNDA4XzA1NDYvQW5ncnktQmlyZHMuODgxNTktTW92aWUtSEQuanBnO21lZGlhL3RodW1iLzE3MDQwOF8wNTQ2L0FuZ3J5LUJpcmRzLjg4MTU5LU1vdmllLUhEX21vYmlsZS5qcGc%3D&key=57443b94a0716a10f23334135e9979e1&ua=5f2bf4be52e6a7bdd37bca5ebabb7b2c\" width=\"100%\" height=\"100%\" style=\"border:none;overflow:hidden\" scrolling=\"no\" frameborder=\"0\" allowTransparency=\"true\" allowFullScreen=\"true\" allow=\"accelerometer; autoplay; encrypted-media; gyroscope; picture-in-picture\"><\/iframe>",
"embed": true,
"html5": false,
"type": "",
"sv": "hydrax",
"download_get": ""
}
Using the url from the iframe's src
.
(referer
is needed)
curl \
--referer "https://kisscartoon.nz/" \
"https://embed.streamx.me?k=756ecb2dfb240248235934ab3ac1b943&li=71028&tham=1597515866<=hydrax&check_hot=1&qlt=720p&spq=p&prv=bWVkaWEvdGh1bWIvMTcwNDA4XzA1NDYvQW5ncnktQmlyZHMuODgxNTktTW92aWUtSEQuanBnO21lZGlhL3RodW1iLzE3MDQwOF8wNTQ2L0FuZ3J5LUJpcmRzLjg4MTU5LU1vdmllLUhEX21vYmlsZS5qcGc%3D&key=57443b94a0716a10f23334135e9979e1&ua=5f2bf4be52e6a7bdd37bca5ebabb7b2c"
returns a lot of HTML, but the interesting bit is
<script>
new Player({
"key": "e6d3ce91839c03915de9125be5ed1f47",
"type": "slug",
"value": "3aOgVSRom39",
"width": "100%",
"height": "100%",
"cast": "{}",
"autostart": true,
tracks: [[]]
});
</script>
Which finally gives us the slug
as 3aOgVSRom39
.
The same works for https://kimcartoon.si/
, simply by replacing kisscartoon.nz
with kimcartoon.si
.
Update: https://kissanime.nz
also seems to use the same setup, but they have a LOT of scraping protection (captchas, csrf tokens).
Please help me download this type of link: https://playhydrax.com/?v=GDUc-uLQN Sorry I don't know how to use script, I install on Chrome and open website but still can not load video.
Hi @Thanhcaro Please refer to the usage instructions: https://github.com/thorio/KGrabber#installation-and-usage
Hi @Thanhcaro Please refer to the usage instructions: https://github.com/thorio/KGrabber#installation-and-usage
The script is currently compatible with:
kissanime.ru kimcartoon.to kissasian.sh kisstvshow.to
my website is not in this list.
@Thanhcaro
https://github.com/thorio/KGrabber#compatibility
If you'd like to add a site, please open a new issue on GitHub.
Can you open an issue?
@Thanhcaro
https://github.com/thorio/KGrabber#compatibility
If you'd like to add a site, please open a new issue on GitHub.
Can you open an issue?
Oh Sorry. I will open new issue. Thanks for your help.
Sorry, i thought script can download all hydrax link from any website. I already open new issue, pls do me a favour. Thank you.
Vào 00:46, T.3, 6 Th10, 2020 Kartik Soneji [email protected] đã viết:
Can you let us know the name of your website?
— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/thorio/KGrabber/issues/35#issuecomment-703785742, or unsubscribe https://github.com/notifications/unsubscribe-auth/AQZLVXLK6I7JTD7NFWAGKS3SJIA6XANCNFSM4JTINIAA .
I have been lurking this issue for a while now and was able to implement the hydrax downloader in Python.
But recently, something about the https://ping.iamcdn.net/
has changed. It now returns a 502
error.
Using the question from #54 as an example:
curl "https://ping.iamcdn.net/" -d "slug=7fUvtRbiYk"
Returns:
<html>
<head><title>502 Bad Gateway</title></head>
<body>
<center><h1>502 Bad Gateway</h1></center>
<hr><center>nginx/1.19.6</center>
</body>
</html>
However, it appears that the following steps remained the same. I could download files via trial and error by using the old domains such as: curl "https://x00vfu277x.buzz/" -H "Referer: https://playhydrax.com/?v=TZTopeD-V" -o "test01.mp4"
I tried to dig in the js files for clues, but couldn't find anything :/
@mrwunderbar666 Seems like the ping.iamcdn.net
step is no longer necessary.
Running:
curl -L https://playhydrax.com/?v=TZTopeD-V
The interesting bit:
<script type="text/javascript">
//...
if ("undefined" != typeof ADS)
new ADS(atob("eyJzdGF0dXMiOnRydWUsIm1kNV9pZCI6MTAzNTc3MjUsInNvdXJjZXMiOlsic2QiLCJoZCJdLCJzb3VyY2VfYnJlYWtzIjpbXSwiZG9tYWluIjoiNDd5YW9odWhpbi5idXp6IiwidXNlX3AycCI6dHJ1ZSwidXNlX2Rvd24iOmZhbHNlfQ=="))
var styleMinion = "width:100%;height:100% !important;background: url(https://iamcdn.net/img/block2.jpg) center center no-repeat;background-size:contain;position:absolute"
</script>
decoding the base64 data gives:
{
"status": true,
"md5_id": 10357725,
"sources": ["sd", "hd"],
"source_breaks": [],
"domain": "47yaohuhin.buzz",
"use_p2p": true,
"use_down": false
}
Shell script:
curl -sL "https://playhydrax.com/?v=TZTopeD-V" |\
grep "ADS(" |\
sed -r 's/.*"(.*)".*/\1/' |\
base64 -di
@KartikSoneji Thanks that did the trick!
That is now much easier to implement.
I just noticed that some of the domains have issues with SSL. No matter how I connect to some of them, some throw a handshake error:
HTTPSConnectionPool(host='www.x0g2gwwwr7.xyz', port=443): Max retries exceeded with url: / (Caused by SSLError(SSLError(1, '[SSL: SSLV3_ALERT_HANDSHAKE_FAILURE] sslv3 alert handshake failure (_ssl.c:1123)')))
I wonder, how the browser resolves that, because in curl the option --insecure
does not help for handshake errors
@KartikSoneji @mrwunderbar666 Thanks that did the trick!
That is now much easier to implement.
I just noticed that some of the domains have issues with SSL. No matter how I connect to some of them, some throw a handshake error:
HTTPSConnectionPool(host='www.x0g2gwwwr7.xyz', port=443): Max retries exceeded with url: / (Caused by SSLError(SSLError(1, '[SSL: SSLV3_ALERT_HANDSHAKE_FAILURE] sslv3 alert handshake failure (_ssl.c:1123)')))
I wonder, how the browser resolves that, because in curl the option
--insecure
does not help for handshake errors
You are so talent. Can you tell me the way to get hydrax link to download?
@KartikSoneji @mrwunderbar666 Thanks that did the trick! That is now much easier to implement. I just noticed that some of the domains have issues with SSL. No matter how I connect to some of them, some throw a handshake error:
HTTPSConnectionPool(host='www.x0g2gwwwr7.xyz', port=443): Max retries exceeded with url: / (Caused by SSLError(SSLError(1, '[SSL: SSLV3_ALERT_HANDSHAKE_FAILURE] sslv3 alert handshake failure (_ssl.c:1123)')))
I wonder, how the browser resolves that, because in curl the option
--insecure
does not help for handshake errorsYou are so talent. Can you tell me the way to get hydrax link to download?
A fool does not read the comments above.
wow. seems it was blocked by encryption js
it seems that hydrax.net has become abyss.to. Can anyone still download the files with the curl? If you can, please share the result.
Thanks
@KartikSoneji Do you know about abyss.to? how to do
@lavara123 If you find a website that uses abyss.to, then I'll be happy to update the script.
@lavara123 If you find a website that uses abyss.to, then I'll be happy to update the script.
Thanks a lot, I'm looking forward to it
curl 'https://embed2.megaxfer.ru/embed2/01bded490f99430277078b8ad5ae9af8' \
-H 'user-agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/91.0.4472.114 Safari/537.36' \
-H 'referer: https://www.2embed.ru/' \
--compressed
curl output:
<iframe src="https://playhydrax.com/?v=2FKgbNJtB&sub=https://cc.1clickcdn.ru/8d/3b/8d3b7506346ba4ef945e5274488b094e/eng-6.vtt" width="100%" height="100%" scrolling="no" frameborder="0"
allowFullScreen="true" webkitallowfullscreen="true"
mozallowfullscreen="true"></iframe>
and next link : https://geoip.redirect-ads.com/?v=2FKgbNJtB
Looks like they are using CloudFlare, this is not going to be as easy.
Looks like they are using CloudFlare, this is not going to be as easy.
wget -d --header="User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/91.0.4472.77 Safari/537.36" https://playhydrax.com/?v=2FKgbNJtB
@KartikSoneji @lavara123 did you ever figure out how to bypass the cloudflare I have been stuck for days in figuring out that section I don’t know what checks are being performed
Hi @crypto-0 Before, I was able to bypass Cloudflare's captcha with these options, but I haven't tested it recently.
curl -L --user-agent "Mozilla/5.0" --cipher AES256-SHA256 --tls-max 1.2 "<url>"
In nodejs, you have to set the cipher explictly:
const https = require("https");
async function fetch(url, options){
return new Promise((resolve, reject) => {
let req = https.request(url, options, res => {
let data = [];
res.on("data", chunk => data.push(chunk));
res.on("end", e =>
resolve({
headers: res.headers,
status: res.statusCode,
text: async () => Buffer.concat(data).toString()
})
);
});
req.on("error", reject);
req.end();
});
}
// use
let response = await fetch(url, {
headers: { "User-Agent": "Mozilla/5.0" },
ciphers: "AES256-SHA256"
});
Hope that helps.
Hi @crypto-0 Before, I was able to bypass Cloudflare's captcha with these options, but I haven't tested it recently.
curl -L --user-agent "Mozilla/5.0" --cipher AES256-SHA256 --tls-max 1.2 "<url>"
In nodejs, you have to set the cipher explictly:
const https = require("https"); async function fetch(url, options){ return new Promise((resolve, reject) => { let req = https.request(url, options, res => { let data = []; res.on("data", chunk => data.push(chunk)); res.on("end", e => resolve({ headers: res.headers, status: res.statusCode, text: async () => Buffer.concat(data).toString() }) ); }); req.on("error", reject); req.end(); }); } // use let response = await fetch(url, { headers: { "User-Agent": "Mozilla/5.0" }, ciphers: "AES256-SHA256" });
Hope that helps.
Thank for share. how to download stream ? Can you do it ?
Any chance someone found a working solution to this issue?