anigrab icon indicating copy to clipboard operation
anigrab copied to clipboard

AnimePahe started providing fake kwik links in the api response

Open newaccforhc opened this issue 3 years ago • 11 comments


name: "🐞 Bug report" about: Report a bug title: "[Bug] AnimePahe started providing fake kwik links in the api response" labels: "bug"


Describe the bug

AnimePahe started providing fake kwik links in the api response (those links go to 404). So, downloads from Animepahe aren't working. Now you need to go through adfly link before getting the real kwik link.

Command You Entered

anigrab -u https://animepahe.com/anime/1e1d8732-a053-24d1-72cb-c8c6a895e3cb

Expected behavior

Should give direct download links.

Actual behavior

Threw an error.

Other details

Maybe some adfly bypassing method can help to resolve this.

On a positive note, they removed captcha to open kwik links now. So, cheers!

newaccforhc avatar Nov 20 '20 16:11 newaccforhc

I'm not sure how to handle this one but it might be that some header is required or something. I've checked but I honestly don't know, I believe the extractor is still fine but as you've stated, it's being given a fake link.

ngomile avatar Nov 20 '20 19:11 ngomile

@ngomile Oh.. No.. it has nothing to do with headers.. It's just you get a totally different link if you go through adfly or shst link..

Maybe doing something to bypass adfly or shst links would work.. There are many packages/modules to do that on github but, most of them are not maintained now.. So, idk if they still work or not.. But, you can check.

newaccforhc avatar Nov 20 '20 20:11 newaccforhc

I got this error on "animePahe"

Bypassing captcha please wait... (node:6004) UnhandledPromiseRejectionWarning: TypeError: response.body.match is not a function or its return value is not iterable at module.exports.extract (C:\Users\------\AppData\Roaming\npm\node_modules\anigrab\src\extractors\kwik.js:51:44) (node:6004) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). (rejection id: 1) (node:6004) [DEP0018] DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise rejections that are not handled will terminate the Node.js process with a non-zero exit code.

is this issue on my system or with code?

uali6981 avatar Dec 01 '20 19:12 uali6981

@ngomile looks like you can refer here: https://github.com/KevCui/animepahe-dl/issues/8

newaccforhc avatar Dec 03 '20 06:12 newaccforhc

@newaccforhc my bad that tool in m3u8. So I did not get enough downloading speed on that software

uali6981 avatar Dec 03 '20 20:12 uali6981

Oh no.. I'm not talking about m3u8. Sorry for not mentioning before, What I wanted to say is that, you can get the real kwik links by referring to that tool. That tool is using stream links which have /e/ in them. If you replace it with /f/ then you get the kwik download links.

newaccforhc avatar Dec 04 '20 02:12 newaccforhc

So I was doing a little experimenting trying to get this working again like it was, I haven't looked deeply into the link @newaccforhc provided. I'll do that soon though seeing as how what I came up with is inefficient and uses way too much data and CPU. Essentially, I tried to use puppeteer-core to get the link from adfly. Starts too many Chrome processes and is too slow. For the curious souls the code looked like this:

async function getRealKwikLink(url) {
    const BLOCKED_RESOURCES_REG = /ads|inpagepush|css|facebook|analytics|cloud|png|hcaptcha|gif/;
    let kwikURL = '';
    let browser;

    try {
        browser = await puppeteer.launch({
            executablePath: browserExecutablePath,
        });
        const page = await browser.newPage();
        page.setRequestInterception(true);
        page.on('request', request => {
            if (BLOCKED_RESOURCES_REG.test(request.url())) {
                request.abort();
            } else {
                request.continue();
            }
        });

        await page.goto(url, { waitUntil: 'networkidle2' });
        await page.waitForSelector('#skip_bu2tton', { visible: true });
        kwikURL = await page.evaluate(
            '(() => document.querySelector("#skip_bu2tton").getAttribute("href"))()'
        );
    } catch (error) {
        console.error(error);
    } finally {
        await browser.close();
    }

    return kwikURL;
}

But it actually worked as shown below after entering this command anigrab beastars -u -s animepahe:

Capture

Just way too slow, this will be sorted out eventually... maybe.

ngomile avatar Dec 06 '20 11:12 ngomile

Screenshot (14) help it keeps going like this

IshigamiYuu avatar Dec 09 '20 06:12 IshigamiYuu

@IshigamiYuu animepahe isn't working at the moment.

ngomile avatar Dec 09 '20 06:12 ngomile

The "kwik" links aren't fake. Kwik has a "DRM" system (really not a DRM but since you're having issues with it, we're going to act like it is one).

I'm not familiar with Node so I'll just drop you the method to get valid stream url(s) from AnimePahe.

First, make an API call to get the "kwik" URL. Then, make a GET request to that URL with "https://animepahe.com/" as the referer (referer in headers). This now allows you to access the site for further extraction. Now, you need to extract the m3u8 URL. There should be some tools in Node that allows JS evaluation but for a faster approach, use regex, here's the algorithm to extract "kwik" m3u8.

To stream/download the m3u8 is also quite troublesome due to the above mentioned "DRM" system, hence, you need to use the "kwik" url (not https://kwik.cx but the url from which you extracted the m3u8) as the referer in the download header.

Then, you should be able to stream/download from AnimePahe.

This is also the reason why web browsers are failing to stream even from the official AnimePahe site.

Good luck!

justfoolingaround avatar May 16 '21 06:05 justfoolingaround