Javinizer icon indicating copy to clipboard operation
Javinizer copied to clipboard

WARNING: [Javinizer] Unable reach Javlibrary, enter websession/cookies to use the scraper

Open jvlflame opened this issue 3 years ago • 20 comments

Updated June 16 2024

Join the discord server for support: https://discord.gg/Pds7xCpzpc

Download the updated files: >> 2.5.17_-_v1.01.zip <<

Instructions for both CLI and GUI

1. Open the javinizer powershell module directory with CLI command Javinizer -OpenModule 2. Download the updated script files in the attached 2.5.17.zip and extract the files DIRECTLY into the module folder and replace all existing files

3a. Install and run the flaresolverr application. If you modify the port or need to access the flaresolverr API from a different address, take note of it for the next step. https://github.com/FlareSolverr/FlareSolverr

3b. If you set flaresolverr to a different port or address from its default, open up the files Public/Get-JavlibraryUrl.ps1, Public/Get-JavlibraryData.ps1 and Public/Javinizer.ps1 and update the $FLARESOLVERR_URL variable to point to the correct URL and save.

  1. Check that the scraper.movie.javlibrary (or whichever language) setting is enabled and that javlibrary is added into your scraper metadata settings

  2. Set the throttlelimit setting to 1. The scraper may break often if it is greater than 1.

  3. Test that javlibrary scraper is working in the CLI. View your flaresolverr logs to check that it is solving the cloudflare challenges.

Javinizer -Find SSNI-776 -Javlibrary

Instructions for GUI (Windows)

  1. Folllow the CLI and GUI instructions above to extract the updated files to your module directory

  2. Open PowerShell 7 in administrator mode and run the following to replace your current GUI installation Javinizer -InstallGUI -Force

  3. Open the GUI and confirm that scraping works Javinizer -OpenGUI

Instructions for GUI and CLI (Docker)

  1. SSH into your running container
# Replace "<container_name>" with the name of your javinizer container

docker exec -it <container_name> /bin/bash
  1. Find the javinizer module path inside the container. In most cases it will be /root/.local/share/powershell/Modules/Javinizer/2.5.17. If not, enter PowerShell using pwsh and run (Get-InstalledModule).InstalledLocation.

  2. Use curl (or alternative) to download the 2.5.17.zip file below into the container and copy the contents from Private and Public into their respective directories in the module folder.

  3. If you want to update the GUI, copy and replace the javinizergui.ps1 file from under Universal/Repository in the zip file into home/data/Repository/javinizergui.ps1

  4. You may need to restart the container to propagate changes. (You will need to repeat these steps if you delete/reinstall the container)

jvlflame avatar Dec 25 '20 01:12 jvlflame

I have tried changing the javlibrary.baseurl setting to http://n53i.com, still getting the same error message.

Can you provide more detailed instructions on copying the cookie contents and user agent details? It's not very clear from your instructions

cavalia88 avatar Jul 08 '21 02:07 cavalia88

I have tried changing the javlibrary.baseurl setting to http://n53i.com, still getting the same error message.

Can you provide more detailed instructions on copying the cookie contents and user agent details? It's not very clear from your instructions

@cavalia88 Can you try setting the javlibrary.baseurl to http://www.n53i.com instead? Not sure if that will change anything but I'm not having any issues from that mirror right now.

Basically you want to access javlibrary.com (or whatever your javlibrary.baseurl is set as) via your browser and complete the cloudflare captcha when prompted. After doing so, check the site's browser cookies and copy each of the corresponding cookie content over to Javinizer when it prompts you.

image

Though I did just try it right now and javlibrary isn't giving me the proper cookies in the browser. It's possible that Cloudflare is now properly detecting legitimate (browser) vs bot (javinizer) traffic and isn't providing the cookies in the browser...

EDIT: If the proper cookies aren't showing up, try deleting the cf_clearance cookie from your browser and then refreshing. Rinse and repeat until all the proper cookies show up:

  • cf_chl_2
  • cf_chl_prog
  • cf_clearance

jvlflame avatar Jul 08 '21 02:07 jvlflame

Thank you for the clarification. I realized what the initial problem was on my side. I updated the Javinizer version to the latest version, but was amending the settings file of the earlier version. Changing the javlibrary.baseurl setting to http://n53i.com works now for me

When Javinizer updates to a new version, the old settings are not automatically migrated? We need to manually bring the settings over? I'm using the CLI version via Powershell on Windows 10

cavalia88 avatar Jul 08 '21 03:07 cavalia88

Glad it's working for you now!

If you use the native PowerShell method, Update-Module Javinizer it won't migrate settings. If you use the custom update function Javinizer -UpdateModule then it will.

jvlflame avatar Jul 08 '21 04:07 jvlflame

It seems like n53i.com now have cloudflare as well, and I've been trying to fill the cookies values from javlibrary while on it's baseurl + cookies value from n53i while using it's baseurl as well but both has been failing consistently with errors that shows i need to input captcha

damenootoko avatar Jul 11 '21 18:07 damenootoko

@damenootoko I'm using the default HTTPS javlibrary site to get the cookies and it's still working for me. Make sure when you paste your browser useragent it's all on a single line. I noticed when copying from google it gets separated into 2 lines so you'll need to paste it somewhere first and copy it again if you're using that method.

image

jvlflame avatar Jul 12 '21 00:07 jvlflame

Same here, n53i.com no longer works. Does Javinizer allow scraping from other websites? Or it currently only works with Javlibrary?

Since Javlibrary is causing so many issues, thought it may be better if i switch to another website.

cavalia88 avatar Jul 12 '21 01:07 cavalia88

@cavalia88 Yes, you can disable the javlibrary scraper completely and set your metadata priorities to any scraper of your choice.

Open the settings via the commandline by using Javinizer -OpenSettings.

jvlflame avatar Jul 12 '21 01:07 jvlflame

I found an issue where not using the -Path parameter when running Javinizer would fail to append to necessary cookies onto the scraper.

E.g. running Javinizer -Recurse would fail, but Javinizer -Path . -Recurse would work. Going to push a new release with this fix which will hopefully resolve some people's issues.

jvlflame avatar Jul 12 '21 05:07 jvlflame

I ran into this issue because I was using https://www.b49t.com as a mirror (can't remember why I started using b49t). https://www.n53i.com wasn't working either, so I changed the base url back to https://www.javlibrary.com itself and it worked. I noticed I couldn't find any of the aforementioned cookies for any of the 3 sites on any of Chrome, Firefox, Safari, and I'm not sure why. It looked like I was failing n53i's cloudflare check when I looked. Anyway, hope this helps someone.

yumesakiri avatar Aug 05 '21 20:08 yumesakiri

Looks like this one is happening again. Started a week ago.

Have tried the options but my cookies for the JAVLibrary site do not match what the tool needs, and the mirrors either ask for cookie https://www.n53i.com info or dont seem to work (https://www.b49t.com)

Has anyone got this to work?

natsocenis avatar Jan 26 '22 14:01 natsocenis

Looks like this one is happening again. Started a week ago.

Have tried the options but my cookies for the JAVLibrary site do not match what the tool needs, and the mirrors either ask for cookie https://www.n53i.com info or dont seem to work (https://www.b49t.com)

Has anyone got this to work?

Load the site (www.o58c.com) in an incognito browser window (chrome) to generate the full list of cookies. Even after clearing cookies and site data in a normal tab, I couldn't get all the cookies until I did this.

Should say there are 9 cookies. You will need to select cookies under o58c.com and www.o58c.com directories as the required info is spread across both entries.

Flute7 avatar Feb 24 '22 15:02 Flute7

Having this issue again. I've tried all the different solutions in this thread but I cannot get any of the various mirrors suggested here to give me the cookies that Javinizer wants. Another thing that is strange is that if I run the CLI Javinizer multiple times, it will intermittently sometimes be able to scrape one or two movies (that it had given an error for previously.)

chronotank79 avatar Nov 16 '22 18:11 chronotank79

UPDATE: using a VPN while scraping seems to have fixed the issue. Turn off VPN, error again. Turn on VPN, works again.

I think if you are sorting a large number of files, cloudflare may eventually decide to require additional security from your IP address. What is strange about this is that Javlibrary works perfectly normally in my normal web browsers.

chronotank79 avatar Nov 18 '22 18:11 chronotank79

Cannot locate all cookie values. Have tried Firefox, Chrome and Edge. Are there other workarounds?

S1lv3rL10n avatar Feb 20 '24 16:02 S1lv3rL10n