trpg-dnd-4e-db
trpg-dnd-4e-db copied to clipboard
Sheep-y
“SSL Handshake failed”
Whenever I try to connect I get this error. Login and password are fine since I cut and pasted the same one directly to the DDI website and it works fine. Tested under both Windoes10 and Mac High Sierra. I also have a subscription.
Thanks.
I am having the same issue under 64-bit Linux.
15:39:00 [INFO] Status: Initialising.
15:39:00 [FINE] State: Busy
15:39:00 [CONFIG] Thread count set to 7 plus one controll thread
15:39:00 [CONFIG] Java 9.0.4+11 on Linux amd64
15:39:00 [CONFIG] Max memory 8,028MB
15:39:00 [INFO] Main GUI initialised.
15:39:00 [INFO] Status: Opening database.
15:39:00 [FINE] State: Busy
15:39:00 [INFO] Opening local database /home/redacted/4e-compendium/dnd4_compendium.database
15:39:00 [CONFIG] Opened database dnd4_compendium.database
15:39:00 [INFO] Status: Checking data.
15:39:00 [FINE] Tables loaded. Checking data version.
15:39:00 [CONFIG] Database version 20160718, opened.
15:39:00 [FINE] Loading categories.
15:39:00 [FINE] Loaded 18 categories.
15:39:00 [INFO] No need to back up dnd4_compendium.database (11,264 <= 0 or 12k)
15:39:00 [FINE] Progress: 0.
15:39:00 [FINE] Progress: 0.
15:39:00 [FINE] Progress: 0.
15:39:00 [FINE] State: Can Download
15:39:00 [INFO] Status: Ready to download.
15:39:00 [FINE] Progress: 0.
15:39:00 [INFO] Skipping update check. Next check at 2019-01-12T14:15:15.274056Z
15:39:06 [INFO] Status: Starting download.
15:39:06 [FINE] State: Running
15:39:06 [FINE] Progress: -1.
15:39:06 [INFO] Initialise web crawler
15:39:06 [INFO] Initialise download browser
15:39:06 [CONFIG] WebView Agent: Mozilla/5.0 (Linux x86_64) AppleWebKit/604.1 (KHTML, like Gecko) JavaFX/9 Safari/604.1
15:39:06 [CONFIG] Timeout 30,000 ms / Interval 750 ms
15:39:06 [FINE] State: Running
15:39:06 [INFO] Status: Testing login.
15:39:07 [WARNING] Testing login finished exceptionally: java.util.concurrent.ExecutionException: java.lang.Throwable: SSL handshake failed
15:39:07 [WARNING] Download failed: java.lang.Throwable: SSL handshake failed
at javafx.web/javafx.scene.web.WebEngine$LoadWorker.describeError(WebEngine.java:1431)
at javafx.web/javafx.scene.web.WebEngine$LoadWorker.dispatchLoadEvent(WebEngine.java:1370)
at javafx.web/javafx.scene.web.WebEngine$LoadWorker.access$1200(WebEngine.java:1248)
at javafx.web/javafx.scene.web.WebEngine$PageLoadListener.dispatchLoadEvent(WebEngine.java:1235)
at javafx.web/com.sun.webkit.WebPage.fireLoadEvent(WebPage.java:2499)
at javafx.web/com.sun.webkit.WebPage.fwkFireLoadEvent(WebPage.java:2343)
at javafx.web/com.sun.webkit.network.URLLoader.twkDidFail(Native Method)
at javafx.web/com.sun.webkit.network.URLLoader.notifyDidFail(URLLoader.java:892)
at javafx.web/com.sun.webkit.network.URLLoader.lambda$didFail$6(URLLoader.java:875)
at javafx.graphics/com.sun.javafx.application.PlatformImpl.lambda$runLater$9(PlatformImpl.java:418)
at java.base/java.security.AccessController.doPrivileged(Native Method)
at javafx.graphics/com.sun.javafx.application.PlatformImpl.lambda$runLater$10(PlatformImpl.java:417)
at javafx.graphics/com.sun.glass.ui.InvokeLaterDispatcher$Future.run(InvokeLaterDispatcher.java:96)
at javafx.graphics/com.sun.glass.ui.gtk.GtkApplication._runLoop(Native Method)
at javafx.graphics/com.sun.glass.ui.gtk.GtkApplication.lambda$runLoop$11(GtkApplication.java:277)
at java.base/java.lang.Thread.run(Thread.java:844)
15:39:07 [FINE] State: Can Download
15:39:07 [INFO] Status: SSL handshake failed.
15:39:07 [FINE] Progress: 0.
It might possibly have to do with attempting a connection using HTTP rather than HTTPS. Trying to connect to http://www.wizards.com/dndinsider/compendium/glossary.aspx?id=138 gets me redirected to the "main page" of the compendium. Whereas https://www.wizards.com/dndinsider/compendium/glossary.aspx?id=138 actually gets me to the entry for immobilized. This is using SeaMonkey 2.49.4.
Below is the output from the worker tab in the Compendium downloader:
18:14:57 [FINE] Browsing http://www.wizards.com/dndinsider/compendium/glossary.aspx?id=138: READY => SCHEDULED
18:14:57 [FINE] Browsing http://www.wizards.com/dndinsider/compendium/glossary.aspx?id=138: SCHEDULED => RUNNING
18:14:58 [FINE] Browsing https://www.wizards.com/dndinsider/compendium/: RUNNING => FAILED
18:14:58 [WARNING] Failed to load webpage https://www.wizards.com/dndinsider/compendium/
In SceneMain.java after line 420 in the "action_download" method I added a single line of code that seems to have resolved the issue. The trustAllSSL method was already in the code so I just had to make a call to it. After doing this the SSL Handshake error goes away, but I am not sure if everything works because I don't have a valid ddi login.
setStatus( "Starting download" );
Net.trustAllSSL(); //<---- new line i added
loader.startDownload();
stateRunning();
Here is a link to a new jar with this single change if anyone wants to try it out. http://thancode.org/downloads/4e_compendium_trust_all_downloader.zip
Great, thanks. I tested it. It logins in now but gets "download error"
If it helps I can share my login/pwd with you via messaging. I bought a month sub just to test this and its just a test account and pwd.
I think I have it figured out. WotC must have changed their site to force https and updated their certs at the same time. When the downloader attempts to load an http page it will reload as https, but it does not load the same page, it loads a generic page. That is why when it tries to load the first set of date it fails. It attempts to go to the page in http mode and gets reloaded to a generic https page. To fix this I have updated all the links to use https.
I am currently running the download and it seems to be working. I will verify it in the morning and if so I can upload a new jar along with the code files I changed so the author can easily fix the issue in the official repo.
Here is a link to the final fix. This includes a patch to the latest codebase to fix the issue as well as the latest jar file for those who still want to use this in the meantime.
http://thancode.org/downloads/ssh_trust_all_patch_plus_jar.zip
Trying it now, so far so good. Thanks very much, natiels. I have been paying 10$/month for a decade just to use the compendium as the other parts now don't work (like the links from the Compendium) so prefer to have an offline version.
Even more thanks to Sheep-y. His downloader has been a God send to me. The online site is just too sluggish and lacks some critical search features. After the effort he put in and how much time it has saved me I am more than happy to donate a few hrs to help keep it going.
Hi, am I missing something, or is there something specific to do to run this updated jar? I still get the SSL error despite running this version...
01:38:34 [INFO] Status: Starting download.
01:38:34 [FINE] State: Running
01:38:34 [FINE] Progress: -1.
01:38:34 [CONFIG] WebView Agent: Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/604.1 (KHTML, like Gecko) JavaFX/8.0 Safari/604.1
01:38:34 [CONFIG] Timeout 30,000 ms / Interval 1,000 ms
01:38:34 [FINE] State: Running
01:38:34 [INFO] Status: Testing login.
01:38:35 [FINE] Testing login finished normally.
01:38:35 [INFO] Requires login: https://www.wizards.com/dndinsider/compendium/login.aspx?page=glossary&id=62
01:38:36 [INFO] Status: Opening login page.
01:38:37 [WARNING] Opening login page finished exceptionally: java.util.concurrent.ExecutionException: java.lang.Throwable: SSL handshake failed
01:38:37 [WARNING] Download failed: java.lang.Throwable: SSL handshake failed
at javafx.scene.web.WebEngine$LoadWorker.describeError(WebEngine.java:1463)
at javafx.scene.web.WebEngine$LoadWorker.dispatchLoadEvent(WebEngine.java:1402)
at javafx.scene.web.WebEngine$LoadWorker.access$1200(WebEngine.java:1280)
at javafx.scene.web.WebEngine$PageLoadListener.dispatchLoadEvent(WebEngine.java:1267)
at com.sun.webkit.WebPage.fireLoadEvent(WebPage.java:2499)
at com.sun.webkit.WebPage.fwkFireLoadEvent(WebPage.java:2343)
at com.sun.webkit.network.URLLoader.twkDidFail(Native Method)
at com.sun.webkit.network.URLLoader.notifyDidFail(URLLoader.java:883)
at com.sun.webkit.network.URLLoader.lambda$didFail$104(URLLoader.java:866)
at com.sun.javafx.application.PlatformImpl.lambda$null$172(PlatformImpl.java:295)
at java.security.AccessController.doPrivileged(Native Method)
at com.sun.javafx.application.PlatformImpl.lambda$runLater$173(PlatformImpl.java:294)
at com.sun.glass.ui.InvokeLaterDispatcher$Future.run(InvokeLaterDispatcher.java:95)
at com.sun.glass.ui.win.WinApplication._runLoop(Native Method)
at com.sun.glass.ui.win.WinApplication.lambda$null$147(WinApplication.java:177)
at java.lang.Thread.run(Unknown Source)
01:38:37 [FINE] State: Can Download
01:38:37 [INFO] Status: SSL handshake failed.
01:38:37 [FINE] Progress: 0.
You'll have to download the jar I listed in the link above and execute it manually with "java -jar 4e_compendium_downloader.jar". The project has not been updated so just downloading from releases won't work for you. I just tested it again a minute ago to see if I got any errors and I didn't.
I managed to make it work by re-updating my java version. I'm not sure if it was outdated before (I forgot to note down the version), but now it's finally downloading. Thank you for the fix, it's a lifesaver!
The patch worked for me. Here is the patch in case the download natiels provided goes down.
Index: java/db4e/controller/Crawler.java
IDEA additional info:
Subsystem: com.intellij.openapi.diff.impl.patch.CharsetEP
<+>UTF-8
===================================================================
--- java/db4e/controller/Crawler.java (revision fe795de964b8b7edee7d588b9721d68c89e2121a)
+++ java/db4e/controller/Crawler.java (date 1546982772462)
@@ -25,6 +25,8 @@
private final WebEngine browser;
+ public static final String COMPENDIUM_URL = "https://www.wizards.com/dndinsider/compendium/";
+
public Crawler ( WebEngine browser ) {
this.browser = browser;
}
@@ -77,7 +79,7 @@
}
void openLoginPage () {
- browse( "http://ddi.wizards.com/" );
+ browse( "https://ddi.wizards.com/" );
}
synchronized void login ( String username, String password ) throws InterruptedException, TimeoutException {
@@ -87,7 +89,7 @@
}
void getCategoryXsl ( Category cat ) {
- browse( "http://www.wizards.com/dndinsider/compendium/xsl/" + cat.id + ".xsl" );
+ browse( COMPENDIUM_URL + "xsl/" + cat.id + ".xsl" );
}
Document getCategoryXsl () throws InterruptedException, TimeoutException {
@@ -97,7 +99,7 @@
}
void getCategoryData ( Category cat ) {
- browse( "http://www.wizards.com/dndinsider/compendium/CompendiumSearch.asmx/ViewAll?tab=" + cat.id );
+ browse( COMPENDIUM_URL + "CompendiumSearch.asmx/ViewAll?tab=" + cat.id );
}
List<Entry> openCategory () throws InterruptedException, TimeoutException {
@@ -132,7 +134,7 @@
}
private void openEntry ( String url ) {
- browse( "http://www.wizards.com/dndinsider/compendium/" + url );
+ browse( COMPENDIUM_URL + url );
}
void getEntry ( Entry entry ) throws InterruptedException, TimeoutException {
Index: java/db4e/data/EntryDownloaded.java
IDEA additional info:
Subsystem: com.intellij.openapi.diff.impl.patch.CharsetEP
<+>UTF-8
===================================================================
--- java/db4e/data/EntryDownloaded.java (revision fe795de964b8b7edee7d588b9721d68c89e2121a)
+++ java/db4e/data/EntryDownloaded.java (date 1547043302938)
@@ -1,5 +1,7 @@
package db4e.data;
+import db4e.controller.Crawler;
+
public class EntryDownloaded extends Entry {
private boolean hasContent; // Indicate whether this entry has content in database.
@@ -22,7 +24,7 @@
}
@Override public String getUrl() {
- return "http://www.wizards.com/dndinsider/compendium/" + getId();
+ return Crawler.COMPENDIUM_URL + getId();
}
@Override public <T extends Entry> T cloneTo( T copy ) {
Index: java/db4e/Main.java
IDEA additional info:
Subsystem: com.intellij.openapi.diff.impl.patch.CharsetEP
<+>UTF-8
===================================================================
--- java/db4e/Main.java (revision fe795de964b8b7edee7d588b9721d68c89e2121a)
+++ java/db4e/Main.java (date 1547000466827)
@@ -18,6 +18,7 @@
import java.util.regex.Matcher;
import java.util.regex.Pattern;
import javax.swing.JOptionPane;
+import sheepy.util.Net;
import sheepy.util.ResourceUtils;
import sheepy.util.Utils;
@@ -48,6 +49,7 @@
simulate.set( false );
log.setLevel( Level.CONFIG );
try {
+ Net.trustAllSSL();
Class.forName( "javafx.stage.Stage" ); // OpenJDK does not come with JavaFX by default
MainApp.run( args );
} catch ( ClassNotFoundException ex ) {
I'm also getting an error message, although it's slightly different:
MacBook-Pro:ssh_trust_all_patch_plus_jar Resist$ java -jar "4e_compendium_downloader.jar" -Djdk.tls.client.protocols="TLSv1" 15:20:07 [WARNING] Opening login page finished exceptionally: java.util.concurrent.ExecutionException: java.lang.Throwable: SSL handshake failed 15:20:07 [WARNING] Download failed: java.lang.Throwable: SSL handshake failed at javafx.scene.web.WebEngine$LoadWorker.describeError(WebEngine.java:1461) at javafx.scene.web.WebEngine$LoadWorker.dispatchLoadEvent(WebEngine.java:1400) at javafx.scene.web.WebEngine$LoadWorker.access$1200(WebEngine.java:1278) at javafx.scene.web.WebEngine$PageLoadListener.dispatchLoadEvent(WebEngine.java:1265) at com.sun.webkit.WebPage.fireLoadEvent(WebPage.java:2443) at com.sun.webkit.WebPage.fwkFireLoadEvent(WebPage.java:2287) at com.sun.webkit.network.URLLoader.twkDidFail(Native Method) at com.sun.webkit.network.URLLoader.notifyDidFail(URLLoader.java:883) at com.sun.webkit.network.URLLoader.lambda$didFail$102(URLLoader.java:866) at com.sun.javafx.application.PlatformImpl.lambda$null$173(PlatformImpl.java:295) at java.security.AccessController.doPrivileged(Native Method) at com.sun.javafx.application.PlatformImpl.lambda$runLater$174(PlatformImpl.java:294) at com.sun.glass.ui.InvokeLaterDispatcher$Future.run(InvokeLaterDispatcher.java:95)
The same thing happens whether I use the -Djdk.tls.client.protocols="TLSv1" or not.
I'm really wanting to get a download before the end of the year, when DDI and the compendium will end! Argh! Thanks for the help everyone, esp @natiels !
Linkyu mentioned getting it to work by updating java. Might be worth a try.
Hmmm, I've just uninstalling and then re-installing the latest Java from the Java website, but no luck. I'm on a Mac, and I've also tried it using the .exe on Windows 7. I'm launching it with java -jar 4e_compendium_downloader.jar and I've also tried it with java -jar "4e_compendium_downloader.jar" -Djdk.tls.client.protocols="TLSv1". That opens the app and does the "Checking data". Then it says "Ready to download". I've copy-pasted my username and password in. I then click "Download" and it says "Testing login", then "Opening login page", and then goes to "SSL Handshake failed".
Is it possible to just grab the dnd4_compendium.database file from someone else? Is that all that is needed? If so, please email at t.hugb.s@ the popular email provided by google.
Or do you have any other ideas?