jadx icon indicating copy to clipboard operation
jadx copied to clipboard

java.lang.IllegalArgumentException: Comparison method violates its general contract!

Open w296488320 opened this issue 2 years ago • 26 comments

Please describe what you did before the error occurred. IMPORTANT! If the error occurs with a specific APK file please attach or provide link to apk file!

  • Jadx version: 1.4.3
  • Java version: 11.0.12
  • Java VM: JetBrains s.r.o OpenJDK 64-Bit Server VM
  • Platform: Windows 10 (10.0 amd64)
  • Max heap size: 30688 MB
  • Program args: -Xms128M -XX:MaxRAMPercentage=70.0 -XX:+UseG1GC -Dawt.useSystemAAFontSettings=lcd -Dswing.aatext=true
java.lang.IllegalArgumentException: Comparison method violates its general contract!
	at java.base/java.util.ComparableTimSort.mergeHi(ComparableTimSort.java:870)
	at java.base/java.util.ComparableTimSort.mergeAt(ComparableTimSort.java:487)
	at java.base/java.util.ComparableTimSort.mergeCollapse(ComparableTimSort.java:413)
	at java.base/java.util.ComparableTimSort.sort(ComparableTimSort.java:213)
	at java.base/java.util.Arrays.sort(Arrays.java:1249)
	at java.desktop/sun.awt.shell.Win32ShellFolderManager2.get(Win32ShellFolderManager2.java:313)
	at java.desktop/sun.awt.shell.ShellFolder.get(ShellFolder.java:259)
	at java.desktop/javax.swing.plaf.metal.MetalFileChooserUI$DirectoryComboBoxModel.addItem(MetalFileChooserUI.java:1029)
	at java.desktop/javax.swing.plaf.metal.MetalFileChooserUI.doDirectoryChanged(MetalFileChooserUI.java:717)
	at java.desktop/javax.swing.plaf.metal.MetalFileChooserUI$5.propertyChange(MetalFileChooserUI.java:806)
	at java.desktop/java.beans.PropertyChangeSupport.fire(PropertyChangeSupport.java:341)
	at java.desktop/java.beans.PropertyChangeSupport.firePropertyChange(PropertyChangeSupport.java:333)
	at java.desktop/java.beans.PropertyChangeSupport.firePropertyChange(PropertyChangeSupport.java:266)
	at java.desktop/java.awt.Component.firePropertyChange(Component.java:8754)
	at java.desktop/javax.swing.JFileChooser.setCurrentDirectory(JFileChooser.java:608)
	at java.desktop/javax.swing.JFileChooser.<init>(JFileChooser.java:362)
	at java.desktop/javax.swing.JFileChooser.<init>(JFileChooser.java:338)
	at jadx.gui.ui.dialog.FileDialog$FileChooser.<init>(FileDialog.java:156)
	at jadx.gui.ui.dialog.FileDialog.buildFileChooser(FileDialog.java:138)
	at jadx.gui.ui.dialog.FileDialog.show(FileDialog.java:74)
	at jadx.gui.ui.MainWindow.openFileOrProject(MainWindow.java:295)
	at jadx.gui.ui.MainWindow.processCommandLineArgs(MainWindow.java:252)
	at jadx.gui.ui.MainWindow.init(MainWindow.java:247)
	at java.desktop/java.awt.event.InvocationEvent.dispatch(InvocationEvent.java:313)
	at java.desktop/java.awt.EventQueue.dispatchEventImpl(EventQueue.java:776)
	at java.desktop/java.awt.EventQueue$4.run(EventQueue.java:727)
	at java.desktop/java.awt.EventQueue$4.run(EventQueue.java:721)
	at java.base/java.security.AccessController.doPrivileged(Native Method)
	at java.base/java.security.ProtectionDomain$JavaSecurityAccessImpl.doIntersectionPrivilege(ProtectionDomain.java:85)
	at java.desktop/java.awt.EventQueue.dispatchEvent(EventQueue.java:746)
	at java.desktop/java.awt.EventDispatchThread.pumpOneEventForFilters(EventDispatchThread.java:203)
	at java.desktop/java.awt.EventDispatchThread.pumpEventsForFilter(EventDispatchThread.java:124)
	at java.desktop/java.awt.EventDispatchThread.pumpEventsForHierarchy(EventDispatchThread.java:113)
	at java.desktop/java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:109)
	at java.desktop/java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:101)
	at java.desktop/java.awt.EventDispatchThread.run(EventDispatchThread.java:90)

w296488320 avatar Aug 11 '22 07:08 w296488320

Same issue here, see #1606

xxr0ss avatar Aug 11 '22 08:08 xxr0ss

Same issue here, see #1606 Is there a better solution right now , Very strange, yesterday or good to use, today suddenly appeared this problem, the computer restart also doesn't work

w296488320 avatar Aug 11 '22 09:08 w296488320

It's likely to be a bug introduced in a recent Windows 11 update. I did a temporary fix: https://github.com/xxr0ss/jadx/tree/fix-broken-filedialog.

xxr0ss avatar Aug 11 '22 10:08 xxr0ss

@xxr0ss But this time it is not Windows 11 but 10 according to the bug report. Unfortunately I wasn't able to reproduce this bug on both Windows versions. The affected systems seem to have a special property like a special connected device (e.g. via USB) or a installed software.

jpstotz avatar Aug 11 '22 10:08 jpstotz

So as @xxr0ss in his fix uses java.awt.FileDialog, I think I will add it as an option to use as a workaround for this issue. This change is also related to #1213 and commit 79405f94e0c64639b9faee50da8eff21d0cb96d6 in branch file-dialog.

skylot avatar Aug 11 '22 12:08 skylot

@jpstotz @skylot I've debugged FileDialog and it turned out to be a bug in javax.swing.JFileChooser.

Here's the quick fix: Add -Djava.util.Arrays.useLegacyMergeSort=true to JVM_OPTS

xxr0ss avatar Aug 11 '22 12:08 xxr0ss

@xxr0ss Thanks for your investigation. But I am not sure If I understand the result you have posted. If the bug is caused by JFileChooser how can switching the sort algorithm prevent it from happening? Can you give us more details on the problem or are you preparing a OoenJDK bug report?

jpstotz avatar Aug 11 '22 15:08 jpstotz

So here's the detail @jpstotz : ShellFolder is being sorted when the exception throws. JFileChooser use Arrays.sort() to sort ShellFolder. The NEW TimSort is introduced in JDK 7, which requires strict comparison: 0 should be returned when two comparables are equal. Let's see how sun.awt.shell.ShellFolder implements comparasion:

    private static final Comparator<File> FILE_COMPARATOR = new Comparator<File>() {
        public int compare(File f1, File f2) {
            ShellFolder sf1 = null;
            ShellFolder sf2 = null;

            if (f1 instanceof ShellFolder) {
                sf1 = (ShellFolder) f1;
                if (sf1.isFileSystem()) {
                    sf1 = null;
                }
            }
            if (f2 instanceof ShellFolder) {
                sf2 = (ShellFolder) f2;
                if (sf2.isFileSystem()) {
                    sf2 = null;
                }
            }

            if (sf1 != null && sf2 != null) {
                return sf1.compareTo(sf2);
            } else if (sf1 != null) {
                // Non-file shellfolders sort before files
                return -1;
            } else if (sf2 != null) {
                return 1;
            } else {
                String name1 = f1.getName();
                String name2 = f2.getName();

                // First ignore case when comparing
                int diff = name1.compareToIgnoreCase(name2);
                if (diff != 0) {
                    return diff;
                } else {
                    // May differ in case (e.g. "mail" vs. "Mail")
                    // We need this test for consistent sorting
                    return name1.compareTo(name2);
                }
            }
        }
    };

You can see the return value can only be 1 or -1. And that's what the TimSort is unhappy about. So a quick fix is to force Arrays.sort() to use legacy sort.

    public static void sort(Object[] a) {
        if (LegacyMergeSort.userRequested)
            legacyMergeSort(a);
        else
            ComparableTimSort.sort(a, 0, a.length, null, 0, 0);
    }

To me this is quite unusual😂!!! I've been using Java 11 for a long time, but I didn't have this problem until recently.

xxr0ss avatar Aug 11 '22 17:08 xxr0ss

@jpstotz I don't have an account in OpenJDK Community and I cann't even find where to sign up one😓. And I've being a bit busy lately. It'll be nice of you to report this bug to OpenJDK!

xxr0ss avatar Aug 11 '22 18:08 xxr0ss

@xxr0ss Thanks for the detailed explanation, I think I got the point but I have to look at the code myself because name1.compareToIgnoreCase and name1.compareTo can return 0.

Last time I checked the Java bug tracker was still "ruled" (provided) by Oracle and they don't provide accounts unless you have contributed code to OpenJDK. Therefore for Java developers there is only an email address where you can send bug reports which will create a new issue. But without account you can't comment or anything else.

I will try to create a minimal reproduceable example to demonstrate the bug.

jpstotz avatar Aug 11 '22 18:08 jpstotz

Understood👍@jpstotz

I will also try to find out what's so special about the ShellFolders on my computer that causes TimSort to fail, just when I have time.

xxr0ss avatar Aug 11 '22 19:08 xxr0ss

@xxr0ss Debugging Jadx and the code at java.desktop/sun.awt.shell.Win32ShellFolderManager2.get(Win32ShellFolderManager2.java:313) I have to admit that I don't see how sun.awt.shell.ShellFolder.FILE_COMPARATOR is used. May be you have discovered a second code path of the same bug?

According to the stack trace the error starts here: Win32ShellFolderManager2

public Object get(String key)
    ...
    File[] secondLevelFolders = checkFiles(desktop.listFiles());
    Arrays.sort(secondLevelFolders);

At run-time if I look into secondLevelFolders it contains at the beginning special folders like This PC, Network, Libraries and then all files on my desktop. I don't think the real files on the desktop cause problems, so most likely the special folders are the problem. Is it possible that on your system is one special folder twice in this list?

I tried to create a simple example of an Array containing special folder entries. Unfortunately the module system of Java does not allow to use/invoke ShellFolder and related classes outside of the module java.desktop even with reflection. At such times you really want to throw this damn module system against the next wall :(

jpstotz avatar Aug 12 '22 13:08 jpstotz

@jpstotz 😵I'm sorry I messed things up!! I debugged again and FILE_COMPARATOR is not used. The comparison that is used is in sun.awt.shell.Win32ShellFolderManager2

static int compareShellFolders(Win32ShellFolder2 sf1, Win32ShellFolder2 sf2)

Duplicated folder entries do exist on my computer! I logged Arrays.sort()

// sun.awt.shell.Win32ShellFolderManager2
        folders.add(desktop);
        // Add all second level folders
        File[] secondLevelFolders = checkFiles(desktop.listFiles());
log>    Arrays.sort(secondLevelFolders);
        for (File secondLevelFolder : secondLevelFolders) {
Documents, ::{20D04FE0-3AEA-1069-A2D8-08002B30309D}, ::{F02C1A0D-BE21-4350-88B0-7367FC96EF3C}, 
::{031E4825-7B94-4DC3-B131-E946B44C8DD5}, OneDrive, ShellFolder: , Music, Downloads, Pictures, 
Videos, Desktop, Desktop, Downloads, Documents, Pictures, XXX

You can see Desktop, Downloads, Pictures, Documents are duplicated.

And I'm not sure if the return value of comparison for TimSort is requied be no other than -1, 0, 1?

// sun.awt.shell.Win32ShellFolder2
    public int compareTo(File file2) {
        ...
        return Win32ShellFolderManager2.compareShellFolders(this, (Win32ShellFolder2) file2);
    }

This comparison returns value like below, which

14, 5, 3, -1, 25932, 25923, -10

xxr0ss avatar Aug 12 '22 14:08 xxr0ss

@xxr0ss values other than [-1, 0, 1] returned by compareTo should not be a problem:

Return a negative integer, zero, or a positive integer as this object is less than, equal to, or greater than the specified object.

jpstotz avatar Aug 12 '22 18:08 jpstotz

@jpstotz are there any ShellFolders on your computer that have the same name, while one is a special entry and the other not?

I logged isSpecial() for ShellFolders being compared. See here: image 0 is returned here, but I do not think a non-special ShellFolder should be equal to a special one when they have the same name.

xxr0ss avatar Aug 12 '22 19:08 xxr0ss

@xxr0ss That is indeed strange and might be the cause. I assume you are printing the elements using toString(). I noticed that special folders return a different value (usually the used CLSID) using getPath() (and getPath() is used by `Win32ShellFolder2.equals).

Can you identify which element on your Windows Desktop is creating this special documents folder entry?

jpstotz avatar Aug 13 '22 13:08 jpstotz

@jpstotz Oh! I just stopped getting the exception after I removed all special folders (Documents, Pictures ...) from Quick Access.

And I went to C:\Users\<UserName> and added them all (Documents, Pictures...) back to Quick Access, I debugged again and the special and non-special Documents both exist and the code here was executed, however, the exception does not show up any more.

image

xxr0ss avatar Aug 13 '22 14:08 xxr0ss

I came across this issue while troubleshooting the same problem for a user on a different project. Here is quite a long thread detailing that effort. If you scroll all the way to the bottom, you'll find two posts where I explain the cause.

The short version is: This line in sun.awt.shell.Win32ShellFolderManager2.compareShellFolders() is wrong. The condition should be a conjunction instead of a disjunction.

I submitted a bug report to OpenJDK earlier today. I'd encourage you to do the same; maybe we can light a fire under them to get this fixed.

uckelman avatar Mar 25 '23 15:03 uckelman

looks like the JDK tracks it as https://bugs.openjdk.org/browse/JDK-8305072

mbien avatar Aug 03 '23 23:08 mbien

looks like the JDK tracks it as https://bugs.openjdk.org/browse/JDK-8305072

They seem to have decided to close it being unable to reproduce.

edemen avatar Aug 07 '23 17:08 edemen

@edemen if someone knows how to reproduce it.. let them know. The test code is in the bug report, what is missing is the right combination of "special" and "non special" windows folders - whatever that is.

mbien avatar Aug 07 '23 17:08 mbien

Nice. They don't need to be able to reproduce the problem to see what's wrong---it should be obvious just from reading the code. Argh.

uckelman avatar Aug 07 '23 18:08 uckelman

@edemen if someone knows how to reproduce it.. let them know. The test code is in the bug report, what is missing is the right combination of "special" and "non special" windows folders - whatever that is.

For a long time I had two desktop.ini files on my Desktop. Somehow then one of them disappeared months later. I suspect this has something to do with mixing shortcuts/settings that are configured for the individual user, and the ones shared for all users. I don't have the time to verify this yet though. Specifically, the user may not always be able to override shared shortcuts/settings, so perhaps this can lead to this multiple files with the same name in a virtual/combined special folder, like Desktop? Idk.

edemen avatar Aug 07 '23 20:08 edemen

Just a quick note to let you know I'm facing this same problem with someone using OpenJDK 21. Is there anything I can do to help?

uwemock avatar Feb 22 '24 13:02 uwemock

@uwemock workaround for this issue (legacy sort JVM flag: -Djava.util.Arrays.useLegacyMergeSort=true) already in master, so you can use latest unstable build. This issue kept open to track progress on JDK fix, but looks like fix will not be applyied because this issue is hard to reproduce :slightly_frowning_face:

skylot avatar Feb 22 '24 18:02 skylot

My fix was merged: https://github.com/openjdk/jdk/pull/18126

It ought to appear in the next JDK release.

uckelman avatar Apr 29 '24 12:04 uckelman