haskell-language-server `hls-eval-plugin` test failures on Gentoo Linux

I am getting a lot of test failures from the hls-eval-plugin test suite using 55d90241440c5282b9e60cf80a98fd8698e1f8d9:

62 out of 65 tests failed (272.92s)

Full build log is here.

Your environment

GHC 9.0.2 on Gentoo Linux. All packages are installed through the system package manager.

I am building the following packages from 55d90241440c5282b9e60cf80a98fd8698e1f8d9:

ghcide
haskell-language-server
hls-alternate-number-format-plugin
hls-change-type-signature-plugin
hls-code-range-plugin
hls-eval-plugin
hls-floskell-plugin
hls-fourmolu-plugin
hls-graph
hls-hlint-plugin
hls-plugin-api
hls-rename-plugin
hls-retrie-plugin
hls-test-utils

I am building the rest from Hackage with the following versions (let me know if you need more):

aeson-2.0.3.0
data-default-0.7.1.1
diff-0.4.0
dlist-1.0
extra-1.7.10
ghc-paths-0.1.0.12
hashable-1.4.0.2
lens-5.1.1
lsp-1.5.0.0
lsp-types-1.5.0.0
megaparsec-9.2.1
parser-combinators-1.3.0
pretty-simple-4.1.1.0
quickcheck-2.14.2
safe-exceptions-0.1.7.1
unliftio-0.2.18
unordered-containers-0.2.17.0

Aug 27 '22 16:08 hololeap

Are these tests passing elsewhere? If so, can someone give me a link to where they are passing?

Sep 02 '22 17:09 hololeap

They are passing in the Github CI. link

Sep 02 '22 17:09 pepeiborra

Perhaps you want to test using the environment variables LSP_TEST_LOG_STDERR=1 and LSP_TEST_LOG_MESSAGES=1 to get more information on why it fails.

Sep 07 '22 11:09 wz1000

@wz1000, thanks for the hint about the environment variables.

It looks like this has something to do with Gentoo's build sandboxing. All the tests pass when run outside the sandbox.

I am seeing blocked network calls being logged. It looks like cabal is being run as a part of the test, which is trying to download something.

{
    "jsonrpc": "2.0",
    "method": "textDocument/publishDiagnostics",
    "params": {
        "diagnostics": [
            {
                "message": "Error when calling cabal exec -v0 -- ghc --print-libdir\n\ndieVerbatim: user error (cabal: '/usr/bin/curl' exited with an error:\ncurl: (6) Could not resolve host: hackage.haskell.org\n)\n\n",
                "range": {
                    "end": {
                        "character": 0,
                        "line": 1
                    },
                    "start": {
                        "character": 0,
                        "line": 0
                    }
                },
                "severity": 1,
                "source": "cradle"
            }
        ],
        "uri": "file:///var/tmp/portage/dev-haskell/hls-eval-plugin-1.3.0.0/work/hls-eval-plugin-1.3.0.0/test/testdata/TLocalImport.hs",
        "version": 0
    }
}

Full build log with LSP_TEST_LOG_STDERR=1 and LSP_TEST_LOG_MESSAGES=1 is here.

Sep 23 '22 18:09 hololeap

It looks like I'm running into the same thing for hls-module-rename-plugin-1.1.0.0:

{
    "jsonrpc": "2.0",
    "method": "textDocument/publishDiagnostics",
    "params": {
        "diagnostics": [
            {
                "message": "Error when calling cabal exec -v0 -- ghc --print-libdir\n\ndieVerbatim: user error (cabal: '/usr/bin/curl' exited with an error:\ncurl: (6) Could not resolve host: hackage.haskell.org\n)\n\n",
                "range": {
                    "end": {
                        "character": 0,
                        "line": 1
                    },
                    "start": {
                        "character": 0,
                        "line": 0
                    }
                },
                "severity": 1,
                "source": "cradle"
            }
        ],
        "uri": "file:///var/tmp/portage/dev-haskell/hls-module-name-plugin-1.1.0.0/work/hls-module-name-plugin-1.1.0.0/test/testdata/canonicalize/Lib/A.hs",
        "version": 0
    }
}

FAIL (16.59s)
    Received an unexpected message from the server:
    Was parsing: Pattern match failure in do expression at test/Main.hs:48:7-38
    But the last message received was:
    {
        "id": 2,
        "jsonrpc": "2.0",
        "result": []
    }
    Use -p '/Fix#3047/' to rerun this test only.

1 out of 6 tests failed (24.41s)
Test suite tests: FAIL

Full build log

This test passes outside of the sandbox.

Sep 23 '22 21:09 hololeap

There are more seemingly-related test failures, this time from ghcide:

typecheck-all-parents-of-interest

FAIL (2.13s)
      test/exe/Main.hs:724:
      Incorrect number of diagnostics for Uri {getUri = "file:///var/tmp/portage/dev-haskell/ghcide-1.8.0.0/temp/extra-dir-8862631353299/A.hs"}, expected [] but got [Diagnostic {_range = Range {_start = Positio
n {_line = 5, _character = 4}, _end = Position {_line = 5, _character = 5}}, _severity = Just DsError, _code = Just (InR "-Wdeferred-type-errors"), _source = Just "typecheck", _message = "\8226 Couldn't match e
xpected type \8216Int\8217 with actual type \8216Bool\8217\n\8226 In the expression: y\n  In an equation for \8216x\8217: x = y", _tags = Nothing, _relatedInformation = Nothing},Diagnostic {_range = Range {_sta
rt = Position {_line = 5, _character = 4}, _end = Position {_line = 5, _character = 5}}, _severity = Just DsError, _code = Just (InR "-Wdeferred-type-errors"), _source = Just "typecheck", _message = "\8226 Coul
dn't match expected type \8216Int\8217 with actual type \8216Bool\8217\n\8226 In the expression: y\n  In an equation for \8216x\8217: x = y", _tags = Nothing, _relatedInformation = Nothing}]
      Use -p '/typecheck-all-parents-of-interest/' to rerun this test only.

simple plugin

FAIL (7.77s)
    test/exe/Main.hs:1196:
    Could not find (DsError,(9,15),"Variable not in scope: c",Nothing) in [Diagnostic {_range = Range {_start = Position {_line = 0, _character = 0}, _end = Position {_line = 1, _character = 0}}, _severity = Just DsError, _code = Nothing, _source = Just "cradle", _message = "Error when calling cabal exec -v0 -- ghc --print-libdir\n\ndieVerbatim: user error (cabal: '/usr/bin/curl' exited with an error:\ncurl: (6) Could not resolve host: hackage.haskell.org\n)\n\n", _tags = Nothing, _relatedInformation = Nothing}]
    Use -p '/simple plugin/' to rerun this test only.

notification handlers run in priority order

FAIL
      Exception: Timed out waiting to receive a message from the server.
      Use -p '/notification handlers run in priority order/' to rerun this test only.

ignore-fatal-warning

FAIL (9.15s)
        test/exe/Main.hs:2512:
        Got unexpected diagnostics for Uri {getUri = "file:///var/tmp/portage/dev-haskell/ghcide-1.8.0.0/temp/extra-dir-8862631353468/IgnoreFatal.hs"} got [Diagnostic {_range = Range {_start = Position {_line = 0, _character = 0}, _end = Position {_line = 1, _character = 0}}, _severity = Just DsError, _code = Nothing, _source = Just "cradle", _message = "Error when calling cabal exec -v0 -- ghc --print-libdir\n\ndieVerbatim: user error (cabal: '/usr/bin/curl' exited with an error:\ncurl: (6) Could not resolve host: hackage.haskell.org\n)\n\n", _tags = Nothing, _relatedInformation = Nothing}]
        Use -p '/ignore-fatal-warning/' to rerun this test only.

simple-multi-test

FAIL (7.49s)
        test/exe/Main.hs:2532:
        A should typecheck
        Use -p '$0=="ghcide.cradle.multi.simple-multi-test"' to rerun this test only.

simple-multi-test2

FAIL
        Exception: Timed out waiting to receive a message from the server.
        Last message received:
        {
            "jsonrpc": "2.0",
            "method": "window/logMessage",
            "params": {
                "message": "Live bytes: 0.00MB Heap size: 0.00MB",
                "type": 3
            }
        }
        Use -p '/simple-multi-test2/' to rerun this test only.

simple-multi-test3

FAIL
        Exception: Timed out waiting to receive a message from the server.
        Last message received:
        {
            "jsonrpc": "2.0",
            "method": "window/logMessage",
            "params": {
                "message": "Live bytes: 0.00MB Heap size: 0.00MB",
                "type": 3
            }
        }
        Use -p '/simple-multi-test3/' to rerun this test only.

simple-multi-def-test

FAIL
        Exception: Timed out waiting to receive a message from the server.
        Last message received:
        {
            "jsonrpc": "2.0",
            "method": "window/logMessage",
            "params": {
                "message": "Live bytes: 0.00MB Heap size: 0.00MB",
                "type": 3
            }
        }
        Use -p '/simple-multi-def-test/' to rerun this test only.

simple-subdirectory

FAIL (5.22s)
        test/exe/Main.hs:2520:
        Could not find (DsWarning,(2,0),"Top-level binding",Nothing) in [Diagnostic {_range = Range {_start = Position {_line = 0, _character = 0}, _end = Position {_line = 1, _character = 0}}, _severity = Just DsError, _code = Nothing, _source = Just "cradle", _message = "Error when calling cabal exec -v0 -- ghc --print-libdir\n\ndieVerbatim: user error (cabal: '/usr/bin/curl' exited with an error:\ncurl: (6) Could not resolve host: hackage.haskell.org\n)\n\n", _tags = Nothing, _relatedInformation = Nothing}]
        Use -p '/simple-subdirectory/' to rerun this test only.

ghcide command line.works

  ghcide command line
    works:                                                                            FAIL (4.47s)
      test/exe/Main.hs:2814:
      expected: ExitSuccess
       but got: ExitFailure 1
      Use -p '/ghcide command line.works/' to rerun this test only.

13 out of 344 tests failed (2982.71s)

(Three tests fail regardless of the network-sandbox. See #3221.)

Full build log (raw file is ~15MB)

Sep 24 '22 15:09 hololeap

The problem is that Tasty runs test in parallel by default, and most HLS test suites do not handle this well. Set TASTY_NUM_THREADS to 1

Sep 24 '22 21:09 pepeiborra

Did you try Pepe's suggestion? did it help?

Oct 05 '22 11:10 michaelpj

Closing as old and quiet.

Jan 16 '24 18:01 michaelpj

haskell-language-server haskell-language-server copied to clipboard

`hls-eval-plugin` test failures on Gentoo Linux

Your environment

haskell-language-server
haskell-language-server copied to clipboard