haskell-language-server HLS stuck (init then no progress) since multi home support

HLS stuck (init then no progress) since multi home support

Open guibou opened this issue 6 months ago • 15 comments

First, I apologize for the lack of reproducibility in this issue. I'm a bit lost here and I need your help. I'll try to gather as much information as possible, but for now I don't have much.

Since we upgraded at work HLS from 2.4.0.0 to 2.6.0.0, HLS is "stuck" on some (but not all) of our projects.

By stuck, I mean that HLS starts, gather information for the build flags thank to our hie.yaml / cradle:bios:shell command, and then nothing else is happening. CPU runs at 100% (of one core) and the latest entry in the log is:

2024-02-04T12:10:01.928343Z | Info | Making new HscEnv. In-place unit ids: [ main-5f50f5bd074bca15292d92df75278f6ab5b479ff

Your environment

We are using Linux. I'm debuging that on NixOS, but users had observed the same problem on Ubuntu.

GHC 9.6.4 (But same problem with 9.8.1), installed from nixpkgs. HLS is installed from nixpkgs too.

The project is built with a custom build system using nix and we use a cradle:bios:shell command to output the build flags. Note that the command works fine, so HLS is not blocked on this.

A few notes about our build system.

Our build system is based on nix, but that's orthogonal to our problem (I think) because mostly what it does is populating the environment with the correct version of GHC and providing flags for HLS.

Note that we have one specifitity which may be responsible for the problem. We have a "main project" and multiples subprojects.

The main project have a ghc database populated only with packages from hackage and built using nixpkgs and it contains 2000+ haskell module. HLS works fine here!
Our subprojects have a ghc database populated with packages from hackage as well as from the main project. The ones from hackage are built using nixpkgs haskell infrastructure, but the ones from our main project are built using our custom build system. So maybe there is something wrong / incorrect in the way we are populating the package database. Note that ghc-pkg check only warns about missing haddock entries. None of our subproject work correctly with HLS, they are all stuck.

HLS bisect

I've bisected HLS (starting on tag 2.6.0.0 and 2.4.0.0 as "bad" and "good" and I hit:

7db6215c6da39b6550d7c1ce6ea5b1bbe6e6dfde is the first bad commit
commit 7db6215c6da39b6550d7c1ce6ea5b1bbe6e6dfde
Author: wz1000 <[email protected]>
Date:   Wed Nov 22 16:08:04 2023 -0800

    Add support for multi unit argument syntax (#3462)

As the possibly offending commit.

Note that we are not using the multi unit argument syntax in our project, but next step (after having a look in diagonal on this commit to see if I can see something which could lead to the "live lock") will be to try to convert our codebase to the multi unit syntax.

Steps to reproduce

No simple reproducer yet, sorry. The main codebase is private and includes client IP that we cannot unfortunately share easilly. I'm working on making a simpler reproducer.

Expected behaviour

Actual behaviour

Debug information

Here is the content of the log, using --debug --log ...:

(some name had been replaced by foo or bar because that's a private "client" project)

2024-02-04T12:10:01.453174Z | Info | haskell-language-server version: 2.4.0.0 (GHC: 9.6.4) (PATH: /home/guillaume/srcs/haskell-language-server/dist-newstyle/build/x86_64-linux/ghc-9.6.4/haskell-language-server-2.4.0.0/x/haskell-language-server/build/haskell-language-server/haskell-language-server) (GIT hash: 7db6215c6da39b6550d7c1ce6ea5b1bbe6e6dfde)
2024-02-04T12:10:01.454151Z | Info | Directory: /home/guillaume/foo/bar
2024-02-04T12:10:01.454302Z | Info | Starting (haskell-language-server) LSP server...
  GhcideArguments {argsCommand = LSP, argsCwd = Nothing, argsShakeProfiling = Nothing, argsTesting = False, argsExamplePlugin = False, argsLogLevel = Debug, argsLogFile = Just "/tmp/hls.log", argsLogStderr = True, argsLogClient = False, argsThreads = 0, argsProjectGhcVersion = False}
  PluginIds: [ pragmas-suggest
             , pragmas-completion
             , retrie
             , ghcide-completions
             , alternateNumberFormat
             , ghcide-code-actions-bindings
             , explicit-fields
             , ghcide-hover-and-symbols
             , ghcide-code-actions-type-signatures
             , fourmolu
             , ghcide-extend-import-action
             , ghcide-code-actions-fill-holes
             , importLens
             , LSPRecorderCallback
             , cabal
             , qualifyImportedNames
             , moduleName
             , splice
             , stylish-haskell
             , changeTypeSignature
             , hlint
             , class
             , ormolu
             , callHierarchy
             , stan
             , ghcide-type-lenses
             , codeRange
             , cabal-fmt
             , eval
             , rename
             , ghcide-code-actions-imports-exports
             , gadt
             , overloaded-record-dot
             , ghcide-core
             , explicit-fixity
             , pragmas-disable ]
2024-02-04T12:10:01.456361Z | Info | Logging heap statistics every 60.00s
2024-02-04T12:10:01.459423Z | Info | Starting LSP server...
  If you are seeing this in a terminal, you probably should have run WITHOUT the --lsp option!
  PluginIds: [ pragmas-suggest
             , pragmas-completion
             , retrie
             , ghcide-completions
             , alternateNumberFormat
             , ghcide-code-actions-bindings
             , explicit-fields
             , ghcide-hover-and-symbols
             , ghcide-code-actions-type-signatures
             , fourmolu
             , ghcide-extend-import-action
             , ghcide-code-actions-fill-holes
             , importLens
             , LSPRecorderCallback
             , cabal
             , qualifyImportedNames
             , moduleName
             , splice
             , stylish-haskell
             , changeTypeSignature
             , hlint
             , class
             , ormolu
             , callHierarchy
             , stan
             , ghcide-type-lenses
             , codeRange
             , cabal-fmt
             , eval
             , rename
             , ghcide-code-actions-imports-exports
             , gadt
             , overloaded-record-dot
             , ghcide-core
             , explicit-fixity
             , pragmas-disable ]
2024-02-04T12:10:01.459712Z | Info | Starting server
2024-02-04T12:10:01.460728Z | Debug | LSP: set new config: {  }
2024-02-04T12:10:01.461119Z | Info | Started LSP server in 0.00s
2024-02-04T12:10:01.493040Z | Debug | ghc --print-libdir
2024-02-04T12:10:01.494652Z | Debug | Setting initial dynflags...
2024-02-04T12:10:01.494725Z | Debug | shouldRunSubset: True
2024-02-04T12:10:01.494834Z | Debug | Initializing exports map from hiedb
2024-02-04T12:10:01.495046Z | Info | Registering IDE configuration: IdeConfiguration {workspaceFolders = fromList [NormalizedUri 5128461851892427305 "file:///home/guillaume/foo/bar"], clientSettings = hashed (Just (Object (fromList [])))}
2024-02-04T12:10:01.495942Z | Debug | LSP: set new config: {
  "cabalFormattingProvider": "cabalfmt",
  "formattingProvider": "ormolu",
  "plugin": {
    "ghcide-completions": { "config": { "snippetsOn": false } },
    "hlint": { "diagnosticsOn": false }
  }
}
2024-02-04T12:10:01.496076Z | Debug | Configuration changed: Config {checkParents = CheckOnSave, checkProject = True, formattingProvider = "ormolu", cabalFormattingProvider = "cabalfmt", maxCompletions = 40, plugins = fromList [(PluginId "ghcide-completions",PluginConfig {plcGlobalOn = True, plcCallHierarchyOn = True, plcCodeActionsOn = True, plcCodeLensOn = True, plcDiagnosticsOn = True, plcHoverOn = True, plcSymbolsOn = True, plcCompletionOn = True, plcRenameOn = True, plcSelectionRangeOn = True, plcFoldingRangeOn = True, plcConfig = fromList [("snippetsOn",Bool False)]}),(PluginId "hlint",PluginConfig {plcGlobalOn = True, plcCallHierarchyOn = True, plcCodeActionsOn = True, plcCodeLensOn = True, plcDiagnosticsOn = False, plcHoverOn = True, plcSymbolsOn = True, plcCompletionOn = True, plcRenameOn = True, plcSelectionRangeOn = True, plcFoldingRangeOn = True, plcConfig = fromList []})]}
2024-02-04T12:10:01.496597Z | Debug | Shake session initialized
2024-02-04T12:10:01.497159Z | Debug | Finished build session
AsyncCancelled
2024-02-04T12:10:01.497167Z | Debug | Restarting build session due to config change
Action Queue: []
Keys: [GetClientSettings; ]
Aborting previous build session took 0.00s 
2024-02-04T12:10:01.497460Z | Debug | LSP: set new config: {
  "cabalFormattingProvider": "cabalfmt",
  "formattingProvider": "ormolu",
  "plugin": {
    "ghcide-completions": { "config": { "snippetsOn": false } },
    "hlint": { "diagnosticsOn": false }
  }
}
2024-02-04T12:10:01.497595Z | Debug | Configuration changed: Config {checkParents = CheckOnSave, checkProject = True, formattingProvider = "ormolu", cabalFormattingProvider = "cabalfmt", maxCompletions = 40, plugins = fromList [(PluginId "ghcide-completions",PluginConfig {plcGlobalOn = True, plcCallHierarchyOn = True, plcCodeActionsOn = True, plcCodeLensOn = True, plcDiagnosticsOn = True, plcHoverOn = True, plcSymbolsOn = True, plcCompletionOn = True, plcRenameOn = True, plcSelectionRangeOn = True, plcFoldingRangeOn = True, plcConfig = fromList [("snippetsOn",Bool False)]}),(PluginId "hlint",PluginConfig {plcGlobalOn = True, plcCallHierarchyOn = True, plcCodeActionsOn = True, plcCodeLensOn = True, plcDiagnosticsOn = False, plcHoverOn = True, plcSymbolsOn = True, plcCompletionOn = True, plcRenameOn = True, plcSelectionRangeOn = True, plcFoldingRangeOn = True, plcConfig = fromList []})]}
2024-02-04T12:10:01.497777Z | Debug | Finished build session
AsyncCancelled
2024-02-04T12:10:01.497789Z | Debug | Restarting build session due to config change
Action Queue: []
Keys: [GetClientSettings; ]
Aborting previous build session took 0.00s 
2024-02-04T12:10:01.497976Z | Debug | VFS: opening file:///home/guillaume/foo/bar/bar-simulation/src/BioModels/Bar/Model.hs
2024-02-04T12:10:01.498242Z | Debug | LSP: set new config: {
  "cabalFormattingProvider": "cabalfmt",
  "formattingProvider": "ormolu",
  "plugin": {
    "ghcide-completions": { "config": { "snippetsOn": false } },
    "hlint": { "diagnosticsOn": false }
  }
}
2024-02-04T12:10:01.498374Z | Debug | Configuration changed: Config {checkParents = CheckOnSave, checkProject = True, formattingProvider = "ormolu", cabalFormattingProvider = "cabalfmt", maxCompletions = 40, plugins = fromList [(PluginId "ghcide-completions",PluginConfig {plcGlobalOn = True, plcCallHierarchyOn = True, plcCodeActionsOn = True, plcCodeLensOn = True, plcDiagnosticsOn = True, plcHoverOn = True, plcSymbolsOn = True, plcCompletionOn = True, plcRenameOn = True, plcSelectionRangeOn = True, plcFoldingRangeOn = True, plcConfig = fromList [("snippetsOn",Bool False)]}),(PluginId "hlint",PluginConfig {plcGlobalOn = True, plcCallHierarchyOn = True, plcCodeActionsOn = True, plcCodeLensOn = True, plcDiagnosticsOn = False, plcHoverOn = True, plcSymbolsOn = True, plcCompletionOn = True, plcRenameOn = True, plcSelectionRangeOn = True, plcFoldingRangeOn = True, plcConfig = fromList []})]}
2024-02-04T12:10:01.498481Z | Debug | Set files of interest to: fromList [(NormalizedFilePath "/home/guillaume/foo/bar/bar-simulation/src/BioModels/Bar/Model.hs",Modified {firstOpen = True})]
2024-02-04T12:10:01.498524Z | Debug | Finished build session
AsyncCancelled
2024-02-04T12:10:01.498636Z | Debug | Restarting build session due to config change
Action Queue: []
Keys: [ GetClientSettings; 
      , IsFileOfInterest; /home/guillaume/foo/bar/bar-simulation/src/BioModels/Bar/Model.hs
      , GetModificationTime; /home/guillaume/foo/bar/bar-simulation/src/BioModels/Bar/Model.hs ]
Aborting previous build session took 0.00s 
2024-02-04T12:10:01.498819Z | Debug | Finished build session
AsyncCancelled
2024-02-04T12:10:01.498826Z | Debug | Restarting build session due to /home/guillaume/foo/bar/bar-simulation/src/BioModels/Bar/Model.hs (modified)
Action Queue: []
Keys: [ GetClientSettings; 
      , IsFileOfInterest; /home/guillaume/foo/bar/bar-simulation/src/BioModels/Bar/Model.hs
      , GetModificationTime; /home/guillaume/foo/bar/bar-simulation/src/BioModels/Bar/Model.hs ]
Aborting previous build session took 0.00s 
2024-02-04T12:10:01.499013Z | Debug | Opened text document: file:///home/guillaume/foo/bar/bar-simulation/src/BioModels/Bar/Model.hs
2024-02-04T12:10:01.499710Z | Info | Cradle path: bar-simulation/src/BioModels/Bar/Model.hs
2024-02-04T12:10:01.500061Z | Debug | Cradle: Cradle{ cradleRootDir = "/home/guillaume/foo/bar", cradleOptsProg = CradleAction: Bios}
2024-02-04T12:10:01.500133Z | Info | invoking build tool to determine build flags (this may take some time depending on the cache)
2024-02-04T12:10:01.501180Z | Info | nix run .#haskell_hie_bios
  Environment Variables
    HIE_BIOS_OUTPUT: /tmp/nix-shell.VGJ87T/HIE_BIOS_OUTPUT2310259-0
    HIE_BIOS_DEPS: /tmp/nix-shell.VGJ87T/HIE_BIOS_DEPS2310259-1
    HIE_BIOS_ARG: /home/guillaume/foo/bar/bar-simulation/src/BioModels/Bar/Model.hs
2024-02-04T12:10:01.507464Z | Debug | Done initializing exports map from hiedb. Size: 232
2024-02-04T12:10:01.523970Z | Debug | LSP: set new config: {
  "cabalFormattingProvider": "cabalfmt",
  "formattingProvider": "ormolu",
  "plugin": {
    "ghcide-completions": { "config": { "snippetsOn": false } },
    "hlint": { "diagnosticsOn": false }
  }
}
2024-02-04T12:10:01.524150Z | Debug | Configuration changed: Config {checkParents = CheckOnSave, checkProject = True, formattingProvider = "ormolu", cabalFormattingProvider = "cabalfmt", maxCompletions = 40, plugins = fromList [(PluginId "ghcide-completions",PluginConfig {plcGlobalOn = True, plcCallHierarchyOn = True, plcCodeActionsOn = True, plcCodeLensOn = True, plcDiagnosticsOn = True, plcHoverOn = True, plcSymbolsOn = True, plcCompletionOn = True, plcRenameOn = True, plcSelectionRangeOn = True, plcFoldingRangeOn = True, plcConfig = fromList [("snippetsOn",Bool False)]}),(PluginId "hlint",PluginConfig {plcGlobalOn = True, plcCallHierarchyOn = True, plcCodeActionsOn = True, plcCodeLensOn = True, plcDiagnosticsOn = False, plcHoverOn = True, plcSymbolsOn = True, plcCompletionOn = True, plcRenameOn = True, plcSelectionRangeOn = True, plcFoldingRangeOn = True, plcConfig = fromList []})]}
2024-02-04T12:10:01.524468Z | Debug | Restarting build session due to config change
Action Queue: [Outline]
Keys: [ GetClientSettings; 
      , IsFileOfInterest; /home/guillaume/foo/bar/bar-simulation/src/BioModels/Bar/Model.hs ]
Aborting previous build session took 0.00s 
2024-02-04T12:10:01.524470Z | Debug | Finished build session
AsyncCancelled
2024-02-04T12:10:01.524619Z | Debug | LSP: set new config: {
  "cabalFormattingProvider": "cabalfmt",
  "formattingProvider": "ormolu",
  "plugin": {
    "ghcide-completions": { "config": { "snippetsOn": false } },
    "hlint": { "diagnosticsOn": false }
  }
}
2024-02-04T12:10:01.524767Z | Debug | Configuration changed: Config {checkParents = CheckOnSave, checkProject = True, formattingProvider = "ormolu", cabalFormattingProvider = "cabalfmt", maxCompletions = 40, plugins = fromList [(PluginId "ghcide-completions",PluginConfig {plcGlobalOn = True, plcCallHierarchyOn = True, plcCodeActionsOn = True, plcCodeLensOn = True, plcDiagnosticsOn = True, plcHoverOn = True, plcSymbolsOn = True, plcCompletionOn = True, plcRenameOn = True, plcSelectionRangeOn = True, plcFoldingRangeOn = True, plcConfig = fromList [("snippetsOn",Bool False)]}),(PluginId "hlint",PluginConfig {plcGlobalOn = True, plcCallHierarchyOn = True, plcCodeActionsOn = True, plcCodeLensOn = True, plcDiagnosticsOn = False, plcHoverOn = True, plcSymbolsOn = True, plcCompletionOn = True, plcRenameOn = True, plcSelectionRangeOn = True, plcFoldingRangeOn = True, plcConfig = fromList []})]}
2024-02-04T12:10:01.525189Z | Debug | Finished build session
AsyncCancelled
2024-02-04T12:10:01.525192Z | Debug | Restarting build session due to config change
Action Queue: [Outline]
Keys: [ GetClientSettings; 
      , IsFileOfInterest; /home/guillaume/foo/bar/bar-simulation/src/BioModels/Bar/Model.hs ]
Aborting previous build session took 0.00s 
2024-02-04T12:10:01.666155Z | Debug | Using saved setting for 'extra-substituters = s3://devops-ci-infra-prod-caching-nix?region=eu-central-1&profile=nix-daemon' from ~/.local/share/nix/trusted-settings.json.
2024-02-04T12:10:01.666345Z | Debug | Using saved setting for 'extra-trusted-public-keys = footwo:04t6bF1/peQlZWVpYPN0BraxIV2pdlN2005Vi0hUvso=' from ~/.local/share/nix/trusted-settings.json.
2024-02-04T12:10:01.861453Z | Debug | ghc --print-libdir
2024-02-04T12:10:01.861671Z | Debug | Session loading result: Right (ComponentOptions {componentOptions = ["-package-db","/nix/store/jfsgbpj35qawldf8l4hsdf3n02hkk3aa-ghc-9.6.4-with-packages/lib/ghc-9.6.4/lib/package.conf.d","-package-id","Glob-0.10.2-CpOsTsmdARvEPdjPebI11A","-package-id","HTTP-4000.4.1-EKnzcD6hfpl1BQJiPiwloJ","-package-id","HUnit-1.6.2.0-2aRzyZJbb23A4S28an7ssH","-package-id","KdTree-0.2.2.1-MBdfGOdlmx3EQ6yp2OYeY","-package-id","MonadPrompt-1.0.0.5-5XVDi5Bd1nRJOYeU6Yp9j8","-package-id","MonadRandom-0.6-9xVn8iouXSf1b7xrdfDSvH","-package-id","OneTuple-0.4.1.1-413yz2qDv5HLK54zgzfSJ3","-package-id","Only-0.1-2OJr6Dq0xOm75uZ80XlI7x","-package-id","PyF-0.11.1.1-BM6WWdGi3w5JMhibLrvv5o","-package-id","QuickCheck-2.14.3-1qva61ERgFK7Yb7Pf064C7","-package-id","StateVar-1.2.2-LMwI0SGrUtP1MlPXMPkYAS","-package-id","ad-4.5.4-6XNetNqHe1o2yQlSttIufa","-package-id","adjunctions-4.4.2-DsQWuFmAS8I9HEc39gS0yh","-package-id","aeson-2.1.2.1-DVku1e5GfueGwbaN3T30pl","-package-id","aeson-extra-0.5.1.3-9CoSk7OK8ha5iGv2yMIg5","-package-id","aeson-pretty-0.8.10-BObHbmtCyOt18MUgBikOUO","-package-id","algebraic-graphs-0.7-LlwSMGVdwGcBH6Pla4IbMc","-package-id","ansi-terminal-1.0.2-GmRyinTsbXe6lY3Bm6ersV","-package-id","ansi-terminal-types-0.11.5-7QpUULXXpCL6RLITE1JUOk","-package-id","ap-normalize-0.1.0.1-IUcGeS13Dzf8YYrPtNt3Df","-package-id","appar-0.1.8-2UYkbPOz3CVDU6UBsF1opz","-package-id","asn1-encoding-0.9.6-KEgc1BnKiZAtQTmIgzYPx","-package-id","asn1-parse-0.9.5-Im15w3mNjgH14NY9cepoiM","-package-id","asn1-types-0.3.4-9cGpIWMIXWzLpCuCITp1OC","-package-id","assoc-1.1-JHIqWCoBZTM80hzyPBJB13","-package-id","async-2.2.5-CPZtCUdA82NK4rSfETI3Rt","-package-id","attoparsec-0.14.4-1Dl2rWDHAlAHqZP2120DHU","-package-id","attoparsec-aeson-2.1.0.0-3XMhm8NkcfFFMYh3vxPqjs","-package-id","attoparsec-iso8601-1.1.0.1-9joUqNBWA4pGGHHTAMCIDp","-package-id","auto-update-0.1.6-ADHZym4WmY5G8RwewI8v4E","-package-id","barbies-2.0.5.0-1EwXzonZ9zoH6ThlshSNUF","-package-id","base-compat-0.13.1-G7NGIqA1Gm7N8bViVREwP","-package-id","base-compat-batteries-0.13.1-HJCVv1SaM2B87GijOeLP77","-package-id","base-orphans-0.9.1-KAZ8G7uKs3r7jrb0e6niG4","-package-id","base16-1.0-G40XvPvZIJnFTZJFDIrgor","-package-id","base64-bytestring-1.2.1.0-K1erZvkjHObEU0kBt3tz3o","-package-id","basement-0.0.16-DD6H8SGD4AW2EysJRRfDZR","-package-id","bifunctors-5.6.1-8maClQJd97MJ2wVcNtRfqA","-package-id","bimap-0.5.0-Hx2wlFtc9YLD5hjJlSwfPG","-package-id","binary-conduit-1.3.1-82tJ1uwH2enCfMFVZQnGTi","-package-id","binary-orphans-1.0.4.1-E2idqkuDfLZEzctQfrC3wO","-package-id","bitvec-1.1.5.0-AkWgGFxQPsG9A65w5Ah69t","-package-id","blaze-builder-0.4.2.3-KqOYhpYf1HiJNpffbEN3Kt","-package-id","blaze-html-0.9.1.2-Dr5La8L5ON7CNEQVAn4WLC","-package-id","blaze-markup-0.8.3.0-2vtdi1m30QR7YYEGi2s89A","-package-id","boring-0.2.1-5JtY8TZFlNj4ApXrgxIoZY","-package-id","bsb-http-chunked-0.0.0.4-IC9ZPBN6tEZ1OZx3ALxxCt","-package-id","byteorder-1.0.4-KgHnyAnKoyVGvIK2rOLowO","-package-id","bytestring-lexing-0.5.0.11-GcyNCHm993OIqJep7KOhS7","-package-id","call-stack-0.4.0-5ydnsiRoTiXIBqpTNw0DxP","-package-id","case-insensitive-1.2.1.0-GcCJoF0rEAz80ykmwVW4ch","-package-id","cassava-0.5.3.0-9yvOsSzDAmMD2aYWSZlnGz","-package-id","cereal-0.5.8.3-5oOEuXlphCGDvFB2ea5zJH","-package-id","cereal-text-0.1.0.2-6NOTPFJDXSyFKU2zDXzVmr","-package-id","cereal-vector-0.2.0.1-7Wm2YVjoznDKaq1xljEZZj","-package-id","charset-0.3.10-4cZeWp1CChW2F1desszldf","-package-id","clock-0.8.4-GWdt9eQOakoIoTGtE10Wmd","-package-id","cmdargs-0.10.22-7L0JErRtQhFG11RoysOo0I","-package-id","code-page-0.2.1-IixcuQlTQsB50QZtfgryhO","-package-id","colour-2.3.6-L6nsTBN10g6LEOaPT1DhwH","-package-id","comonad-5.0.8-LL0tV9AQnCn4If9xVN6hG0","-package-id","conduit-1.3.5-HVOlorGGlir96PVYnA93CA","-package-id","conduit-extra-1.3.6-E9LuDIL0lgyGlAB5ibaeJR","-package-id","constraints-0.14-2RQcE2jPPJM3x5H93Nebrh","-package-id","continued-fractions-0.10.0.2-3EMU94l1GHwAomlJONhoob","-package-id","contravariant-1.5.5-I125tgQh8MTA4fBNbkUir","-package-id","converge-0.1.0.1-Fs1ZPW9Szxy2NCbzGFUMNj","-package-id","cookie-0.4.6-5AmBimRtl1b5x4z9gzLDpZ","-package-id","criterion-1.6.3.0-5Lnu7NtnQIVL2uiyRNlaT2","-package-id","criterion-measurement-0.2.1.0-NlFQqKzqjR1qA9RSV05yS","-package-id","cryptohash-md5-0.11.101.0-LlbmVuBbccu9tXNi6TxuHk","-package-id","cryptohash-sha1-0.11.101.0-52VKih8lnvCJbZmLfShIh1","-package-id","cryptohash-sha256-0.11.102.1-HVJdf35wrFU48WLj8sGfXD","-package-id","crypton-0.34-Ltjs6KkrUi7G4f2ufh38d6","-package-id","crypton-connection-0.3.1-JC2brknaEey5U0s7IsDtY1","-package-id","crypton-x509-1.7.6-4wNBYXqlIZ57ZL9KYUh1VV","-package-id","crypton-x509-store-1.6.9-KOK35eib7j2LyRF58u4pv","-package-id","crypton-x509-system-1.6.7-JzsOwFWoG62Hc5rjMuMF2A","-package-id","crypton-x509-validation-1.6.12-Eg9A2ncCwayHvrCAOvFco8","-package-id","cryptonite-0.30-DGOdoWS6AAkJrH1hRJEPJF","-package-id","data-default-0.7.1.1-FdtrmQTXowRI1d2dz3rHco","-package-id","data-default-class-0.1.2.0-DWa0eFdALruJzXj3E1ovxz","-package-id","data-default-instances-containers-0.0.1-1Tp8hLWMDhAJooGCiJSFBW","-package-id","data-default-instances-dlist-0.0.1-12Qq1tWnVrgykaFcYXxdb","-package-id","data-default-instances-old-locale-0.0.1-7WOAXjACin92DHfZz0TgUY","-package-id","data-fix-0.3.2-99DVtzOqq3K3rbaNUFYnZV","-package-id","data-reify-0.6.3-9hDZkOwo0khABOKqkI9M0Q","-package-id","dec-0.0.5-mAN7EeM3yHJNahx53MzMS","-package-id","dense-linear-algebra-0.1.0.0-HcRRkJm3K5j24DhJ9TJ5lG","-package-id","digest-0.0.2.0-VKU91AxOrCAOkah68ega6","-package-id","direct-sqlite-2.3.28-6IxArIBYCqO68KZ0NLpYHq","-package-id","distributive-0.6.2.1-Hwt5zImrukd6KUzKE6M50e","-package-id","dlist-1.0-Fw5TUYtsnR3X3pZf0TOak","-package-id","easy-file-0.2.5-HQl9n3284dh5L5XuBaANaK","-package-id","either-5.0.2-GaAWGzwT4mw3J92Kk9pFrE","-package-id","entropy-0.4.1.10-2AvZFx3ITaqJDxhZPxg8mf","-package-id","erf-2.0.0.0-3j8ejB8wxc15VzirbFfRzX","-package-id","errors-2.3.0-2OtChDaYdO7LPwBI0wI8Xx","-package-id","extra-1.7.14-2dRdDSKU8g0KK8FZELat6j","-package-id","fail-4.9.0.0-BAHmj60kS5K7NVhhKpm9J5","-package-id","fast-logger-3.2.2-9iaxGeuwhu8AyX02AUcUEk","-package-id","file-embed-0.0.15.0-J0eeXTE7Jjb1Q1xZHAEFiF","-package-id","filtrable-0.1.6.0-AY8bxQUVcqNDdoIFvEjEB8","-package-id","foldl-1.4.15-G8SWTeKgfSK1wqhMEnllMa","-package-id","foldl-statistics-0.1.5.1-6a7OdjmUTLKBti5ugwWbcL","-package-id","free-5.2-Eds3sr9QClXKBhFaNOt2ha","-package-id","gamma-0.10.0.0-DaOtDzcTWkpCni2QjsPw9q","-package-id","generic-data-1.1.0.0-9nwVSovSwQlBNSyGPKCXIv","-package-id","generic-lens-2.2.2.0-HJLUldVHSjfE87wyFzZtqj","-package-id","generic-lens-core-2.2.1.0-8o1nHq83vQBLCa3Etw0CG3","-package-id","generic-random-1.5.0.1-9CBSqLOTNQh8ndPWdKYIep","-package-id","generically-0.1.1-FYC7OFpiEMrCzXM2eHlEir","-package-id","generics-sop-0.5.1.3-D4XAYPGcTYn9a7uU7aGDxQ","-package-id","ghc-9.6.4","-package-id","githash-0.1.7.0-2YiOrFvDf1JHUxzMWOO3CY","-package-id","hashable-1.4.3.0-Ctl6W5355VO4VN8AZbmQBS","-package-id","haskell-lexer-1.1.1-9lFSMJU001IAJFjnPh3Dyl","-package-id","hedis-0.15.2-5ZzcufCGubSCsJACYUsLcf","-package-id","hmatrix-0.20.2-LiG2czQQdVkJlUTT2DVZbv","-package-id","hmatrix-gsl-0.19.0.1-7exJ4sDiiYHJAnEUroTaZ2","-package-id","hmatrix-nlopt-0.2.0.0-DJke59W26XrDFC0sW4PuI7","-package-id","hmatrix-sundials-0.20.0.0-KqKIe5H17DuBgQVgCxIxwy","-package-id","hostname-1.0-ELNQIqGKmZRKzZEh3HJzii","-package-id","hourglass-0.2.12-J4zGVbaIRUFFodVsYab0JK","-package-id","hspec-2.11.7-FArtEtolMcH8ifLbh4olCK","-package-id","hspec-core-2.11.7-7nl5Y6JjUgv7s5hGGPgUNc","-package-id","hspec-discover-2.11.7-8cHIhEjdSkL2H7GD5229vV","-package-id","hspec-expectations-0.8.4-FQpbXuDGSJL9yJ8jeLwoy8","-package-id","http-api-data-0.5.1-5kXNNSlCvmMB4lGGmNdXEu","-package-id","http-client-0.7.16-8l8nZarHOyJHrekx8R2vPf","-package-id","http-client-tls-0.3.6.3-3O6s0TxPDw5IrKoUmdh2vh","-package-id","http-conduit-2.3.8.3-Jnls6dSOCq9I2r4SmP855K","-package-id","http-date-0.0.11-CQPce8QfOizB06I4QalbWb","-package-id","http-media-0.8.1.1-FWnAH26aL1Y2XpJLUamjeS","-package-id","http-types-0.12.4-KyKFj8rLDF95NhhumjHp9G","-package-id","http2-5.0.1-EJgRwBSkmdCezPaW49w4S","-package-id","hvega-0.12.0.7-JpMMe2eVR4xKfTqfm9Lyyt","-package-id","indexed-profunctors-0.1.1.1-Bftps1PC8I3Dd8NVci7qFZ","-package-id","indexed-traversable-0.1.3-Fg9XTu6yBtF3vGsnIicBgn","-package-id","indexed-traversable-instances-0.1.1.2-4hFriChJFN9AwnmcrN7iDB","-package-id","inline-c-0.9.1.10-6Yu0lzfxsbDDbL0bBqBluD","-package-id","insert-ordered-containers-0.2.5.3-IDx1RBROxtj9imphXyCK2O","-package-id","integer-conversion-0.1.0.1-5lsMS25HqYNCV92OW0pr2y","-package-id","integer-logarithms-1.0.3.1-5eh1poygNvPGBKHOuxo0w2","-package-id","integration-0.2.1-1G6d4yQFS1WIu7mdHY5cLg","-package-id","invariant-0.6.2-7Z53OwmXlMuIbtYdDgu6qn","-package-id","iproute-1.7.12-8AqkVxkmdET8jpnHo0aRYH","-package-id","js-chart-2.9.4.1-FFb0ce54QXtLDcZq3p4vfP","-package-id","json-stream-0.4.5.3-DOMpmoz8B166lp5CTnpTbW","-package-id","kan-extensions-5.2.5-23mw3jt4yB2DcrZuUyMlNj","-package-id","katip-0.8.8.0-6TvDVujg33pIdfBm1RCx2H","-package-id","keys-3.12.3-IPLidPuw1fo6TkcO5YwCRW","-package-id","kmeans-0.1.3-6tOZvn5BO6V2kSsyynP6Mm","-package-id","lens-5.2.3-JYAMYsCkSGbJSelc2xzviN","-package-id","lexer-applicative-2.1.0.2-2r6YQ2G6iou12mpHkk8e4T","-package-id","libyaml-0.1.2-559pSEGQXa098QgUVb07An","-package-id","llvm-hs-9.0.1.1-68sl9FvDsZ32JzqVSG2KxU","-package-id","llvm-hs-pure-9.0.1.1-3GltH8UpMPYKz2DRFH0IRP","-package-id","matchable-0.1.2.1-Hsq1LpTc0ROCtxgiOXHwTM","-package-id","math-functions-0.3.4.3-IJIeptC2OOsDXJw2Yb65CP","-package-id","megaparsec-9.5.0-Gwb6N57nNDzLBG5T7ijNWw","-package-id","memory-0.18.0-2xcpOrHdfEZx7Okkm9YgZ","-package-id","microlens-0.4.13.1-Ke4m5S0ig5NfuZJJcuPpw","-package-id","microlens-th-0.4.3.14-5lhNaffYzncGXINdooJA5Q","-package-id","microstache-1.0.2.3-IyzIcAXelTvGr7r4CQhiwI","-package-id","mime-types-0.1.2.0-7AIyzFVOyaLFRt5vBidQWK","-package-id","mmorph-1.2.0-CL2L86AVMMK2m1mUXOKELU","-package-id","monad-classes-0.2.2.1-4ax0efZFcClIx7gfuGqVHE","-package-id","monad-control-1.0.3.1-1liAm4UhBFKk7yfA2UY0O","-package-id","monad-loops-0.4.3-F60GSOPZI2tARnFVt6Q4D7","-package-id","mono-traversable-1.0.15.3-25gMchV5EaRGqiXyePeCgi","-package-id","mtl-compat-0.2.2-FGd4mS4fxMO6dWBau0cSm3","-package-id","mwc-random-0.15.0.2-AT6EovQE5ly1Ylew8yZiU","-package-id","nats-1.1.2-1IJUekZt6ps5IWVCKMBPeK","-package-id","network-3.1.4.0-AJEoDmw4KkzB0MEWVViGHB","-package-id","network-bsd-2.8.1.0-LCXDdLNXNTnK27X4kMD5hU","-package-id","network-byte-order-0.1.7-C8YIpF2qsXO91d4T9JkWGz","-package-id","network-control-0.0.2-68Q0HWmQ295mn7TlwwLSD","-package-id","network-info-0.2.1-GxkFgeAA01UnFjchqERNu","-package-id","network-simple-0.4.5-6m6AIhCdlVWF9iUcp0ywzF","-package-id","network-uri-2.6.4.2-6l8DJQa6cbjJCwTA5dAAo","-package-id","nlopt-haskell-0.1.3.0-7kL6ZICRTtHLW2S9WvF1Vs","-package-id","old-locale-1.0.0.7-9ClBrOuOIT6EwyhqtYJ951","-package-id","old-time-1.1.0.4-2NGSN81F5XGBsIJ7yHs5p3","-package-id","openapi3-3.2.3-FfagyVDpyx73EEnV2XKKK4","-package-id","optics-core-0.4.1.1-JSk3s0jsQniIzEMijXbclJ","-package-id","optics-extra-0.4.2.1-Bo5SvqtRmpVG32YAXnfbrb","-package-id","optics-th-0.4.1-LCWVhJaFX2l2BQhyLiiSQM","-package-id","optparse-applicative-0.18.1.0-Ecg7YCGP0vSCAYXLEDkzTj","-package-id","ordered-containers-0.2.3-AA490DMjqlgLfTf6yRaWVC","-package-id","owen-0.1.0.0-KoxWmDgd4bK8wulycVCqJD","-package-id","parallel-3.2.2.0-DxVg4bSZb4m3D30yt9UQFE","-package-id","parser-combinators-1.3.0-CLzvY25Tdrt6arBuVaQF22","-package-id","parsers-0.12.11-24xUSw1UMsFLcafGRentsR","-package-id","pem-0.2.4-FHRJ2uFrzbvEvwEl6zREm2","-package-id","pipes-4.3.16-G8duxVJQ3qH1HfI17hP8hR","-package-id","pointed-5.0.4-3klif9GgSWd8TeJyUW0oCN","-package-id","pqueue-1.5.0.0-CHPPLkvbiT1Cu83P8z9mj","-package-id","pretty-show-1.10-7QJQR28ntpqAaVVCcnieFO","-package-id","pretty-simple-4.1.2.0-5vbW3SnRgprAJl2yMkAbnL","-package-id","prettyprinter-1.7.1-Hej1zHKCrxbGcezwwa85RK","-package-id","prettyprinter-ansi-terminal-1.1.3-GeWnq98690VA7w4EexRSel","-package-id","primitive-0.8.0.0-BcoUEAnXhk925AP6fFV7Ue","-package-id","profunctors-5.6.2-7p9FluqpJFcDwA7HHZMH90","-package-id","psqueues-0.2.8.0-xnSah4CPAj9AOeBxqF9ga","-package-id","quickcheck-instances-0.3.30-6MNWuxwf9Wo7u3jy2FYAyn","-package-id","quickcheck-io-0.2.0-CeWNe3YNnijDKjlzaj9XSZ","-package-id","quickcheck-text-0.1.2.1-CQJBGlBjKlRD8fnT4am2vc","-package-id","random-1.2.1.1-HFZoHECXAgh282B9F4kODN","-package-id","random-fu-0.3.0.1-50w7IjH4ZDNLEEd3tHELwJ","-package-id","random-shuffle-0.0.4-3e3aicnflsnHOs24vct59q","-package-id","recursion-schemes-5.2.2.5-5n2RZ8FyylHC2FSgPoN1ez","-package-id","recv-0.1.0-3hINh6uDGndH8K3RkUBOdV","-package-id","reflection-2.1.7-5aIfTIHPtlD81iIpE2UVB2","-package-id","regex-1.1.0.2-E7n6srHFwTOKXkbQ2rSlkL","-package-id","regex-applicative-0.3.4-ILZolKCsQjQDvUNUeqdWOY","-package-id","regex-base-0.94.0.2-F4CJ0UC4BXoIqNDucZCXty","-package-id","regex-compat-0.95.2.1-C3ROROSybfnJqWLmZZ6k87","-package-id","regex-pcre-builtin-0.95.2.3.8.44-1vYMkq1MHXY2aNMJRMJQP8","-package-id","regex-posix-0.96.0.1-4A3qC1NTpFzB51CFKA97bs","-package-id","regex-tdfa-1.3.2.2-5SUCkdo12yhK96s88l01tr","-package-id","resource-pool-0.4.0.0-Da66QlpmoPsABaVE3tAt9D","-package-id","resourcet-1.3.0-8L75Ae2EE5ZCiTvZzFMUCL","-package-id","retry-0.9.3.1-8YImD4QbWNwHJ2n18EmhaH","-package-id","rvar-0.3.0.2-DBSH9xOU49rDzxgFtmX4Ym","-package-id","safe-0.3.21-75cmjghR87ODCXLWpE0pIZ","-package-id","safe-exceptions-0.1.7.4-3nSgnRDkUlC4nQNQLXkPtQ","-package-id","scanner-0.3.1-Idtl9byttljK4qKHMXnJgj","-package-id","scientific-0.3.7.0-6X5dSNcGT5u8fKL6TMkq0h","-package-id","selda-0.5.2.1-ClbuMgpSKFiK4TQpOf6nne","-package-id","selda-sqlite-0.1.7.2-Bx0Dl1we4hIjkRyejw30m","-package-id","semialign-1.3-4ufaaBqE7mnAZqkbCCceOY","-package-id","semigroupoids-6.0.0.1-Dr5vYfbebkB7TTSCEPIlXd","-package-id","semigroups-0.20-2b6SpEF9Mq04Tyyiixe3Ft","-package-id","servant-0.20.1-K9SotPSp5RVHGb9IhhVP2K","-package-id","servant-server-0.20-9kpEgMTMuijLyLZjZgwayR","-package-id","show-combinators-0.2.0.0-DkibGQorrlB2o0rGkZZhi7","-package-id","simple-sendfile-0.2.32-96qlBD90IOxEBEbQ7BAKZc","-package-id","simwork-compatibility-streaming-utils-0.0.0.0-abcdefgh","-package-id","simwork-core-definitions-annotation-0.0.0.0-abcdefgh","-package-id","simwork-core-definitions-equation-0.0.0.0-abcdefgh","-package-id","simwork-core-definitions-event-0.0.0.0-abcdefgh","-package-id","simwork-core-definitions-model-0.0.0.0-abcdefgh","-package-id","simwork-core-definitions-model-result-0.0.0.0-abcdefgh","-package-id","simwork-core-definitions-pc-0.0.0.0-abcdefgh","-package-id","simwork-core-definitions-simwork-exceptions-0.0.0.0-abcdefgh","-package-id","simwork-core-definitions-species-0.0.0.0-abcdefgh","-package-id","simwork-core-definitions-topsort-0.0.0.0-abcdefgh","-package-id","simwork-core-delayed-lib-0.0.0.0-abcdefgh","-package-id","simwork-core-formulaic-internal-0.0.0.0-abcdefgh","-package-id","simwork-core-formulaic-lib-0.0.0.0-abcdefgh","-package-id","simwork-core-formulaic-type-0.0.0.0-abcdefgh","-package-id","simwork-core-inlined-lib-0.0.0.0-abcdefgh","-package-id","simwork-core-unit-checking-lib-0.0.0.0-abcdefgh","-package-id","simwork-core-webservice-core-webservice-base-0.0.0.0-abcdefgh","-package-id","simwork-core-webservice-errors-0.0.0.0-abcdefgh","-package-id","simwork-core-webservice-manifest-0.0.0.0-abcdefgh","-package-id","simwork-core-webservice-openapi-utils-0.0.0.0-abcdefgh","-package-id","simwork-core-webservice-project-item-0.0.0.0-abcdefgh","-package-id","simwork-core-webservice-protocol-design-0.0.0.0-abcdefgh","-package-id","simwork-core-webservice-scoring-design-0.0.0.0-abcdefgh","-package-id","simwork-core-webservice-sql-0.0.0.0-abcdefgh","-package-id","simwork-formulae-formulae-0.0.0.0-abcdefgh","-package-id","simwork-id-id-0.0.0.0-abcdefgh","-package-id","simwork-id-tagged-id-0.0.0.0-abcdefgh","-package-id","simwork-jinquantities-jinquantities-0.0.0.0-abcdefgh","-package-id","simwork-lib-v0-v0-0.0.0.0-abcdefgh","-package-id","simwork-math-anova-0.0.0.0-abcdefgh","-package-id","simwork-math-basic-0.0.0.0-abcdefgh","-package-id","simwork-math-constants-0.0.0.0-abcdefgh","-package-id","simwork-math-data-cats-0.0.0.0-abcdefgh","-package-id","simwork-math-diff-0.0.0.0-abcdefgh","-package-id","simwork-math-factorial-design-0.0.0.0-abcdefgh","-package-id","simwork-math-factorial-design-cbits-haskell-0.0.0.0-abcdefgh","-package-id","simwork-math-interp-0.0.0.0-abcdefgh","-package-id","simwork-math-foontainers-0.0.0.0-abcdefgh","-package-id","simwork-math-jinkostats-0.0.0.0-abcdefgh","-package-id","simwork-math-latex-0.0.0.0-abcdefgh","-package-id","simwork-math-law-estimation-0.0.0.0-abcdefgh","-package-id","simwork-math-ndouble-0.0.0.0-abcdefgh","-package-id","simwork-math-novarandom-0.0.0.0-abcdefgh","-package-id","simwork-math-ode-0.0.0.0-abcdefgh","-package-id","simwork-math-optimization-0.0.0.0-abcdefgh","-package-id","simwork-math-root-0.0.0.0-abcdefgh","-package-id","simwork-math-smooth-0.0.0.0-abcdefgh","-package-id","simwork-mathjs-mathjs-0.0.0.0-abcdefgh","-package-id","simwork-model-manager-model-0.0.0.0-abcdefgh","-package-id","simwork-pipeline-config-0.0.0.0-abcdefgh","-package-id","simwork-pipeline-core-0.0.0.0-abcdefgh","-package-id","simwork-pipeline-exception-0.0.0.0-abcdefgh","-package-id","simwork-pipeline-paths-0.0.0.0-abcdefgh","-package-id","simwork-pipeline-pipeline-0.0.0.0-abcdefgh","-package-id","simwork-pipeline-tasks-0.0.0.0-abcdefgh","-package-id","simwork-pipeline-test-helpers-0.0.0.0-abcdefgh","-package-id","simwork-pipeline-tools-docrecords-0.0.0.0-abcdefgh","-package-id","simwork-pipeline-tools-jsonstream-0.0.0.0-abcdefgh","-package-id","simwork-pipeline-tools-katip-formats-0.0.0.0-abcdefgh","-package-id","simwork-pipeline-tools-porcupine-core-0.0.0.0-abcdefgh","-package-id","simwork-pipeline-tools-porcupine-jobs-0.0.0.0-abcdefgh","-package-id","simwork-pipeline-tools-porcupine-servant-0.0.0.0-abcdefgh","-package-id","simwork-pipeline-tools-porcupine-sql-base-0.0.0.0-abcdefgh","-package-id","simwork-pipeline-tools-porcupine-sqlite-0.0.0.0-abcdefgh","-package-id","simwork-pipeline-tools-reader-soup-0.0.0.0-abcdefgh","-package-id","simwork-pipeline-utils-0.0.0.0-abcdefgh","-package-id","simwork-result-manager-lib-0.0.0.0-abcdefgh","-package-id","simwork-sbml-bindings-0.0.0.0-abcdefgh","-package-id","simwork-sbml-core-0.0.0.0-abcdefgh","-package-id","simwork-sbml-sbml-0.0.0.0-abcdefgh","-package-id","simwork-scenario-conversion-0.0.0.0-abcdefgh","-package-id","simwork-scenario-manager-0.0.0.0-abcdefgh","-package-id","simwork-scenario-pipeline-0.0.0.0-abcdefgh","-package-id","simwork-scenario-scenario-0.0.0.0-abcdefgh","-package-id","simwork-scoring-lib-0.0.0.0-abcdefgh","-package-id","simwork-scoring-manager-0.0.0.0-abcdefgh","-package-id","simwork-scoring-measure-0.0.0.0-abcdefgh","-package-id","simwork-scoring-score-0.0.0.0-abcdefgh","-package-id","simwork-selection-selection-0.0.0.0-abcdefgh","-package-id","simwork-simulation-manager-status-0.0.0.0-abcdefgh","-package-id","simwork-symbolic-symbolic-0.0.0.0-abcdefgh","-package-id","simwork-trial-lib-0.0.0.0-abcdefgh","-package-id","simwork-trial-model-0.0.0.0-abcdefgh","-package-id","simwork-units-units-0.0.0.0-abcdefgh","-package-id","simwork-utils-control-0.0.0.0-abcdefgh","-package-id","simwork-utils-csv-0.0.0.0-abcdefgh","-package-id","simwork-utils-data-0.0.0.0-abcdefgh","-package-id","simwork-utils-error-doc-0.0.0.0-abcdefgh","-package-id","simwork-utils-hashing-0.0.0.0-abcdefgh","-package-id","simwork-utils-hashmap-0.0.0.0-abcdefgh","-package-id","simwork-utils-json-0.0.0.0-abcdefgh","-package-id","simwork-utils-list-0.0.0.0-abcdefgh","-package-id","simwork-utils-novapretty-0.0.0.0-abcdefgh","-package-id","simwork-utils-orphans-0.0.0.0-abcdefgh","-package-id","simwork-utils-results-0.0.0.0-abcdefgh","-package-id","simwork-utils-rng-0.0.0.0-abcdefgh","-package-id","simwork-utils-sanity-0.0.0.0-abcdefgh","-package-id","simwork-utils-utils-0.0.0.0-abcdefgh","-package-id","simwork-version-version-0.0.0.0-abcdefgh","-package-id","simwork-vpop-analysis-0.0.0.0-abcdefgh","-package-id","simwork-vpop-design-0.0.0.0-abcdefgh","-package-id","simwork-vpop-override-model-0.0.0.0-abcdefgh","-package-id","simwork-vpop-paths-0.0.0.0-abcdefgh","-package-id","simwork-vpop-raw-0.0.0.0-abcdefgh","-package-id","simwork-vpop-remotable-0.0.0.0-abcdefgh","-package-id","simwork-vpop-scalar-array-0.0.0.0-abcdefgh","-package-id","simwork-vpop-scalar-result-0.0.0.0-abcdefgh","-package-id","singleton-bool-0.1.7-HlkNlLSI2x688GvudscXoz","-package-id","socks-0.6.1-HTGeiS0CuQB5xnG2dKnQY6","-package-id","some-1.0.6-9KkUW9N9iYX3gQ9ifquABF","-package-id","sop-core-0.5.0.2-EYRwKKEWmNxKHMQ0MeNGxN","-package-id","split-0.2.5-1ZSy0xYACSg5uwyBvCYVUs","-package-id","splitmix-0.1.0.5-Bfmh89T202E2QU5m0YBt2H","-package-id","srcloc-0.6.0.1-JjjsMlNEXKtGlXlc32l2gZ","-package-id","statistics-0.16.2.1-6eyzLbJFbUlJdD20d2bGO3","-package-id","storable-complex-0.2.3.0-HkwA7XPbanN8Q1mYZMNutL","-package-id","storable-record-0.0.7-6sdC2M0cAUg5NMsLcUJHYw","-package-id","storable-tuple-0.1-5g6AsjoIR8FKNBiiUt3W1K","-package-id","streaming-0.2.4.0-51TDKZo3g0TC9cBQG6i1xz","-package-id","streaming-bytestring-0.3.2-9mLN7cGx7fWIjU0eyUnX85","-package-id","streaming-commons-0.2.2.6-LkZBwP8prTKF1Ies7T7HsH","-package-id","streaming-utils-0.2.3.0-68X1J5ZrXF19l7WLYbpSAD","-package-id","strict-0.5-BrLZi9HPs3MGb2lC5Y8MDY","-package-id","string-conv-0.2.0-B9HtxL9Dyp6KvVidqt8Lqc","-package-id","string-conversions-0.4.0.1-5tKOYjhmIyGFbWLIEzcjjt","-package-id","syb-0.7.2.4-DHdyWMYMgCuFivy4DkG6c","-package-id","tagged-0.8.8-NwKdK9CJixGyu4p6ivyqd","-package-id","tardis-0.4.4.0-eh7xExhTcQ3TI9p3JntSH","-package-id","tasty-1.4.3-1Nt2RYTkXJlEeZtDh0AJmw","-package-id","tasty-golden-2.3.5-FVZPfRHbeZLGepLT0nm9QH","-package-id","tasty-hunit-0.10.1-BgejLvDNlU7K07ubsSm8kf","-package-id","temporary-1.3-3vseRfxvLrSCUX0K0S5Kxw","-package-id","text-short-0.1.5-BohssFTCYkz2pEBmfnkQb8","-package-id","tf-random-0.5-BAOtJoQW1hkc50JPnHIsT","-package-id","th-abstraction-0.5.0.0-4W5LehiF65R9RGG2u3u9tk","-package-id","th-compat-0.1.4-Gfio53PRTgL3kHLgxibJcs","-package-id","these-1.2-9I7INPNBHhOCpNFq5Uiv4Q","-package-id","time-compat-1.9.6.1-20qEEqKRTcZ4os4wPxaReC","-package-id","time-locale-compat-0.1.1.5-BxC3jG4YqHjHlwdygAgUai","-package-id","time-manager-0.0.1-89XomeXoHtM9L3UiSApNjQ","-package-id","tls-1.8.0-2HiUdQxPedJID8g4pQ1aIA","-package-id","transformers-base-0.4.6-EaFDlO9TcWFCxFOri3PWyd","-package-id","transformers-compat-0.7.2-E6o5MUlxoOH3uwowdNHcwI","-package-id","trie-simple-0.4.2-80QQ9SY6oZ850UHmdAk8Fh","-package-id","tuples-homogenous-h98-0.1.1.0-B1fPef7FuqRJLDvpNP7aN4","-package-id","type-equality-1-HChXi6uMt0A7ToWSodD5vd","-package-id","typed-process-0.2.11.1-1doJzpRPwmTGAxjGqZzdEt","-package-id","uglymemo-0.1.0.1-E3afEKo6HRT55gyWszUJTt","-package-id","unicode-data-0.4.0.1-Daz8gceMYT9AEu05Iw5WMT","-package-id","unicode-transforms-0.4.0.1-Lqlz0AN5mZT2gr2eNrRR4L","-package-id","unix-compat-0.7.1-ATKCdjm8LSjCLkrGNM2fiq","-package-id","unix-time-0.4.11-7jIUZ78clhQK0REclivIHE","-package-id","unliftio-0.2.25.0-AVDzlJk5zvABFSDDubGBn5","-package-id","unliftio-core-0.2.1.0-jCMAjQfJQAFcRO0d4vNHT","-package-id","unordered-containers-0.2.20-7B6s4pEM0cd3m6yCex75Sv","-package-id","url-2.1.3-FnH9A2Ly6gPK7nmwur7oSj","-package-id","utf8-string-1.0.2-FimbYq1wIkv3u9JmaBWYAg","-package-id","utility-ht-0.0.17-Cup4exKnyA1H4E8SzAKHKG","-package-id","uuid-1.3.15-FrkMzjDDHs92ipRr7V3OdF","-package-id","uuid-types-1.0.5.1-62rpHF29Z6L5rrt1fAsQ02","-package-id","vault-0.3.1.5-2ofa9ypAKZE71esY1VQobb","-package-id","vector-0.13.1.0-5ffdaJBGONU8uzeLZ1XNT5","-package-id","vector-algorithms-0.9.0.1-Gv6pkIJ2manC7r5rzeKyQh","-package-id","vector-binary-instances-0.2.5.2-GdkycB1qBLSAEDJviJFpyH","-package-id","vector-instances-3.4.2-6zoeM6TPXFCLY6KpCBv6Ah","-package-id","vector-stream-0.1.0.1-3kAAjYjKcymLd80Bqq9Xxz","-package-id","vector-th-unbox-0.2.2-5QaFKkoInIGKS1umO1K4fK","-package-id","versions-6.0.4-5PhohLiSkPzBs5wuyvI5ig","-package-id","vinyl-0.14.3-8UV2yE2I5aFLMPR0ZYA607","-package-id","void-0.7.3-G56k4lUEr85OMOwlEFyBI","-package-id","wai-3.2.4-Hubh67v5chXCdxi3ubjtQJ","-package-id","wai-app-static-3.1.8-I7hRHPe2mGK6OcEyjJwJTV","-package-id","wai-extra-3.1.14-4t1lIqKThBuDob3AFLYqwz","-package-id","wai-logger-2.4.0-L1qZagPifsw8Mq62pcu44T","-package-id","warp-3.3.31-AmUlJFO5lja27WIk3rbfeh","-package-id","witherable-0.4.2-9W09GsF3d4nAfsRVGAhaim","-package-id","word8-0.1.3-IScL71vvani4T8HfwOqi1H","-package-id","x509-1.7.7-LM6PBK3obw84UFB8CLQx87","-package-id","x509-store-1.6.9-L8AH2EHf195BjLHsd1lsya","-package-id","yaml-0.11.11.2-8gRL4rpGVkE2YtE1Md4Zmq","-package-id","zip-stream-0.2.2.0-8lEUGKyWboGEqytBKsiImf","-package-id","zlib-0.6.3.0-8LxZXTqHQaCLReMuto4U00","-ibar-simulation/golden-tests","-ibar-simulation/golden-tests/","-ibar-simulation/src/","-ibar-simulation/test/hspec-test","-ibar-simulation/test/tasty-test","/nix/store/aa59ncm4p7vv3b5p74k3w0vdxgp5qy3r-libsbml-5.20.2/lib/libsbml.so","/nix/store/cy708jqf377n5xl733q6dc1mksq7cr3l-perf.simwork.math.factorial-design_cbits/perf.simwork.math.factorial-design_cbits.so","/nix/store/n8kkv2ragi7ancwhb63k12x4qhgxyd3p-perf.simwork.math.machine-delta-lib/perf.simwork.math.machine-delta-lib.so","-DJINKO_DYNLIBS=\"/nix/store/7q6lhfvsfsnwd9lllfh5wy1wf1mnvdqv-perf.simwork.symbolic.intpow/perf.simwork.symbolic.intpow.so\"","-DTORNADO_SENSITIVITY_SQL=\"/nix/store/xrpri8pihbyzi9czs0kb0fjf81grsvsx-tornadoSensitivitySql/tornado_sensitivity.sql\"","-Wall","-Wcompat","-Wincomplete-record-updates","-Wincomplete-uni-patterns","-Wredundant-constraints","-Wnoncanonical-monad-instances","-Wunused-packages","-Wno-ambiguous-fields","-XHaskell2010","-XBangPatterns","-XConstraintKinds","-XDataKinds","-XDefaultSignatures","-XDeriveDataTypeable","-XDeriveFoldable","-XDeriveFunctor","-XDeriveGeneric","-XDeriveTraversable","-XDerivingStrategies","-XEmptyCase","-XEmptyDataDecls","-XExistentialQuantification","-XFlexibleContexts","-XFlexibleInstances","-XKindSignatures","-XDuplicateRecordFields","-XOverloadedRecordDot","-XLambdaCase","-XMultiParamTypeClasses","-XMultiWayIf","-XNamedFieldPuns","-XNumDecimals","-XPartialTypeSignatures","-XRankNTypes","-XRecordWildCards","-XScopedTypeVariables","-XStandaloneDeriving","-XTupleSections","-XTypeApplications","-XTypeSynonymInstances","-XViewPatterns","-XOverloadedStrings","-XDataKinds","-XGeneralizedNewtypeDeriving","-XOverloadedLabels","-XTypeOperators","-XDerivingVia","-XQuasiQuotes","-stubdir=/tmp","-DGHCI",... (Long list of haskell modules .hs)], componentRoot = "/home/guillaume/jinko/bar", componentDependencies = ["./bar-simulation/build.nix"]},"/nix/store/xhwcdgs6gzgh5zhrpwqy8iqh3y6aam6v-ghc-9.6.4/lib/ghc-9.6.4/lib")
2024-02-04T12:10:01.923148Z | Info | Interface files cache directory: /home/guillaume/.cache/ghcide/main-5f50f5bd074bca15292d92df75278f6ab5b479ff-5f50f5bd074bca15292d92df75278f6ab5b479ff
2024-02-04T12:10:01.928343Z | Info | Making new HscEnv. In-place unit ids: [ main-5f50f5bd074bca15292d92df75278f6ab5b479ff ]

our HIE_BIOS file is like:

-package-db
/nix/store/jfsgbpj35qawldf8l4hsdf3n02hkk3aa-ghc-9.6.4-with-packages/lib/ghc-9.6.4/lib/package.conf.d

LONG LIST OF -package-id nameOfPackage

LONG LIST OF -ilines

/nix/store/aa59ncm4p7vv3b5p74k3w0vdxgp5qy3r-libsbml-5.20.2/lib/libsbml.so
/nix/store/cy708jqf377n5xl733q6dc1mksq7cr3l-perf.simwork.math.factorial-design_cbits/perf.simwork.math.factorial-design_cbits.so
/nix/store/n8kkv2ragi7ancwhb63k12x4qhgxyd3p-perf.simwork.math.machine-delta-lib/perf.simwork.math.machine-delta-lib.so
-DJINKO_DYNLIBS="/nix/store/7q6lhfvsfsnwd9lllfh5wy1wf1mnvdqv-perf.simwork.symbolic.intpow/perf.simwork.symbolic.intpow.so"
-DTORNADO_SENSITIVITY_SQL="/nix/store/xrpri8pihbyzi9czs0kb0fjf81grsvsx-tornadoSensitivitySql/tornado_sensitivity.sql"
-Wall
-Wcompat
-Wincomplete-record-updates
-Wincomplete-uni-patterns
-Wredundant-constraints
-Wnoncanonical-monad-instances
-Wunused-packages
-Wno-ambiguous-fields
-XHaskell2010
-XBangPatterns
-XConstraintKinds
-XDataKinds
-XDefaultSignatures
-XDeriveDataTypeable
-XDeriveFoldable
-XDeriveFunctor
-XDeriveGeneric
-XDeriveTraversable
-XDerivingStrategies
-XEmptyCase
-XEmptyDataDecls
-XExistentialQuantification
-XFlexibleContexts
-XFlexibleInstances
-XKindSignatures
-XDuplicateRecordFields
-XOverloadedRecordDot
-XLambdaCase
-XMultiParamTypeClasses
-XMultiWayIf
-XNamedFieldPuns
-XNumDecimals
-XPartialTypeSignatures
-XRankNTypes
-XRecordWildCards
-XScopedTypeVariables
-XStandaloneDeriving
-XTupleSections
-XTypeApplications
-XTypeSynonymInstances
-XViewPatterns
-XOverloadedStrings
-XDataKinds
-XGeneralizedNewtypeDeriving
-XOverloadedLabels
-XTypeOperators
-XDerivingVia
-XQuasiQuotes
-stubdir=/tmp
-DGHCI

LONG LIST OF HASKELL FILES

Note that if I remove the -package-db argument (and all the -package-id which are not in a "default" package db), HLS starts immediatly and perfectly works (and fails because most of my import cannot be resolved). I tried to remove the -package-id only and keep the -package-db, and then HLS is stuck again. So definitiely, there may be something in my package db which blocks HLS.

Feb 04 '24 12:02 guibou

I think @wz1000 is our best bet for guessing what's going on.

Feb 04 '24 14:02 michaelpj

I would add more traces in Session.hs to figure out exactly where it gets stuck.

Feb 04 '24 14:02 wz1000

@wz1000 I will do that, thank you for the filename, it helps me a lot.

Feb 04 '24 15:02 guibou

I've added a bunch of debug log statment in Session.hs and related files and I'm starting to discover something:

In IDE/Session.hs:

      let targetEnv = (if isBad ci then multi_errs else [], Just henv)
          targetDepends = componentDependencyInfo ci
          res = ( targetEnv, targetDepends)
      liftIO $ logWith recorder Logger.Debug $ LogDLLLoadError "Foo"
      logWith recorder Debug $ LogNewComponentCache res
      liftIO $ logWith recorder Logger.Debug $ LogDLLLoadError "Bar"
      evaluate $ liftRnf rwhnf $ componentTargets ci

Foo is lagged, but not Bar. If I comment the logWith statment, then Bar happen and my session continue (and is stuck on something else later, but we'll first investigate here).

Actually, if I even replace the logWith by a simple liftIO (print res), it is stuck and my console output ((, which is the beginning of the tuple contained in res. So looks like there is a "lazy" lock hidden in the computation of this tuple (either infinite loop, or something worse hidden behind an unsafePerformIO)

Feb 05 '24 04:02 guibou

The tuple in res is (([(NormalizedFilePath, ShowDiagnostic, Diagnostic)], Maybe HscEnvEq), DependencyInfo) and snd as well as snd . fst component are fine. So the problem is in [(NormalizedFilePath, ShowDiagnostic, Diagnostic)]

Feb 05 '24 04:02 guibou

-      let targetEnv = (if isBad ci then multi_errs else [], Just henv)
+     let targetEnv = ([], Just henv)

Completely unlocks my HLS, and I even have diagnostics.

I'll apply this as a patch at work in order to progress forward. I can investigate a bit more if you give me some guidance.

edit isBad ci terminates, the problem is in multi_errs.

Feb 05 '24 05:02 guibou

Okay, so that does seem like it shouldn't get stuck. Do we have a test case for that? It looks like it's specifically for things that violate the closure property.

Obviously we should also give an error, but it looks like we get stuck in the process of trying to work out the error :joy:

Feb 05 '24 08:02 michaelpj

Maybe checkHomeUnitsClosed is not in fact always terminating in the presence of weird package dependencies?

Feb 05 '24 08:02 michaelpj

Yeah this might be a GHC bug. Can you print out (hsc_unit_env hscEnv') and (hsc_all_home_unit_ids hscEnv') please?

Feb 05 '24 08:02 wz1000

Alternatively, looking at the comment on https://gitlab.haskell.org/ghc/ghc/-/blob/master/compiler/GHC/Driver/Make.hs#L1756, perhaps it just has bad complexity and so since you have a massive package db it's just doing some quadratic or otherwise huge amount of work?

Feb 05 '24 08:02 michaelpj

Also printing out the closure_errs might be enlightening. It seems likely that the home units are not closed for your project. This means that there are "home units"(units that you've asked HLS to load) A and B, such that A depends on B and there is a unit C such that A depends on C and C depends on B and C is not a home unit.

Feb 05 '24 08:02 wz1000

@wz1000, sorry, I'm taking too much time to answer.

Apparently I'm not the only one with this issue, so my "weird" setup with nix may not be responsible.

Yeah this might be a GHC bug. Can you print out (hsc_unit_env hscEnv') and (hsc_all_home_unit_ids hscEnv') please?

hsc_unit_env is an UnitEnv which does not have Show. I've tried to add one, but stopped after adding Show instance for dozens of types. Do you know another way to display it? (I may try to start in ghci and use :force or :print, but I'm afraid the setup will be painful).

for hsc_all_home_unit_ids, the result is surprising, it is [main,main-457d6b4053974d5f4ce4d2060404b2ae241df152], so apparently I have two home unit, and one does not have an hash it its name. Is that something expected?

Recall that I'm not using "home unit" yet and my build is only composed of -package-id to hackage, or file references. However it is possible that I have multiples Main module, do you think it can be a problem. I'll check.

Also printing out the closure_errs might be enlightening.

An infinite loop is hidden in closure_errs too. I'll try to dig a bit and see if I can extract a bit of the informations.

Feb 15 '24 14:02 guibou

Note, a few line above, the call closure_errs = checkHomeUnitsClosed (hsc_unit_env hscEnv') (hsc_all_home_unit_ids hscEnv') pkg_deps, I can observe that pkg_deps is finite. However it does not contains any reference to main, only main-457d6b4053974d5f4ce4d2060404b2ae241df152.

Feb 15 '24 14:02 guibou

hsc_unit_env is an UnitEnv which does not have Show

you want to use pprUnitEnvGraph from GHC.

Feb 15 '24 14:02 wz1000

Some news. Removing the main entry in pkg_deps also fix the lock.

Feb 15 '24 14:02 guibou

Thanks @carbolymer, thanks to your reproducer I was able to diagnose the issue and put up a fix in #4109

Mar 01 '24 11:03 wz1000

Thank you @carbolymer, and thank you @wz1000!

Mar 01 '24 16:03 guibou

haskell-language-server haskell-language-server copied to clipboard

HLS stuck (init then no progress) since multi home support

Your environment

A few notes about our build system.

HLS bisect

Steps to reproduce

Expected behaviour

Actual behaviour

Debug information

haskell-language-server
haskell-language-server copied to clipboard