Solve the Index Symbols Reliability Problem

[picnoir]

We are currently using a wrong approach to index a file's used symbols (see indexModule in the ProcessingSteps.hs file). Basically, we are looking at the modules provided by the package dependencies and filtering them accordingly to the imported modules. This is obviously a rough approximation. This results in lots of false positive/missed match and makes this software hardly usable. TL;DR our approximation does not perform really well.

So far, we did not find a way to get the GHC's Name for every symbol coming out of the parser. Basically, we are getting the parser output token stream and we did not find a way to get the resolved Name of the parsed symbols without traversing the whole GHC AST contained in the ModSummary.

I guess the problem here is mostly me being unfamiliar with GHC's API, the solution will hopefully be obvious to somebody with more experience.

I see two potential starting points to address this issue:

1. We keep the current approach of using the parser output and we use HARE's approach and create a mapping between the tokens location and their names. <= @NinjaTrappeur favorite approach
   • HARE's solution: https://github.com/alanz/HaRe/blob/ghc-8.6/src/Language/Haskell/Refact/Utils/MonadFunctions.hs#L541
   • @alanz is working towards including this function in the GHC API itself: https://phabricator.haskell.org/D5330
2. We bite the bullet and try to retrieve every symbol usage by traversing the GHC starting by the desugared ModuleSummary.

[chkl]

I would like to work on this. From my cursory look at the linked discussion at GHC it seems that alanz's inquiry came to a standstill. So I guess the best approach would be to duplicate the functionality of HaRe.

Anything else I should know?

P.S. @NinjaTrappeur : fatal: unable to access 'https://git.alternativebit.fr/NinjaTrappeur/cabal-helper.git/': The requested URL returned error: 502 prevents me from building the project currently.

P.P.S: I saw that ex-hack will be a project at ZuriHac, what do you think about creating a channel in their slack workspace?

[alanz]

FYI there is a GSOC currently running to make use of the .hie files carrying the symbol/type information, which will probably be useful for this too.

Ping @wz1000

[wz1000]

Yes, .hie files contain all the resolved names in a file, and can be generated by ghc 8.8+. See https://gitlab.haskell.org/ghc/ghc/wikis/hie-files and https://github.com/wz1000/HieDb for an example of how to consume the files.

[chkl]

I will need a little time to look at it and get ghc8.8-alpha1 installed, but those hie files look very promising. Thank you for the heads-up.

[picnoir]

@alanz @wz1000 <3

Looks indeed very promising.

---

fatal: unable to access 'https://git.alternativebit.fr/NinjaTrappeur/cabal-helper.git/': The requested URL returned error: 502 prevents me from building the project currently.

My git server was down; my bad.

---

@chkl As I was stating in another issue; cabal-helper was a dirty hack in order to get at least something I could show. I think the way to go here is either though a source plugin either though those .hie index.

I don't have a lot of time ahead ATM to play with those indices. I'll probably do that tomorrow and post the conclusion of the experiment.

P.P.S: I saw that ex-hack will be a project at ZuriHac, what do you think about creating a channel in their slack workspace?

I'm not really using slack regularly, that said, I created a #ex-hack channel on freenode if you need to chat. If you also come to Zurihac, we definitely could work on this symbol resolution/ghc-interface refactoring.