Incomplete/missing system API methods

[nathanvy]

I'm trying to manipulate some binary data represented as a hexadecimal string and I noticed that the language server incorrectly reports that some API methods do not exist. As a specific example (I have not had time to exhaustively go through all the API and see which are reported incorrectly) I would like to highlight Convert.ToHexString

To reproduce this problem:

1. Start a new .NET 6 console application:

$ dotnet new console --framework net6.0

2. Edit the generated Program.cs in emacs. Type just enough to get the autocomplete and syntax checkers to kick in:

using System;

byte[] bytes = new byte[] { 1, 2, 3, 4, 5 };
Convert.ToHexString(bytes);

3. Note the syntax checker informing you that ToHexString is invalid:

And that's all there is to it. I am using emacs with an up-to-date omnisharp (verified by running M-x lsp-update-server) on macOS 12.5 (Monterey).

[333fred]

We'd need to see the log from the omnisharp server. But at a guess, something is messed up with your configuration.

[nathanvy]

Thanks for your reply.

Here is the log from the language server:

Found the following clients for /Users/nathan/Code/csharp/testcase/Program.cs: (server-id omnisharp, priority -1)
The following clients were selected based on priority: (server-id omnisharp, priority -1)
OmniSharp.Services.DotNetCliService: Checking the 'DOTNET_ROOT' environment variable to find a .NET SDK | 
OmniSharp.Services.DotNetCliService: Checking the 'DOTNET_ROOT' environment variable to find a .NET SDK | 
OmniSharp.Services.DotNetCliService: Using the 'dotnet' on the PATH. | 
OmniSharp.Services.DotNetCliService: Using the 'dotnet' on the PATH. | 
OmniSharp.Services.DotNetCliService: DotNetPath set to dotnet | 
OmniSharp.Services.DotNetCliService: DotNetPath set to dotnet | 
OmniSharp.MSBuild.Discovery.MSBuildLocator: Located 1 MSBuild instance(s)
            1: Mono 16.10.1 - "/Library/Frameworks/Mono.framework/Versions/6.12.0/lib/mono/msbuild/Current/bin" | 
OmniSharp.MSBuild.Discovery.MSBuildLocator: Located 1 MSBuild instance(s)
            1: Mono 16.10.1 - "/Library/Frameworks/Mono.framework/Versions/6.12.0/lib/mono/msbuild/Current/bin" | 
OmniSharp.CompositionHostBuilder: It looks like you have Mono installed which contains a MSBuild lower than 17.0.0 which is the minimum supported by the configured .NET Core Sdk.
 Try updating Mono to the latest stable or preview version to enable better .NET Core Sdk support. | 
OmniSharp.CompositionHostBuilder: It looks like you have Mono installed which contains a MSBuild lower than 17.0.0 which is the minimum supported by the configured .NET Core Sdk.
 Try updating Mono to the latest stable or preview version to enable better .NET Core Sdk support. | 
OmniSharp.MSBuild.Discovery.MSBuildLocator: Registered MSBuild instance: Mono 16.10.1 - "/Library/Frameworks/Mono.framework/Versions/6.12.0/lib/mono/msbuild/Current/bin" | 
OmniSharp.MSBuild.Discovery.MSBuildLocator: Registered MSBuild instance: Mono 16.10.1 - "/Library/Frameworks/Mono.framework/Versions/6.12.0/lib/mono/msbuild/Current/bin" | 
OmniSharp.WorkspaceInitializer: Invoking Workspace Options Provider: OmniSharp.Roslyn.CSharp.Services.CSharpFormattingWorkspaceOptionsProvider, Order: 0 | 
OmniSharp.WorkspaceInitializer: Invoking Workspace Options Provider: OmniSharp.Roslyn.CSharp.Services.CSharpFormattingWorkspaceOptionsProvider, Order: 0 | 
OmniSharp.Cake.CakeProjectSystem: Detecting Cake files in '/Users/nathan/Code/csharp/testcase'. | 
OmniSharp.Cake.CakeProjectSystem: Detecting Cake files in '/Users/nathan/Code/csharp/testcase'. | 
OmniSharp.Cake.CakeProjectSystem: Did not find any Cake files | 
OmniSharp.Cake.CakeProjectSystem: Did not find any Cake files | 
OmniSharp.MSBuild.ProjectSystem: No solution files found in '/Users/nathan/Code/csharp/testcase' | 
OmniSharp.MSBuild.ProjectSystem: No solution files found in '/Users/nathan/Code/csharp/testcase' | 
OmniSharp.MSBuild.ProjectManager: Queue project update for '/Users/nathan/Code/csharp/testcase/testcase.csproj' | 
OmniSharp.MSBuild.ProjectManager: Queue project update for '/Users/nathan/Code/csharp/testcase/testcase.csproj' | 
OmniSharp.Script.ScriptProjectSystem: Detecting CSX files in '/Users/nathan/Code/csharp/testcase'. | 
OmniSharp.Script.ScriptProjectSystem: Detecting CSX files in '/Users/nathan/Code/csharp/testcase'. | 
OmniSharp.Script.ScriptProjectSystem: Did not find any CSX files | 
OmniSharp.Script.ScriptProjectSystem: Did not find any CSX files | 
OmniSharp.WorkspaceInitializer: Configuration finished. | 
OmniSharp.WorkspaceInitializer: Configuration finished. | 
OmniSharp.MSBuild.ProjectManager: Loading project: /Users/nathan/Code/csharp/testcase/testcase.csproj | 
OmniSharp.MSBuild.ProjectManager: Loading project: /Users/nathan/Code/csharp/testcase/testcase.csproj | 
OmniSharp.MSBuild.ProjectLoader: The reference assemblies for ".NETFramework,Version=v6.0" were not found. You might be using an older .NET SDK to target .NET 5.0 or higher. Update Visual Studio and/or your .NET SDK. | 
OmniSharp.MSBuild.ProjectLoader: The reference assemblies for ".NETFramework,Version=v6.0" were not found. You might be using an older .NET SDK to target .NET 5.0 or higher. Update Visual Studio and/or your .NET SDK. | 
OmniSharp.MSBuild.ProjectManager: Failed to load project file '/Users/nathan/Code/csharp/testcase/testcase.csproj'. | 
OmniSharp.MSBuild.ProjectManager: Failed to load project file '/Users/nathan/Code/csharp/testcase/testcase.csproj'. | 
OmniSharp.Extensions.LanguageServer.Server.LspServerOutputFilter: Tried to send request or notification before initialization was completed and will be sent later OmniSharp.Extensions.JsonRpc.Client.OutgoingNotification | @Request='OmniSharp.Extensions.JsonRpc.Client.OutgoingNotification'
OmniSharp.MSBuild.ProjectManager: Attempted to update project that is not loaded: /Users/nathan/Code/csharp/testcase/testcase.csproj | 
OmniSharp.MSBuild.ProjectManager: Attempted to update project that is not loaded: /Users/nathan/Code/csharp/testcase/testcase.csproj | 
Creating watchers for following 1 folders:
  /Users/nathan/Code/csharp/testcase
OmniSharp.LanguageServerProtocol.LanguageServerHost: Omnisharp server running using Lsp at location '/Users/nathan/Code/csharp/testcase' on host -1. | 
OmniSharp.LanguageServerProtocol.LanguageServerHost: Omnisharp server running using Lsp at location '/Users/nathan/Code/csharp/testcase' on host -1. | 

Of particular note, I guess, are the lines about an unsupported version of MSbuild. I'm using the latest stable version of mono, which I guess ships version 16.10, so that's pretty frustrating.

The line regarding reference assemblies not being found is very strange since I'm assuredly running an up-to-date SDK:

$ dotnet --list-sdks
6.0.400 [/usr/local/share/dotnet/sdk]
$ dotnet --version
6.0.400
$ msbuild --version
Microsoft (R) Build Engine version 16.10.1 for Mono
Copyright (C) Microsoft Corporation. All rights reserved.

16.10.1.31401

Unfortunately I'm at an impasse here because even the Preview build ships with a (slightly newer) version 16.10:

$ msbuild --version
Microsoft (R) Build Engine version 16.10.1 for Mono
Copyright (C) Microsoft Corporation. All rights reserved.

16.10.1.40401

Any ideas on what I can try next? Microsoft ships their own msbuild for macOS; can I point omnisharp at that executable instead of mono? Honestly I'd rather remove mono entirely. Having two msbuild versions on the same system seems like unneeded complexity.

[nathanvy]

Waiiiiit a minute:

Quoting from the project README (emphasis mine):

OmniSharp is built with the .NET Core SDK on Windows and Mono on OSX/Linux. It targets the net472 target framework. For platforms other than Windows, OmniSharp ships with an embedded Mono which is based on version 6.12.0, includes MSBuild 16.8.0 and is provisioned during the build script. If Mono is globally installed on the system, OmniSharp will prefer it over the embedded version, however version >=6.4.0 is required (the lowest version with at least MSBuild 16.3.0).

I had taken this for granted, but I see now there was a merged PR that removed Mono, however there remains an implicit runtime dependency on mono in this script.

Because omnisharp is not built as a self-contained executable I can't just dotnet run it. I'm far from an expert in the intricacies of building dotnet apps but I'll play around with cakebuild and see what I can come up with.

If anyone has any input I'd welcome it

[333fred]

From the log, it doesn't look like O# was able to find your dotnet installation, which will be why it can't find the reference assemblies. How did you install dotnet?

[nathanvy]

From here: https://dotnet.microsoft.com/en-us/download/dotnet/6.0

When I encountered the issue with things being highlighted as missing that was the first thing I checked, and a reinstall did not rectify the issue.

[333fred]

I'm not particularly familiar with OSX troubleshooting, maybe @filipw has an idea on how fix it?

[nathanvy]

I suspect, since macOS is a Unix, that it would be similar to troubleshooting on Linux.

What would you recommend to a Linux user?

[333fred]

I don't think it's similar at all, as osx applications generally install in a very different way than Linux applications. I wouldn't expect to see this issue on Linux, as dotnet would be in /usr/bin.

[nathanvy]

Well, dotnet is under /usr/share/* so it's not so different IMHO.

I still think it's a mismatch between mono and dotnet. The project builds just fine, despite omnisharp not being able to see the reference assemblies and I can still use Convert.ToHexString() even though omnisharp thinks it doesn't exist.

In other words, dotnet is working fine on my machine, but omnisharp isn't, maybe because omnisharp requires mono to run and mono's having trouble picking up the PATH or something.

[333fred]

Omnisharp does not require mono to run. What it should be doing is finding the location of the dotnet msbuild (not the mono msbuild) and using the reference assemblies from that installation. However, from your log it's not seeing it at all. This is the part that I do not know how to troubleshoot on a Mac.

[nathanvy]

Perhaps I should open another issue?

I'm aware of that commit as I linked to it earlier, but I'm afraid you're mistaken. If I remove mono,

sudo rm -rf /Library/Frameworks/Mono.framework
sudo pkgutil --forget com.xamarin.mono-MDK.pkg
sudo rm /etc/paths.d/mono-commands

... then omnisharp will not start, citing the following error:

/Users/nathan/.emacs.d/.cache/lsp/omnisharp-roslyn/latest/omnisharp-roslyn/run: line 19: mono: command not found

Process omnisharp stderr finished

[333fred]

The emacs startup scripts you're using not using the modern version of OmniSharp does not mean that OmniSharp requires mono to run. It just means that the emacs scripts you're using to launch it are old. Indeed, that is literally the mono-packaging folder.

[nathanvy]

I don't understand. The script you're referring to is part of omnisharp, not part of my emacs configuration.

I can walk you through it in greater detail later today.

[333fred]

You're using the old mono launch scripts to launch OmniSharp. Those are, of course, going to be using mono. OmniSharp can (and really, should) be launched using the modern .NET 6 version of the application, though I don't believe we've created generalized launch scripts for that like we have with mono. I would highly suggest moving to the .NET 6 version if you can, as I suspect it will fix your issues (as well as generally being much more performant.

[nathanvy]

Hmm, well, in emacs with lsp-mode, omnisharp is automatically installed, so it's just being invoked however lsp-mode invokes it by default. I've read through the omnisharp README and the configuration settings page on the wiki and saw no mention of different versions of the application.

In the Releases directory there are several osx releases. I chose omnisharp-osx-x64-net6.0.zip, as it seemed like the most applicable one. I changed the emacs lsp-mode url for omnisharp to point to that release and unfortunately it's still choking on mono not existing:

/Users/nathan/.emacs.d/.cache/lsp/omnisharp-roslyn/latest/omnisharp-roslyn/run: line 3: exec: mono: not found

Process omnisharp stderr finished

I seem to be doing something wrong. How would I go about moving to the .NET 6 version?

[333fred]

How would I go about moving to the .NET 6 version?

Don't invoke mono (as that script is doing). You need to call dotnet /path/to/omnisharp.dll.

[nathanvy]

(as that script is doing)

I want to be clear that that script is something that ships with omnisharp and that it's not something I'm doing intentionally.

There must be a bug with the way emacs is installing omnisharp because in the server install directory there is no omnisharp.dll whatsoever.

[333fred]

I want to be clear that that script is something that ships with omnisharp and that it's not something I'm doing intentionally.

I'm aware of this. The script is old, but if we changed it to do something it would break someone's workflow somewhere.

There must be a bug with the way emacs is installing omnisharp because in the server install directory there is no omnisharp.dll whatsoever.

Assuming you're using emacs-lsp, it looks like it should be downloading the right thing: https://github.com/emacs-lsp/lsp-mode/blob/master/clients/lsp-csharp.el#L81-L84. But I'm also not sure how it's actually figuring out what to launch, nor how you're getting that mono script, as the package that code downloads does not have that script in it.

[nathanvy]

I use straight for packaging in emacs so my hunch is now that it's pulling the latest release which is from 2021, so I suspect the function on my disk won't match what's current on github. Will check tonight after the kids are in bed.

Thanks for your help/patience thus far.

[nathanvy]

Yeah, that seems to be it. Here's the function on disk:

(defcustom lsp-csharp-omnisharp-roslyn-download-url
  (concat "https://github.com/omnisharp/omnisharp-roslyn/releases/latest/download/"
          (cond ((eq system-type 'windows-nt)
                 ; On Windows we're trying to avoid a crash starting 64bit .NET PE binaries in
                 ; Emacs by using x86 version of omnisharp-roslyn on older (<= 26.4) versions
                 ; of Emacs. See https://lists.nongnu.org/archive/html/bug-gnu-emacs/2017-06/msg00893.html"
                 (if (and (string-match "^x86_64-.*" system-configuration)
                          (version<= "26.4" emacs-version))
                     "omnisharp-win-x64.zip"
                   "omnisharp-win-x86.zip"))

                ((eq system-type 'darwin)
                 "omnisharp-osx.zip")

                ((and (eq system-type 'gnu/linux)
                      (or (eq (string-match "^x86_64" system-configuration) 0)
                          (eq (string-match "^i[3-6]86" system-configuration) 0)))
                 "omnisharp-linux-x64.zip")

                (t "omnisharp-mono.zip")))
  "Automatic download url for omnisharp-roslyn."
  :group 'lsp-csharp-omnisharp
  :type 'string)

So I guess that's pulling the wrong zip, and I guess that's what I get for assuming lsp was up to date. Thanks for bearing with me through this mess.