Slow performance for --include with single file

[chriselion]

Hi, I'm just experimenting with dotnet-format for the first time. I noticed that running it on a single file has more overhead than I would expect.

# Run on 223 files 
$ git ls-files | grep .cs$  > all_cs.txt
$ dotnet format --folder --include @all_cs.txt
  Formatting code files in workspace '/xyz/ml-agents'.
  ...
  Format complete in 2297ms.

# Run on 1 file
$ dotnet format --folder --include com.unity.ml-agents/Tests/Editor/MultinomialTest.cs
  Formatting code files in workspace '/xyz/ml-agents'.
  Format complete in 1325ms.

Is that expected? It's not horrible, but it would be mildly annoying to run incrementally on git commits.

Platform/version information: MacOS 10.14.6

$ dotnet --version
3.1.401
$ dotnet-format --version
4.1.131201+82a5527cd93d66bffc885c0bb901499f5981db35

[RikkiGibson]

(sorry, I misunderstood the description in my original comment.) It does feel like there's an excessive amount of fixed-size overhead here.

[chriselion]

Thanks, sorry if I was being unclear - the fixed-size overhead was what I was complaining about.

Any profiling information that I can provide to help diagnose? I haven't tried diag verbosity yet; would there be anything useful in there?

[RikkiGibson]

A trace from a sampling profiler of some kind could be helpful. My guess though is that a lot of this overhead is in just spinning up the runtime, loading the compiler assemblies and jitting methods--assuming that we do not use any kind of multiproc or client/server model like we do when you run the compiler in order to build. Do you know @JoeRobich @jmarolf?

[jmarolf]

would be helpful to know if there is any better perf in the latest (unrelesed) version.

dotnet tool install -g dotnet-format --version 5.0.135301 --add-source https://dotnet.myget.org/F/format/api/v3/index.json

in general formatting whitespace should be very fast but semantic formatting (detecting if you are using var in your codebase and whether that is allowed) can take more time. @chriselion could you provide a sample of your editorconfig file?

[chriselion]

FWIW here's the results with -v diag on 4.1.131201.

 $ dotnet format --folder --include com.unity.ml-agents/Tests/Editor/MultinomialTest.cs -v diag
  The dotnet CLI version is '3.1.401'.
  Using MSBuild.exe located in '/usr/local/share/dotnet/sdk/3.1.401/'.
  Formatting code files in workspace '/Users/chris.elion/code/ml-agents'.
  Loading workspace.
  Complete in 966ms.
  Determining formattable files.
  Complete in 233ms.
  Running formatters.
  Complete in 194ms.
  Formatted 0 of 1 files.
  Format complete in 1396ms.

and with the new version

$ dotnet format --folder --include com.unity.ml-agents/Tests/Editor/MultinomialTest.cs -v diag
  The dotnet CLI version is '3.1.401'.
  Using MSBuild.exe located in '/usr/local/share/dotnet/sdk/3.1.401/'.
  Formatting code files in workspace '/Users/chris.elion/code/ml-agents'.
  Loading workspace.
  Complete in 1156ms.
  Determining formattable files.
  Complete in 223ms.
  Running formatters.
  Complete in 183ms.
  Formatted 0 of 1 files.
  Format complete in 1565ms.

The editor config can be found here and the file I'm formatting is here. I don't think there's anything special about that particular file; I just picked it at random.

[jmarolf]

Yep this should be fast. The second we spend discovering files is unnecessary. that at least gets us to 500ms

[chriselion]

Sounds good. Do you need any other info from me?

[jmarolf]

No, I'll likely use your project as the basis for some perf validation.

[chriselion]

Great, thanks.

Just for a little more context, I'm playing around with integrating with https://pre-commit.com/. That passes the list of files to check as command-line arguments (usually just the files being committed, unless you tell it to run on all files).

[jmarolf]

yeah. I'll need to look here but I think a reasonable place to start at is that we should be able to get single file formats under 500ms

[seriouz]

It is still slow as hell. 8 seconds for one file and this is without -s and -a.
go fmt runs in under a second for me.

[asiriak]

Any updates on that? It takes more than 30s to do dotnet format solution.sln --include file.cs for me.

~15s for

All projects are up-to-date for restore.
  Project Api is using configuration from ...
Complete in 14412ms.

~15s for

Running formatters.
  Running Code Style analysis.
  Determining diagnostics...
  Running 1 analyzers on ...
  Complete in 15566ms.

And the worst thing is that I have to check formatting for 6 such solutions in the same repo :(

[erdalsivri]

I have a PR that should help with single file includes a bit: https://github.com/dotnet/format/pull/1485

[codec-abc]

Is this issue actively working on? I am under the impression that dotnet format is somewhat slow which make make it annoying to run locally before committing.

[aradalvand]

Same here. This is the single biggest problem dotnet format suffers from. It's simply unacceptable that it takes multiple seconds to format a single file. Formatters built into Visual Studio or the OmniSharp extension for VS Code typically take under 500 milliseconds to format a single small-to-medium-sized file.

Similar tools in other ecosystems such as go fmt or prettier are much, much more performant as well. This issue needs to be given higher priority, I'd argue.

@jmarolf @JoeRobich Sorry to tag you guys, but just wanted to get some updates on this. Have you guys managed to identify the root cause of this yet?

[zdarovka]

Any updates on this? Is there any workaround?

[aradalvand]

@jmarolf @JoeRobich I would like to urge you guys once again to give this issue a bit more attention.

This is what happens when you try to format ~5 C# files using dotnet format:

34 seconds! And remember that this is typical for dotnet format actually, it's not a one time thing. This is just unacceptable. There must be something horribly wrong with the implementation of this tool for it to yield results like this. It's almost unusable.

Please respond. Thank you.

[jensmunkhansen3shape]

I would have liked to use dotnet format for a pre-commit hook, but it is horribly slow. I am used to work with the GNU Toolchain, make, gcc, gtags etc. Microsoft tools are all significantly slower and sometimes slower by an order of magnitude, but

dotnet format --verify-nochanges SingleFile.cs

is like three orders of magnitude slower than using Artistic Style or Clang for formatting. Come on Microsoft, it is 2023

[zdarovka]

From my personal experience within the team, it is basically unusable (and I am not talking about thousand of files, just a precommit hook for staged files) on Windows machines, but the performance is acceptable on the M1/M2 chips from Apple.

[seriouz]

From my personal experience within the team, it is basically unusable (and I am not talking about thousand of files, just a precommit hook for staged files) on Windows machines, but the performance is acceptable on the M1/M2 chips from Apple.

What about Linux? How is the performance there comparing it with Windows and M1/M2

[aradalvand]

@seriouz I use Ubuntu and the performance is horrendous.

[seriouz]

I've checked a little bit. Seems that the function await MSBuildWorkspaceLoader.LoadAsync (CodeFormatter.cs) is way too slow and costs much of the execution time.

[seriouz]

@RikkiGibson I think the formatter should have a version which does not requires that msbuild loads and scans the whole solution/project. Especially the case when a single file is going to be formatted.

[RikkiGibson]

Using the --folder option and formatting only whitespace should be much faster than formatting based on a project or sln file.

I'm certainly not happy with the level of performance we have here either. I know it is unsatisfying to hear that after years since this issue was filed. It is difficult to carve out time for this in addition to my ordinary responsibilities.

We do care and it does help to have people let us know that this is impacting them. I apologize that this is the state dotnet-format is in currently.

[tomekpaszek]

Thanks for your reply @RikkiGibson! It's good to hear that you guys care and thanks for providing the tools in the first place!

[lucax88x]

I just tried today the combination of husky, lint-staged and dotnet format to just format the staged files before committing and in my sample projects is pretty slow even in an M1 max processor.

Around 10 seconds for 1 single file using include.

[jakoblover]

Same on my end. Consistently 8 seconds to format a single file.

[gdoron]

@RikkiGibson How does formatting to MSFT's standard happen internally inside MSFT? Once you have a medium-large project formatting with dotnet format is VERY slow, I can't even imagine how it would be working on MSFT scale projects.

So my questions are:

1. How do you format your files to maintain a shared coding style standard?
2. How come solving this horrific performance is something you need to do in addition to your daily tasks? How is it not painful and wasteful enough for internal MSFT employees and resources to have it fixed already?

[RikkiGibson]

The Roslyn repo has a CI job which runs a build with analyzers, including IDE0055, at error severity. This prevents people from checking in code with bad formatting.

[RobSwDev]

My repo has ~ 1600 cs files in 30-40 projects. To format a single file (using --include) takes 94 seconds. To format them all takes 115 seconds.

By comparison, doing a complete restore & build (after nuking bin & obj folders from powershell) takes just 21 seconds.

From what I can tell, it's taking at least 80 seconds to generate the *.GeneratedMSBuildEditorConfig.editorconfig This initial cost before doing any formatting seems excessive.

Using dotnet format version 8.0.511601+d237e172b324021b97effa244af44d63d1a8bb7e