Detect Windows MAX_PATH limit and throw descriptive error message

[mthierba]

• https://docs.microsoft.com/en-us/windows/win32/fileio/maximum-file-path-limitation?tabs=cmd

Extracting a PBIX into a deeply nested root folder is likely to cause problems since the pbixproj folder structure adds several extra levels. This is a hard OS limit. But let's at least return a very descriptive error message to the user so they can take action (move the project folder somewhere higher up).

Should also support Win 1607 long path support (via custom app manifest) - although that requires a registry setting on the user machine, so out of our control.

[mthierba]

See https://github.com/dotnet/winforms/issues/2059 for further reference on embedding an application manifest using the .Net Core SDK.

[mthierba]

Rick has it all covered nicely here: https://weblog.west-wind.com/posts/2022/Jan/03/Integrating-Long-Path-Names-in-Windows-NET-Applications

Extended Paths are an alternative. Consider an opt-in setting?

[JamesDBartlett3]

@mthierba I'm getting errors when I try to extract a PBIX file in its git repo folder on my hard drive, even though I've already set HKLM:\SYSTEM\CurrentControlSet\Control\FileSystem\LongPathsEnabled = 1 in my registry. Any ideas what I need to change on the pbi-tools side to make it work?

[mthierba]

Long path support in Windows is tricky. The standard solution requires two changes, one on the user side (which you've made), but also another in the application (pbi-tools in this case). I haven't implemented the latter yet, which is why this issue is still open. Thanks for the reminder!

[agusmba]

I got bit by this today. Some user reports are using text boxes with long text in the title attribute, making for quite long object names. While trying to use pbi-tools on them, it chokes when extracting those objects:

[...]
Version [1.25] extracted to: C:\Users\MYUSER\Downloads\MYREPORTNAME\Version.txt
Connections extracted to: C:\Users\MYUSER\Downloads\MYREPORTNAME\Connections.json
An unhandled exception occurred.
System.IO.DirectoryNotFoundException: No se puede encontrar una parte de la ruta de acceso 'C:\Users\MYUSER\Downloads\MYREPORTNAME\Report\sections\003_Help\visualContainers\23000_Very long text very long text very long text very long text very long text very long text very long text very long text very long text very long text very long text very long text very long text very long text very long text very long text'.
   en System.IO.__Error.WinIOError(Int32 errorCode, String maybeFullPath)
   en System.IO.Directory.InternalCreateDirectory(String fullPath, String path, Object dirSecurityObj, Boolean checkHost)
   en System.IO.Directory.InternalCreateDirectoryHelper(String path, Boolean checkHost)
   en PbiTools.FileSystem.ProjectFolder.WriteFile[T](String path, Func`2 factory, Action`1 callback) en X:\pbi-tools\BRANCHES\1.0.0-rc.2\src\PBI-Tools\FileSystem\ProjectFolder.cs:línea 141
   en PbiTools.FileSystem.ProjectFolder.WriteText(String path, Action`1 onTextWriterAvailable) en X:\pbi-tools\BRANCHES\1.0.0-rc.2\src\PBI-Tools\FileSystem\ProjectFolder.cs:línea 137
   en PbiTools.Utils.JsonExtensions.ExtractAndParseTokenAs[T](JObject parent, String property, IProjectFolder folder) en X:\pbi-tools\BRANCHES\1.0.0-rc.2\src\PBI-Tools\Utils\JsonExtensions.cs:línea 71
   en PbiTools.Serialization.ReportSerializer.Serialize(JObject content) en X:\pbi-tools\BRANCHES\1.0.0-rc.2\src\PBI-Tools\Serialization\ReportSerializer.cs:línea 97
   en PbiTools.Model.PbixModel.ToFolder(String path, PbixProjectSettings settings) en X:\pbi-tools\BRANCHES\1.0.0-rc.2\src\PBI-Tools\Model\PbixModel.cs:línea 276
   en PbiTools.Cli.CmdLineActions.Extract(String pbixPath, Nullable`1 pid, Nullable`1 pbiPort, String extractFolder, ExtractActionCompatibilityMode mode, ModelSerializationMode modelSerialization, MashupSerializationMode mashupSerialization, Boolean watch) en X:\pbi-tools\BRANCHES\1.0.0-rc.2\src\PBI-Tools\Cli\Extract.cs:línea 105
--- Fin del seguimiento de la pila de la ubicación anterior donde se produjo la excepción ---
   en System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   en PowerArgs.ArgAction.Invoke()
   en PbiTools.Program.Main(String[] args) en X:\pbi-tools\BRANCHES\1.0.0-rc.2\src\PBI-Tools\Program.cs:línea 94

Not easy to work around, since the report has docens of text boxes like this...

[mthierba]

@agusmba I wonder whether in this case a folder name trim rule might be the right solution? You'd configure a max length for page/visual folder names, and if the actual name would be longer it gets trimmed. Obviously, there is a slight danger here that you'd end up with ambiguous folder names, but you as the project owner would need to handle that risk. Better that way than not being able to extract subsequent PBIX versions. Ultimately, the folder naming scheme is meant to provide consistency across multiple versions - for git diff to work best, the same set of files (corresponding to the same visual) should always be extracted to the same location within the repo.

What do you think? This would be a rule I could implement quite quickly.

[agusmba]

@agusmba I wonder whether in this case a folder name trim rule might be the right solution? You'd configure a max length for page/visual folder names, and if the actual name would be longer it gets trimmed. Obviously, there is a slight danger here that you'd end up with ambiguous folder names, but you as the project owner would need to handle that risk. Better that way than not being able to extract subsequent PBIX versions. Ultimately, the folder naming scheme is meant to provide consistency across multiple versions - for git diff to work best, the same set of files (corresponding to the same visual) should always be extracted to the same location within the repo.

If the trim option is not too hard to implement, it would provide a workable solution for the case I experienced. I think titles in text boxes or shapes shouldn't be too long and trimming when they are seems like a sensible compromise.

[JamesDBartlett3]

Maybe I'm missing something here, but wouldn't trimming be considered a destructive edit of the data?

Also, the reason I've run into this problem is because the actual path on my system where my local repos are stored exceeds the MAX_PATH limit, so trimming the strings that pbi-tools outputs wouldn't help in my case.

[mthierba]

@JamesDBartlett3 Trimming the extracted folder names, NOT the actual visual titles. The extract/compile flow would still be lossless - the folder names are only for humans, but irrelevant for the PBIX compilation. In that regard it'd be an easy fix. However, not if your repo folder is already at the max length - how does that even work with other Windows applications?

[JamesDBartlett3]

I already have the registry hack enabled on my system to allow longer file paths. I do this on every system I setup.

[mthierba]

This issue with long paths and the extent of it is also covered in someone's outburst about the situation of Microsoft and open source: https://gist.github.com/slimsag/c01bb6508e3dfa744bf3bdafa0cfe07f#i-wish-that-was-the-end-of-the-windows-developer-pain-story

[fredrikheden]

I had to do the following two things to get it working on my machine:

1. Enable long paths in the registry.
2. Recompile the project with an app.manifest file with the following contents:

  <application xmlns="urn:schemas-microsoft-com:asm.v3">
    <windowsSettings xmlns:ws2="http://schemas.microsoft.com/SMI/2016/WindowsSettings">
      <ws2:longPathAware>true</ws2:longPathAware>
    </windowsSettings>
  </application>

[JamesDBartlett3]

@mthierba, I have found a potential workaround for this problem: I created a symlink to my local repository in the root of my C:\ drive like so: New-Item -ItemType SymbolicLink -Path "C:\repo\" -Target "C:\excessively\long\path\to\my\local\git\repository\"

When I open a pwsh terminal, if I navigate to my local repo via the usual path and run (Get-Location).Path.Length, the result is 54. But now, when I navigate to my local repo via the symlink, and run that same command, the result is 13, so my symlink shortened the length of the path to my local repo by 41 characters.

Now, when I run pbi-tools extract '.\My Workspace\My Report\My Report.pbix -extractFolder .\My Workspace\My Report\src, it works without a hitch, so I must have only been over the MAX_PATH limit by 41 characters or less.

For me, this is an acceptable temporary workaround, so I'll be able to start using pbi-tools in production at my organization.

[Seebaer1986]

I had to do the following two things to get it working on my machine:

1. Enable long paths in the registry.
2. Recompile the project with an app.manifest file with the following contents:

  <application xmlns="urn:schemas-microsoft-com:asm.v3">
    <windowsSettings xmlns:ws2="http://schemas.microsoft.com/SMI/2016/WindowsSettings">
      <ws2:longPathAware>true</ws2:longPathAware>
    </windowsSettings>
  </application>

Hey @fredrikheden,

is it possible for you to share your altered codebase in a GitHub Fork? I tried it myself, but I am unable to reproduce what you achieved. This would be a really big help :-)

@mthierba: maybe you could the changed app manifest in your original codebase as well, so users only need to change the registry entry?

Thanks to you guys!

[didierterrien]

Hello @mthierba, Don't you think file short names can be used?

[DllImport("kernel32.dll", CharSet = CharSet.Auto)] public static extern int GetShortPathName([MarshalAs(UnmanagedType.LPTStr)] string path, [MarshalAs(UnmanagedType.LPTStr)] StringBuilder shortPath, int shortPathLength);

[mthierba]

Windows builds of 1.0.0-rc.2 ship with the application manifest embedded: https://learn.microsoft.com/en-us/windows/win32/fileio/maximum-file-path-limitation?tabs=registry Note that users still need to opt into the long path behavior by setting a registry key (requires elevated permissions).

It is also advised to enable long paths for git:

git config --global core.longpaths true