Windows-Containers icon indicating copy to clipboard operation
Windows-Containers copied to clipboard

Build of an MSYS2 image fails

Open raffapen opened this issue 7 months ago • 13 comments

Describe the bug

Building an image that contains an MSYS2 installation (using the Dockerfile specified below) fails with the following message:

re-exec error: exit status 1: output: hcsshim::ImportLayer failed in Win32: The system cannot find the path specified. (0x3)

To Reproduce

With the following Dockerfile:

FROM mcr.microsoft.com/windows/servercore:ltsc2025

SHELL ["powershell", "-Command"]

WORKDIR /tmp
RUN irm -outfile msys2-sfx.exe https://github.com/msys2/msys2-installer/releases/download/nightly-x86_64/msys2-base-x86_64-latest.sfx.exe

WORKDIR /
RUN c:\\tmp\\msys2-sfx.exe

ENV HOME=/home/ContainerAdministrator
ENV MSYS=winsymlinks:native
ENV TZ=America/New_York

RUN C:\\msys64\\usr\\bin\\bash.exe -l -c /usr/bin/true

ENTRYPOINT ["C:\\msys64\\usr\\bin\\bash.exe", "-l"]

Execute: docker build -t msys2-nightly:2025 .

Expected behavior

The image should be created successfully.

Configuration:

  • Edition: Windows 11 Pro 24H2 26100.2605
  • Base Image being used: mcr.microsoft.com/windows/servercore:ltsc2025
  • Container engine: Docker
  • Docker Desktop version: v4.41.2
  • Container Engine version: Docker version 28.1.1, build 4eba377

Additional context

When modifying the Dockerfile to:

FROM mcr.microsoft.com/windows/servercore:ltsc2025

SHELL ["powershell", "-Command"]

WORKDIR /tmp
RUN irm -outfile msys2-sfx.exe https://github.com/msys2/msys2-installer/releases/download/nightly-x86_64/msys2-base-x86_64-latest.sfx.exe

WORKDIR /
RUN c:\\tmp\\msys2-sfx.exe

ENV HOME=/home/ContainerAdministrator
ENV MSYS=winsymlinks:native
ENV TZ=America/New_York

CMD ["powershell"]

the build completes successfully. Then, the following operation yield the same result as the original build:

  • Creating the container via docker run -it msys2-nightly:2025
  • Running C:\\msys64\\usr\\bin\\bash.exe -l -c /usr/bin/true inside the container
  • Exiting the container and inspecting the container ID via docker ps -a
  • Trying to commit: docker commit <SHA> msys2-nightly-1:2025

raffapen avatar Jun 03 '25 08:06 raffapen

Thank you for creating an Issue. Please note that GitHub is not an official channel for Microsoft support requests. To create an official support request, please open a ticket here. Microsoft and the GitHub Community strive to provide a best effort in answering questions and supporting Issues on GitHub.

github-actions[bot] avatar Jun 03 '25 08:06 github-actions[bot]

Fails the same way with:

Client:
 Version:           28.2.2
 API version:       1.50
 Go version:        go1.24.3
 Git commit:        e6534b4
 Built:             Fri May 30 12:07:16 2025
 OS/Arch:           windows/amd64
 Context:           default

Server: Docker Engine - Community
 Engine:
  Version:          28.2.2
  API version:      1.50 (minimum version 1.24)
  Go version:       go1.24.3
  Git commit:       45873be
  Built:            Fri May 30 11:51:15 2025
  OS/Arch:          windows/amd64
  Experimental:     false

(via https://github.com/slonopotamus/stevedore)

raffapen avatar Jun 05 '25 11:06 raffapen

Added to our backlog to repro. Will update soon.

ntrappe-msft avatar Jun 10 '25 21:06 ntrappe-msft

We're still working on repro'ing this.

ntrappe-msft avatar Jun 24 '25 22:06 ntrappe-msft

@ntrappe-msft Thank you! Please advise if there's anything I can do to help with the reproduction.

raffapen avatar Jun 25 '25 12:06 raffapen

Hi @raffapen, I did not get a repro on your issue but I'm curious to know which line your Dockerfile is failing at? Also, can you confirm that msys2 is being correctly installed in the host directory that the Dockerfile expects it to be at when it reaches the RUN commands, for example this line (C:\msys64\usr\bin\bash.exe)

sabinamiezah-msft avatar Jul 02 '25 17:07 sabinamiezah-msft

@sabinamiezah-msft It goes like this: The last RUN command (RUN C:\\msys64\\usr\\bin\\bash.exe -l -c /usr/bin/true) effectively performs the final setup phase of the msys2 installation after it was extracted to disk. This creates some device files (which are only visible from within a msys2 bash shell) but not from powershell/cmd shell (or File Explorer). I suspect this to be the core issue. The failure of the Docker build process occurs when it tries to commit the result of the last RUN operation. Similarly, if we remove the bash invocation from the Dockerfile (as I suggested in the Additional context section of the bug repot), run the container, exit it and try to commit manually, the same problem occurs. Note that if we indeed complete the msys2 installation from a running container, it is completely functional from within the container.

Thanks!

PS From within the container, with msys2 installed:

$ ls -l /dev
total 0
crw-rw-rw- 1 user None 13, 254 Dec  1  2006 clipboard
crw------- 1 user None  5, 255 Jul  5 20:38 conin
crw------- 1 user None  5, 254 Jul  5 20:38 conout
crw------- 1 user None  3,   0 Jul  5 20:38 cons0
crw------- 1 user None  3,   1 Jul  5 20:38 cons1
crw------- 1 user None  3,   2 Jul  5 20:38 cons2
crw------- 1 user None  5,   1 Jul  5 20:38 console
dr-xr-xr-x 1 user None       0 Jul  5 20:38 disk
crw-rw-rw- 1 user None 14,   3 Jul  5 20:38 dsp
lrwxrwxrwx 1 user None       0 Jul  5 20:38 fd -> /proc/self/fd
crw-rw-rw- 1 user None  1,   7 Jul  5 20:38 full
crw-rw-rw- 1 user None 14,   0 Jul  5 20:38 mixer
drwxr-xr-x 1 user None       0 May  6  2024 mqueue
crw-rw-rw- 1 user None  1,   3 Jul  5 20:38 null
crw-rw-rw- 1 user None  5,   2 Jul  5 20:38 ptmx
crw-rw-rw- 1 user None  1,   8 Jul  5 20:38 random
brw-rw-rw- 2 user None 11,   0 Jul  5 20:38 scd0
brw-rw-rw- 1 user None  8,   0 Jul  5 20:38 sda
brw-rw-rw- 1 user None  8,   1 Jul  5 20:38 sda1
brw-rw-rw- 1 user None  8,   2 Jul  5 20:38 sda2
brw-rw-rw- 1 user None  8,   3 Jul  5 20:38 sda3
brw-rw-rw- 1 user None  8,   4 Jul  5 20:38 sda4
drwxr-xr-x 1 user None       0 May  6  2024 shm
brw-rw-rw- 2 user None 11,   0 Jul  5 20:38 sr0
lrwxrwxrwx 1 user None       0 Jul  5 20:38 stderr -> /proc/self/fd/2
lrwxrwxrwx 1 user None       0 Jul  5 20:38 stdin -> /proc/self/fd/0
lrwxrwxrwx 1 user None       0 Jul  5 20:38 stdout -> /proc/self/fd/1
crw------- 1 user None  5,   0 Jul  5 20:38 tty
crw-rw-rw- 1 user None  1,   9 Jul  5 20:38 urandom
crw-rw-rw- 1 user None 13, 255 Jul  5 20:38 windows
crw-rw-rw- 1 user None  1,   5 Jul  5 20:38 zero

raffapen avatar Jul 05 '25 17:07 raffapen

Thank you for sharing these additional details, @raffapen.

To help us better understand what’s happening during the Docker build, could you please try capturing a performance trace using Windows Performance Recorder (WPR)? This will allow us to analyze CPU, file, and disk activity during the build process.

From an Administrator Command Prompt, please run the following:

wpr -start cpu -start fileio -start diskio
docker build -t msys2-nightly:2025 .
wpr -stop repro.etl

Once the trace is captured, please share the repro.etl file so we can review the performance data and identify any bottlenecks or anomalies.

sabinamiezah-msft avatar Jul 10 '25 22:07 sabinamiezah-msft

This issue has been open for 30 days with no updates. no assignees, please provide an update or close this issue.

Hi there @raffapen, I'm just touching base with you regarding this issue to see if the earlier steps about collecting a performance trace were straight forward? I truly hope that we can get some more insight on your issue if it is still persisting.

My Best,

Sabina

smiezah-msft avatar Aug 25 '25 18:08 smiezah-msft

This issue has been open for 30 days with no updates. no assignees, please provide an update or close this issue.

This issue has been open for 30 days with no updates. no assignees, please provide an update or close this issue.

Thanks, for your patience, @raffapen

Solution: Change ENV MSYS=winsymlinks:native to ENV MSYS=winsymlinks:copy in your Dockerfile.

Root Cause: The winsymlinks:native setting creates Windows reparse points (symbolic links) that Docker's layered filesystem cannot properly handle during the build process.

Alternative: Use winsymlinks:copy which tells MSYS2 to copy files instead of creating symbolic links, avoiding the Docker compatibility issue while maintaining full functionality.

Working Dockerfile

FROM mcr.microsoft.com/windows/servercore:ltsc2025

SHELL ["powershell", "-Command"]

WORKDIR /tmp
RUN irm -outfile msys2-sfx.exe https://github.com/msys2/msys2-installer/releases/download/nightly-x86_64/msys2-base-x86_64-latest.sfx.exe

WORKDIR /
RUN c:\\tmp\\msys2-sfx.exe

ENV HOME=/home/ContainerAdministrator
ENV MSYS=winsymlinks:copy  # ← Changed from 'native' to 'copy'
ENV TZ=America/New_York

RUN C:\\msys64\\usr\\bin\\bash.exe -l -c /usr/bin/true

# Optional: Use msys2_shell.cmd for better compatibility
ENTRYPOINT ["C:\\msys64\\msys2_shell.cmd", "-defterm", "-here", "-no-start"]

vagokuln-msft avatar Nov 05 '25 02:11 vagokuln-msft

This issue has been open for 30 days with no updates. no assignees, please provide an update or close this issue.