Long Execution Time for dotnet restore in OpenShift Build Pipeline
During our build pipeline execution on OpenShift, we've encountered a significant delay specifically with the dotnet restore command in the Dockerfile. This delay is causing notable slowdowns in our overall build process.
My Dockerfile
#See https://aka.ms/containerfastmode to understand how Visual Studio uses this Dockerfile to build your images for faster debugging.
FROM mcr.microsoft.com/dotnet/aspnet:8.0 AS base
WORKDIR /app
EXPOSE 8080
EXPOSE 8081
FROM mcr.microsoft.com/dotnet/sdk:8.0 AS build
ARG BUILD_CONFIGURATION=Release
ARG HTTP_PROXY="<my-url-local-proxy>"
ENV DOTNET_NUGET_SIGNATURE_VERIFICATION=false
WORKDIR /src
COPY ["My.API/My.API.csproj", "My.API/"]
COPY ["My.Application/My.Application.csproj", "My.Application/"]
COPY ["My.Infrastructure/My.Infrastructure.csproj", "My.Infrastructure/"]
COPY "My.API/nuget.config" .
RUN dotnet restore -v diag "./My.API/./My.API.csproj"
COPY . .
WORKDIR "/src/My.API"
RUN dotnet build "./My.API.csproj" -c $BUILD_CONFIGURATION -o /app/build
FROM build AS publish
RUN dotnet publish "./My.API.csproj" -c $BUILD_CONFIGURATION -o /app/publish /p:UseAppHost=false
FROM base AS final
WORKDIR /app
COPY --from=publish /app/publish .
ENTRYPOINT ["dotnet", "My.API.dll"]
My nugget.config
<?xml version="1.0" encoding="utf-8"?>
<configuration>
<packageSources>
<clear />
<add key="nuget.org" value="https://api.nuget.org/v3/index.json" protocolVersion="3" />
</packageSources>
</configuration>
Version
I'm using openshift version 4.
Current Result
- Local result: I can work fine with my Dockerfile in my local.
- OpenShift result: The dotnet restore operation consistently takes 1-hour duration and seems to time out during the MSBuild within our OpenShift build pipeline.
Additional Information
- I saw a make sense information about NuGet signed-package verification https://github.com/NuGet/Home/issues/13062 and additional
ENV DOTNET_NUGET_SIGNATURE_VERIFICATION=falsein Dockerfile. But it didn't resolve yet.- You also need it for more information: https://learn.microsoft.com/en-us/dotnet/core/tools/nuget-signed-package-verification
- In my Dockerfile I also add a Proxy variable and use nuget.config file. You can look at it above.
- I tried to create another Node simple application image. But it also stuck on the
npm installstep. I think the same concept with thedotnet restorecommand. - I'm always looking at my resource quota. I don't see CPU, memory,... reach limit:
- I also tried to change
aspnet:8.0toaspnet:7.0. It still didn't success
Any insights or guidance will be expected. Thank you for your attention to this matter.
Update
I'm confused about the requests % in the Observe function
CPU quota
Memory quota
I had a same issue and my workaround is like that : added enviroment values (even for some of them are already used in the sdk):
ENV DOTNET_USE_POLLING_FILE_WATCHER=false
ENV NUGET_XMLDOC_MODE=skip
ENV NUGET_CERT_REVOCATION_MODE=offline
use mount=type=cache in each restore / build / publish
I have multiple services in one docker-compose and with that, I was able to decrease restore/build/publish time (bcs. of using same cache volume) time from like 1hour to 5min
-- I'm using low personal NAS-server for it so time are higher as on normal Win-PC
Dockerfile for single service:
FROM mcr.microsoft.com/dotnet/aspnet:8.0.2-alpine3.19-composite-arm64v8 AS base
WORKDIR /app
RUN apk update && apk add curl
RUN rm -rf /var/cache/apk/*
HEALTHCHECK --interval=5s --timeout=10s --retries=5 --start-period=30s \
CMD curl -f http://localhost:8080/api/health || exit 1
EXPOSE 8080
FROM mcr.microsoft.com/dotnet/sdk:8.0-alpine AS restorer
ARG BUILD_CONFIGURATION=Release
ARG BUILD_PLATFORM=linux-arm64
ENV DOTNET_NUGET_SIGNATURE_VERIFICATION=false
ENV DOTNET_USE_POLLING_FILE_WATCHER=false
ENV NUGET_XMLDOC_MODE=skip
ENV NUGET_CERT_REVOCATION_MODE=offline
RUN mkdir /src
WORKDIR /src
COPY ["Services/Catalog/Catalog.API/Catalog.API.csproj", "./Services/Catalog/Catalog.API/"]
COPY ["Services/Catalog/Catalog.Application/Catalog.Application.csproj", "./Services/Catalog/Catalog.Application/"]
COPY ["Services/Catalog/Catalog.Infrastructure/Catalog.Infrastructure.csproj", "./Services/Catalog/Catalog.Infrastructure/"]
COPY ["Services/Catalog/Catalog.Core/Catalog.Core.csproj", "./Services/Catalog/Catalog.Core/"]
COPY ["Services/Shared/Shared.Api/Shared.Api.csproj", "./Services/Shared/Shared.Api/"]
COPY ["Services/Shared/Shared.Application/Shared.Application.csproj", "./Services/Shared/Shared.Application/"]
COPY ["Services/Shared/Shared.Infrastructure/Shared.Infrastructure.csproj", "./Services/Shared/Shared.Infrastructure/"]
COPY ["Services/Shared/Shared.Core/Shared.Core.csproj", "./Services/Shared/Shared.Core/"]
WORKDIR "/src/Services/Catalog/Catalog.API"
RUN \
--mount=type=cache,id=nugethttpcache,sharing=locked,target=/root/.local/share/NuGet/http-cache \
--mount=type=cache,id=nugetglobalpackages,sharing=locked,target=/root/.nuget/packages/ \
--mount=type=cache,id=nugettemp,sharing=locked,target=/Temp/NuGet \
--mount=type=cache,id=nugetplugins-cache,sharing=locked,target=/root/.local/share/NuGet/plugin-cache \
# dotnet restore "./Catalog.API.csproj" -r $BUILD_PLATFORM --disable-parallel
dotnet restore "./Catalog.API.csproj" -r $BUILD_PLATFORM
FROM restorer AS build
WORKDIR /src
COPY . .
WORKDIR "/src/Services/Catalog/Catalog.API"
RUN \
--mount=type=cache,id=nugethttpcache,sharing=locked,target=/root/.local/share/NuGet/http-cache \
--mount=type=cache,id=nugetglobalpackages,sharing=locked,target=/root/.nuget/packages/ \
--mount=type=cache,id=nugettemp,sharing=locked,target=/Temp/NuGet \
--mount=type=cache,id=nugetplugins-cache,sharing=locked,target=/root/.local/share/NuGet/plugin-cache \
dotnet build "./Catalog.API.csproj" -c $BUILD_CONFIGURATION -r $BUILD_PLATFORM --self-contained false -o /app/build
FROM build AS publish
RUN \
--mount=type=cache,id=nugethttpcache,sharing=locked,target=/root/.local/share/NuGet/http-cache \
--mount=type=cache,id=nugetglobalpackages,sharing=locked,target=/root/.nuget/packages/ \
--mount=type=cache,id=nugettemp,sharing=locked,target=/Temp/NuGet \
--mount=type=cache,id=nugetplugins-cache,sharing=locked,target=/root/.local/share/NuGet/plugin-cache \
dotnet publish "./Catalog.API.csproj" -c $BUILD_CONFIGURATION -r $BUILD_PLATFORM --self-contained false -o /app/publish /p:UseAppHost=false
FROM base AS final
WORKDIR /app
COPY --from=publish /app/publish .
ENTRYPOINT ["dotnet", "/app/Catalog.API.dll"]
This issue may occur when the build pod reaches its limit. Please try adding more resources to the Tekton builder pod, as shown below:
- pipelineTaskName: build
computeResources:
requests:
memory: "512Mi"
cpu: "250m"
limits:
memory: "1024Mi"
cpu: "500m
Read more at: https://github.com/tektoncd/pipeline/blob/main/docs/pipelineruns.md#specifying-taskrunspecs
It works for me if I add taskRunSpecs in the trigger-template.yaml . Thanks @dungphamdev
Issues go stale after 90d of inactivity.
Mark the issue as fresh by commenting /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.
Exclude this issue from closing by commenting /lifecycle frozen.
If this issue is safe to close now please do so with /close.
/lifecycle stale
Stale issues rot after 30d of inactivity.
Mark the issue as fresh by commenting /remove-lifecycle rotten.
Rotten issues close after an additional 30d of inactivity.
Exclude this issue from closing by commenting /lifecycle frozen.
If this issue is safe to close now please do so with /close.
/lifecycle rotten /remove-lifecycle stale
Rotten issues close after 30d of inactivity.
Reopen the issue by commenting /reopen.
Mark the issue as fresh by commenting /remove-lifecycle rotten.
Exclude this issue from closing again by commenting /lifecycle frozen.
/close
@openshift-bot: Closing this issue.
In response to this:
Rotten issues close after 30d of inactivity.
Reopen the issue by commenting
/reopen. Mark the issue as fresh by commenting/remove-lifecycle rotten. Exclude this issue from closing again by commenting/lifecycle frozen./close
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.