[tests] Hosting.Azure tests broken after the move to snapshot based tests

[radical]

Aspire.Hosting.Azure.Tests broke after the two PRs for moving to snapshot based testing - #8930 and #9009 .

https://dev.azure.com/dnceng-public/public/_build/results?buildId=1029256&view=logs&j=dc0c59fc-6459-516f-9358-56938432eb13&t=bd6402dd-1caa-5b2e-8254-6909970e2dbf https://dev.azure.com/dnceng-public/public/_build/results?buildId=1029334&view=ms.vss-test-web.build-test-results-tab

cc @RussKie @danmoseley

Failures from the first commit: Windows:

failed Aspire.Hosting.Azure.Tests.AzureProvisioningResourceExtensionsTests.AsProvisioningParameterTests (2s 522ms)
  Xunit.Runner.InProc.SystemConsole.TestingPlatform.XunitException: VerifyException : Directory: D:\a\_work\1\s\tests\Aspire.Hosting.Azure.Tests
  New:
    - Received: AzureProvisioningResourceExtensionsTests.AsProvisioningParameterTests.received.bicep
      Verified: AzureProvisioningResourceExtensionsTests.AsProvisioningParameterTests.verified.bicep
  
  FileContent:
  
  New:
  
  Received: AzureProvisioningResourceExtensionsTests.AsProvisioningParameterTests.received.bicep
  @description('The location for the resource(s) to be deployed.')
  param location string = resourceGroup().location
  
  param endpointAddressParam string
  
  param someExpressionParam string
  
  resource app 'Microsoft.App/containerApps@2024-03-01' = {
    name: take('app-${uniqueString(resourceGroup().id)}', 32)
    location: location
    properties: {
      template: {
        scale: {
          rules: [
            {
              name: 'temp'
              custom: {
                type: 'external'
                metadata: {
                  address: endpointAddressParam
                  someExpression: someExpressionParam
                }
              }
            }
          ]
        }
      }
    }
  }

Linux:

failed Aspire.Hosting.Azure.Tests.AzureProvisioningResourceExtensionsTests.AsProvisioningParameterTests (66ms)
  Xunit.Runner.InProc.SystemConsole.TestingPlatform.XunitException: System.UnauthorizedAccessException : Access to the path '/mnt/vss' is denied.
  ---- System.IO.IOException : Permission denied
    at System.IO.FileSystem.CreateParentsAndDirectory(String fullPath, UnixFileMode unixCreateMode)
    at System.IO.FileSystem.CreateDirectory(String fullPath, UnixFileMode unixCreateMode)
    at System.IO.Directory.CreateDirectory(String path)
    at IoHelpers.CreateDirectory(String directory) in /_/src/Verify/IoHelpers.cs:94
    at VerifyTests.InnerVerifier.ResolveDirectory(String sourceFile, VerifySettings settings, PathInfo pathInfo) in /_/src/Verify/Verifier/InnerVerifier.cs:368
    at VerifyTests.InnerVerifier..ctor(String sourceFile, VerifySettings settings, String typeName, String methodName, IReadOnlyList`1 methodParameters, PathInfo pathInfo) in /_/src/Verify/Verifier/InnerVerifier.cs:58
    at VerifyXunit.Verifier.BuildVerifier(VerifySettings settings, String sourceFile, Boolean useUniqueDirectory) in /_/src/Verify.XunitV3/Verifier.cs:25
    at VerifyXunit.Verifier.<>c__DisplayClass10_0.<<Verify>b__0>d.MoveNext() in /_/src/Verify.XunitV3/Verifier.cs:99
    --- End of stack trace from previous location ---
    at Aspire.Hosting.Azure.Tests.AzureProvisioningResourceExtensionsTests.AsProvisioningParameterTests() in /mnt/vss/_work/1/s/tests/Aspire.Hosting.Azure.Tests/AzureProvisioningResourceExtensionsTests.cs:62
    --- End of stack trace from previous location ---
    ----- Inner Stack Trace -----

Report

Summary

24-Hour Hit Count	7-Day Hit Count	1-Month Count
0	0	0

[danmoseley]

is the learning here -- before making changes of this scale (if they're worthwhile), let's kick off a private azdo run?

[radical]

is the learning here -- before making changes of this scale (if they're worthwhile), let's kick off a private azdo run?

Yes, definitely. Any infrastructure stuff that isn't github actions specific, or any big test changes should get a azdo run.

[RussKie]

:O Are the verified files pushed to Helix? Or is there something we need to do to push them there?

[radical]

:O Are the verified files pushed to Helix? Or is there something we need to do to push them there?

You can add them as payload for the helix jobs like: https://github.com/dotnet/aspire/blob/44630f56ec00181ab87f97f01b045bf553df1514/tests/helix/send-to-helix-inner.proj#L80-L81

Add them in tests/helix/send-to-helix-basictests.targets.

[captainsafia]

In case it is helpful, here's what I do for using Verify in the ASP.NET core repo. Push them to Helix in the build:

https://github.com/dotnet/aspnetcore/blob/21ba24a7015eee34b6879b4fd2f0b94b5ff8faa6/src/Http/Http.Extensions/test/Microsoft.AspNetCore.Http.Extensions.Tests.csproj#L48

Then update the verification code to resolve from the correct directory based on the environment:

https://github.com/dotnet/aspnetcore/blob/21ba24a7015eee34b6879b4fd2f0b94b5ff8faa6/src/Http/Http.Extensions/test/ValidationsGenerator/ValidationsGeneratorTestBase.cs#L73-L75

[RussKie]

Thank you @captainsafia, this is very helpful.