dotnet
Is it possible to use this package on MacOS with M1?
Apologies if this is my lack of brain power but after two days of trying to get this to work I thought I'd just check it's actually possible. I've had a search through the issues and there seem to be a few from people running on MacOS, but I'm at a loss as to how to get things to work.
Right now I have the following class:
public class DockerSetup : IAsyncLifetime
{
private readonly DockerClient _dockerClient;
private string? _containerId;
private const string ContainerImageUri = "mcr.microsoft.com/azure-sql-edge";
public DockerSetup()
{
_dockerClient = new DockerClientConfiguration(DockerApiUri()).CreateClient();
}
public async Task InitializeAsync()
{
await PullImage();
await StartContainer();
}
private Uri DockerApiUri()
{
var isWindows = RuntimeInformation.IsOSPlatform(OSPlatform.Windows);
if (isWindows)
{
return new Uri("npipe://./pipe/docker_engine");
}
var isLinux = RuntimeInformation.IsOSPlatform(OSPlatform.Linux);
var isMacOs = RuntimeInformation.IsOSPlatform(OSPlatform.OSX);
if (isLinux || isMacOs)
{
return new Uri("unix:/var/run/docker.sock");
}
throw new Exception(
"Was unable to determine what OS this is running on, does not appear to be Windows or Linux!?");
}
private async Task PullImage()
{
await _dockerClient.Images
.CreateImageAsync(new ImagesCreateParameters { FromImage = ContainerImageUri, Tag = "latest" },
new AuthConfig(),
new Progress<JSONMessage>());
}
private async Task StartContainer()
{
var response = await _dockerClient.Containers.CreateContainerAsync(new CreateContainerParameters
{
Image = ContainerImageUri, Env = new[] { "ACCEPT_EULA=y", "MSSQL_SA_PASSWORD=IntegrationTestingFTW!" }, HostConfig = new HostConfig { PortBindings = new Dictionary<string, IList<PortBinding>> { { "1433/tcp", new PortBinding[] { new PortBinding { HostPort = "14333" } } } } }
});
_containerId = response.ID;
await _dockerClient.Containers.StartContainerAsync(_containerId, null);
}
public async Task DisposeAsync()
{
if (_containerId != null)
{
await _dockerClient.Containers.KillContainerAsync(_containerId, new ContainerKillParameters());
}
}
}
But whenever I try to use it in a test I get an error System.Net.Http.HttpRequestException: Connection failed. Here is the info from dotnet --info
.NET SDK:
Version: 7.0.100-rc.2.22477.23
Commit: 0a5360315a
Runtime Environment:
OS Name: Mac OS X
OS Version: 12.6
OS Platform: Darwin
RID: osx.12-arm64
Base Path: /usr/local/share/dotnet/sdk/7.0.100-rc.2.22477.23/
Host:
Version: 7.0.0-rc.2.22472.3
Architecture: arm64
Commit: 550605cc93
.NET SDKs installed:
6.0.402 [/usr/local/share/dotnet/sdk]
7.0.100-rc.2.22477.23 [/usr/local/share/dotnet/sdk]
.NET runtimes installed:
Microsoft.AspNetCore.App 6.0.10 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.App]
Microsoft.AspNetCore.App 7.0.0-rc.2.22476.2 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.App]
Microsoft.NETCore.App 6.0.10 [/usr/local/share/dotnet/shared/Microsoft.NETCore.App]
Microsoft.NETCore.App 7.0.0-rc.2.22472.3 [/usr/local/share/dotnet/shared/Microsoft.NETCore.App]
And I'm running MacOS Monterey 12.6 with Docker 4.13.0.
Your example looks good, it runs on my Windows machine. I am expecting an M1 soon, then I can test it there too. Can you add the full stack trace of your error?
Is this a fresh installation of Docker Desktop (4.13.0)? Can you install 4.12.0 first and then do an update? We noticed some issues with a fresh 4.13.0.
I've removed DOcker 4.13.0 and done a fresh install of 4.12.0 and ran the code again and I'm getting the same error. Here is a full stack trace:
ystem.Net.Http.HttpRequestException: Connection failed
---> System.Net.Sockets.SocketException (49): Can't assign requested address
at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.CreateException(SocketError error, Boolean forAsyncThrow)
at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.ConnectAsync(Socket socket)
at System.Net.Sockets.Socket.ConnectAsync(EndPoint remoteEP, CancellationToken cancellationToken)
at System.Net.Sockets.Socket.ConnectAsync(EndPoint remoteEP)
at System.Net.Sockets.SocketTaskExtensions.ConnectAsync(Socket socket, EndPoint remoteEP)
at Docker.DotNet.DockerClient.<>c__DisplayClass6_0.<<-ctor>b__1>d.MoveNext()
at System.Runtime.CompilerServices.AsyncMethodBuilderCore.Start[TStateMachine](TStateMachine& stateMachine)
at Docker.DotNet.DockerClient.<>c__DisplayClass6_0.<.ctor>b__1(String host, Int32 port, CancellationToken cancellationToken)
at Microsoft.Net.Http.Client.ManagedHandler.ProcessRequestAsync(HttpRequestMessage request, CancellationToken cancellationToken)
at System.Runtime.CompilerServices.AsyncMethodBuilderCore.Start[TStateMachine](TStateMachine& stateMachine)
at Microsoft.Net.Http.Client.ManagedHandler.ProcessRequestAsync(HttpRequestMessage request, CancellationToken cancellationToken)
at Microsoft.Net.Http.Client.ManagedHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
at System.Runtime.CompilerServices.AsyncMethodBuilderCore.Start[TStateMachine](TStateMachine& stateMachine)
at Microsoft.Net.Http.Client.ManagedHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
at System.Net.Http.HttpMessageInvoker.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
at System.Net.Http.HttpClient.<SendAsync>g__Core|83_0(HttpRequestMessage request, HttpCompletionOption completionOption, CancellationTokenSource cts, Boolean disposeCts, CancellationTokenSource pendingRequestsCts, CancellationToken originalCancellationToken)
at System.Runtime.CompilerServices.AsyncMethodBuilderCore.Start[TStateMachine](TStateMachine& stateMachine)
at System.Net.Http.HttpClient.<SendAsync>g__Core|83_0(HttpRequestMessage request, HttpCompletionOption completionOption, CancellationTokenSource cts, Boolean disposeCts, CancellationTokenSource pendingRequestsCts, CancellationToken originalCancellationToken)
at System.Net.Http.HttpClient.SendAsync(HttpRequestMessage request, HttpCompletionOption completionOption, CancellationToken cancellationToken)
at Docker.DotNet.DockerClient.PrivateMakeRequestAsync(TimeSpan timeout, HttpCompletionOption completionOption, HttpMethod method, String path, IQueryString queryString, IDictionary`2 headers, IRequestContent data, CancellationToken cancellationToken)
at System.Runtime.CompilerServices.AsyncMethodBuilderCore.Start[TStateMachine](TStateMachine& stateMachine)
at Docker.DotNet.DockerClient.PrivateMakeRequestAsync(TimeSpan timeout, HttpCompletionOption completionOption, HttpMethod method, String path, IQueryString queryString, IDictionary`2 headers, IRequestContent data, CancellationToken cancellationToken)
at Docker.DotNet.DockerClient.MakeRequestForRawResponseAsync(HttpMethod method, String path, IQueryString queryString, IRequestContent body, IDictionary`2 headers, CancellationToken token)
at Docker.DotNet.ImageOperations.CreateImageAsync(ImagesCreateParameters parameters, Stream imageStream, AuthConfig authConfig, IDictionary`2 headers, IProgress`1 progress, CancellationToken cancellationToken)
at Docker.DotNet.ImageOperations.CreateImageAsync(ImagesCreateParameters parameters, Stream imageStream, AuthConfig authConfig, IProgress`1 progress, CancellationToken cancellationToken)
at Docker.DotNet.ImageOperations.CreateImageAsync(ImagesCreateParameters parameters, AuthConfig authConfig, IProgress`1 progress, CancellationToken cancellationToken)
at WorkAuthor.Web.IntegrationTests.Setup.DockerSetup.PullImage() in /Users/chrissainty/Git/Deployed/WorkAuthor/src/WorkAuthor.Api/tests/WorkAuthor.Api.Tests.Integration/src/Setup/DockerSetup.cs:line 47
The HttpClient is using http://docker.sock as its base address, which I believe is correct. However, I can't connect to that address using cURL (not sure if I should be able to, but I'm guessing so).
I know some developers using Testcontainers for .NET on an M1. Usually, they have issues with running the containers (compatibility), but not with pulling the images. Unfortunately, I do not have an M1 here, I will test it again soon as it arrives.
We currently run Azure sql edge on M1 Macs and regular sql on x86/x64 (for double the compatibility), I managed to get it to work using the TestContainers package that wraps Docker.DotNet with TestContainersBuilder with MSSqlTestContainer, with a couple of changes. This used to work on DockerDesktop 4.9.1, we migrated to Rancher Desktop after that, and it continues to work there, with one gotcha, in that Rancher desktop doesn't have a loopback for localhost so the connection string needed replacing with 127.0.0.1 to work, this shouldn't be needed with Docker desktop though. Hope this code helps you:
public SqlTestContainer Make()
{
var configuration = new MsSqlTestcontainerConfiguration { Password = this.sqlConnectionStringBuilder.Password };
var architecture = RuntimeInformation.ProcessArchitecture;
TestcontainersSettings.Logger ??= this.logger;
MsSqlTestcontainer testContainer;
switch(architecture)
{
case Architecture.X86:
case Architecture.X64:
testContainer = new TestcontainersBuilder<MsSqlTestcontainer>()
.WithDatabase(configuration)
.Build();
break;
case Architecture.Arm64:
const string started = "Recovery is complete. This is an informational message only. No user action is required.";
var stdout = new MemoryStream();
var stderr = new MemoryStream();
var consumer = Consume.RedirectStdoutAndStderrToStream(stdout, stderr);
testContainer = new TestcontainersBuilder<MsSqlTestcontainer>()
.WithDatabase(configuration)
.WithImage("mcr.microsoft.com/azure-sql-edge")
.WithOutputConsumer(consumer)
.WithWaitStrategy(Wait.ForUnixContainer().UntilMessageIsLogged(consumer.Stdout, started))
.Build();
break;
case Architecture.Arm:
case Architecture.Wasm:
case Architecture.S390x:
default:
throw new InvalidOperationException($"No container tested with architecture '{architecture}'");
}
return new SqlTestContainer(testContainer, this.sqlConnectionStringBuilder);
}
Thanks for the info @rossmasday. I think the issue is with connecting with the Docker API. The client is trying to connection to http://docker.sock but that doesn't appear to be responding to anything. I can't connect to it with cURL either. Is there some setting to enable the API or should it be available by default?
@chrissainty Check out the Docker release notes:
By default Docker will not create the /var/run/docker.sock symlink on the host and use the docker-desktop CLI context instead.
Maybe your are running into that?
@HofmeisterAn This looks to be the issue. So updating the DockerApiUrl method's code to this works:
var isLinux = RuntimeInformation.IsOSPlatform(OSPlatform.Linux);
var isMacOs = RuntimeInformation.IsOSPlatform(OSPlatform.OSX);
if (isLinux || isMacOs)
{
return new Uri("unix:///Users/chrissainty/.docker/run/docker.sock");
}
I guess this could be added to user secrets so each dev can specify their own location. I've tried updating the docker context to use default but that still doesn't make the usual unix:///var/run/docker.sock URL work. I'm a bit out of my depth here with Docker... I just want to write some integration tests 😭
So after sinking more hours into this, I'm ready to admit defeat--for now. We're going to setup Docker manually and I'll maybe revisit this in the future. Thanks for the help @HofmeisterAn, it's been much appreciated.
sudo ln -s $HOME/.docker/run/docker.sock /var/run/docker.sock does not help? Maybe this discussion helps: https://github.com/testcontainers/testcontainers-java/discussions/6045.
I'm facing the same issues using Testcontainers on a freshly installed M2, whereas I had it running on my previous M1 @HofmeisterAn.
Update: I downloaded the latest dmg from here and now it is working as expected: https://docs.docker.com/desktop/release-notes/#4170
@riezebosch We got a similar response in Slack a few days ago. According to the Docker release notes nothing has changed. Right now, we guess that depending on the permissions, Docker Desktop for Mac creates the symlink or not.
Is Docker activated and enabled in the macOS "Login Items" (Settings / General / Login Items / Allow in the Background)? Maybe it creats the symlink on startup. Otherwise, you can set the DOCKER_HOST env variable to the user daemon socket.
If I download the latest dmg from here it is working as expected: https://docs.docker.com/desktop/release-notes/#4170
Now, I am a bit confused. What did you use before? BTW, we are thinking to resolve the user Docker daemon automatically too.
@riezebosch We got a similar response in Slack a few days ago. According to the Docker release notes nothing has changed. Right now, we guess that depending on the permissions, Docker Desktop for Mac creates the symlink or not.
Is Docker activated and enabled in the macOS "Login Items" (Settings / General / Login Items / Allow in the Background)? Maybe it creats the symlink on startup. Otherwise, you can set the
DOCKER_HOSTenv variable to the user daemon socket.If I download the latest dmg from here it is working as expected: https://docs.docker.com/desktop/release-notes/#4170
Now, I am a bit confused. What did you use before? BTW, we are thinking to resolve the user Docker daemon automatically too.
I have no idea 🤷🏻 Looking back in my browser history I used the generic download url on 16th of March: https://desktop.docker.com/mac/main/arm64/Docker.dmg
That's probably the exact same version I downloaded today. So I did go from 4.17.0 to 4.16.3 to 4.17.0 again.
Meanwhile I had disabled it in the "Login Items".