Use testcontainer for testing Cosmos DB

[ajcvickers]

https://testcontainers.com/modules/cosmosdb/

[ajcvickers]

Tried this, but the following code ~~just hangs~~ on StartAsync.

var cosmosDbContainer = new CosmosDbBuilder()
    .WithImage("mcr.microsoft.com/cosmosdb/linux/azure-cosmos-emulator:latest")
    .Build();
await cosmosDbContainer.StartAsync();

Turns out you have to have to wait many more minutes than I was doing.

[ajcvickers]

Next issue:

Unhandled exception. System.Net.Http.HttpRequestException: The SSL connection could not be established, see inner exception.
 ---> System.IO.IOException: Received an unexpected EOF or 0 bytes from the transport stream.
   at System.Net.Security.SslStream.ReceiveHandshakeFrameAsync[TIOAdapter](CancellationToken cancellationToken)
   at System.Net.Security.SslStream.ForceAuthenticationAsync[TIOAdapter](Boolean receiveFirst, Byte[] reAuthenticationData, CancellationToken cancellationToken)
   at System.Net.Http.ConnectHelper.EstablishSslConnectionAsync(SslClientAuthenticationOptions sslOptions, HttpRequestMessage request, Boolean async, Stream stream, CancellationToken cancellationToken)
   --- End of inner exception stack trace ---
   at System.Net.Http.ConnectHelper.EstablishSslConnectionAsync(SslClientAuthenticationOptions sslOptions, HttpRequestMessage request, Boolean async, Stream stream, CancellationToken cancellationToken)
   at System.Net.Http.HttpConnectionPool.ConnectAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
   at System.Net.Http.HttpConnectionPool.CreateHttp11ConnectionAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
   at System.Net.Http.HttpConnectionPool.AddHttp11ConnectionAsync(QueueItem queueItem)
   at System.Threading.Tasks.TaskCompletionSourceWithCancellation`1.WaitWithCancellationAsync(CancellationToken cancellationToken)
   at System.Net.Http.HttpConnectionPool.SendWithVersionDetectionAndRetryAsync(HttpRequestMessage request, Boolean async, Boolean doRequestAuth, CancellationToken cancellationToken)
   at System.Net.Http.RedirectHandler.SendAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
   at System.Net.Http.HttpClient.<SendAsync>g__Core|83_0(HttpRequestMessage request, HttpCompletionOption completionOption, CancellationTokenSource cts, Boolean disposeCts, CancellationTokenSource pendingRequestsCts, CancellationToken originalCancellationToken)
   at Microsoft.Azure.Cosmos.CosmosHttpClientCore.ExecuteHttpHelperAsync(HttpRequestMessage requestMessage, ResourceType resourceType, CancellationToken cancellationToken)
   at Microsoft.Azure.Cosmos.CosmosHttpClientCore.SendHttpHelperAsync(Func`1 createRequestMessageAsync, ResourceType resourceType, HttpTimeoutPolicy timeoutPolicy, IClientSideRequestStatistics clientSideRequestStatistics, CancellationToken cancellationToken)
   at Microsoft.Azure.Cosmos.GatewayAccountReader.GetDatabaseAccountAsync(Uri serviceEndpoint)
   at Microsoft.Azure.Cosmos.Routing.GlobalEndpointManager.GetAccountPropertiesHelper.GetAndUpdateAccountPropertiesAsync(Uri endpoint)
   at Microsoft.Azure.Cosmos.Routing.GlobalEndpointManager.GetAccountPropertiesHelper.GetAccountPropertiesAsync()
   at Microsoft.Azure.Cosmos.Routing.GlobalEndpointManager.GetDatabaseAccountFromAnyLocationsAsync(Uri defaultEndpoint, IList`1 locations, Func`2 getDatabaseAccountFn, CancellationToken cancellationToken)
   at Microsoft.Azure.Cosmos.GatewayAccountReader.InitializeReaderAsync()
   at Microsoft.Azure.Cosmos.CosmosAccountServiceConfiguration.InitializeAsync()
   at Microsoft.Azure.Cosmos.DocumentClient.InitializeGatewayConfigurationReaderAsync()
   at Microsoft.Azure.Cosmos.DocumentClient.GetInitializationTaskAsync(IStoreClientFactory storeClientFactory)
   at Microsoft.Azure.Documents.BackoffRetryUtility`1.ExecuteRetryAsync[TParam,TPolicy](Func`1 callbackMethod, Func`3 callbackMethodWithParam, Func`2 callbackMethodWithPolicy, TParam param, IRetryPolicy retryPolicy, IRetryPolicy`1 retryPolicyWithArg, Func`1 inBackoffAlternateCallbackMethod, Func`2 inBackoff
   at Microsoft.Azure.Documents.ShouldRetryResult.ThrowIfDoneTrying(ExceptionDispatchInfo capturedException)
   at Microsoft.Azure.Documents.BackoffRetryUtility`1.ExecuteRetryAsync[TParam,TPolicy](Func`1 callbackMethod, Func`3 callbackMethodWithParam, Func`2 callbackMethodWithPolicy, TParam param, IRetryPolicy retryPolicy, IRetryPolicy`1 retryPolicyWithArg, Func`1 inBackoffAlternateCallbackMethod, Func`2 inBackoff
   at Microsoft.Azure.Documents.BackoffRetryUtility`1.ExecuteRetryAsync[TParam,TPolicy](Func`1 callbackMethod, Func`3 callbackMethodWithParam, Func`2 callbackMethodWithPolicy, TParam param, IRetryPolicy retryPolicy, IRetryPolicy`1 retryPolicyWithArg, Func`1 inBackoffAlternateCallbackMethod, Func`2 inBackoff
   at Microsoft.Azure.Cosmos.AsyncCacheNonBlocking`2.GetAsync(TKey key, Func`2 singleValueInitFunc, Func`2 forceRefresh)
   at Microsoft.Azure.Cosmos.DocumentClient.EnsureValidClientAsync(ITrace trace)
   at Microsoft.Azure.Cosmos.Handlers.RequestInvokerHandler.EnsureValidClientAsync(RequestMessage request, ITrace trace)
   at Microsoft.Azure.Cosmos.Handlers.RequestInvokerHandler.SendAsync(RequestMessage request, CancellationToken cancellationToken)
   at Microsoft.Azure.Cosmos.Handlers.RequestInvokerHandler.SendAsync(String resourceUriString, ResourceType resourceType, OperationType operationType, RequestOptions requestOptions, ContainerInternal cosmosContainerCore, FeedRange feedRange, Stream streamPayload, Action`1 requestEnricher, ITrace trace, Can
   at Microsoft.Azure.Cosmos.CosmosClient.<>c__DisplayClass54_0.<<CreateDatabaseIfNotExistsAsync>b__0>d.MoveNext()
--- End of stack trace from previous location ---
   at Microsoft.Azure.Cosmos.ClientContextCore.RunWithDiagnosticsHelperAsync[TResult](String containerName, String databaseName, OperationType operationType, ITrace trace, Func`2 task, Func`2 openTelemetry, String operationName, RequestOptions requestOptions)
   at Microsoft.Azure.Cosmos.ClientContextCore.OperationHelperWithRootTraceAsync[TResult](String operationName, String containerName, String databaseName, OperationType operationType, RequestOptions requestOptions, Func`2 task, Func`2 openTelemetry, TraceComponent traceComponent, TraceLevel traceLevel)     
   at Program.<Main>$(String[] args) in D:\code\AllTogetherNow\Cosmos\Program.cs:line 22
   at Program.<Main>(String[] args)

Process finished with exit code -532,462,766.

Giving up on this for now.

[roji]

Turns out you have to have to wait many more minutes than I was doing

This was a first-time only thing, right? As the container image needs to be downloaded? In later runs the image is cached so things should start up very fast.

The SSL connection could not be established

For this I imagine you need to configure the SDK to not validate the certificate - I made this change to our test suite in #25528, you can try copying that.

[ajcvickers]

This was a first-time only thing, right? As the container image needs to be downloaded? In later runs the image is cached so things should start up very fast.

Nope. It takes several minutes to start even when the container is downloaded.

For this I imagine you need to configure the SDK to not validate the certificate - I made this change to our test suite in https://github.com/dotnet/efcore/pull/25528, you can try copying that.

Good to know.

[roji]

Nope. It takes several minutes to start even when the container is downloaded.

Wow, that's super weird. I wonder why.

[ajcvickers]

Starting the emulator on my machine takes a similar amount of time, so if it starts the emulator when the test container starts, then that would be why.

[roji]

Several minutes, really? I don't think it was like that for me when I last tried it...

[ajcvickers]

Okay, I timed it, and my perception is a little off, but it's still very slow:

• Starting the emulator on my machine: 2 mins 4 seconds.
• Starting the test container (after initial run to download): 48 seconds.

[roji]

Yeah, 48 seconds is unacceptable for test startup... That definitely seems like a lot...

[SilverioMiranda]

Same for me

[smolesen]

Did anyone find a solution for this?

[martincostello]

On my laptop it takes about a 30 seconds or so to start up once docker has pulled the image. I was able to reduce it by configuring less partitions:

using Testcontainers.CosmosDb;
using Xunit;

namespace OpenTelemetry.Instrumentation.EntityFrameworkCore.Tests;

public sealed class CosmosIntegrationTestsFixture : IAsyncLifetime
{
    public CosmosDbContainer DatabaseContainer { get; } = CreateCosmos();

    public Task InitializeAsync()
        => this.DatabaseContainer.StartAsync();

    public Task DisposeAsync()
        => this.DatabaseContainer.DisposeAsync().AsTask();

    private static CosmosDbContainer CreateCosmos()
        => new CosmosDbBuilder()
               .WithEnvironment("AZURE_COSMOS_EMULATOR_PARTITION_COUNT", "1") // Improves startup time
               .Build();
}

I also had to configure things as below to avoid the TLS errors:

builder.UseCosmos(cosmosConnectionString, "collection", (configure) =>
{
    // Override options to allow insecure connection to self-signed TLS certificate
    // in the CosmosDB test container. See https://stackoverflow.com/a/77973619/1064169.
    configure
        .ConnectionMode(Microsoft.Azure.Cosmos.ConnectionMode.Gateway)
        .LimitToEndpoint()
        .HttpClientFactory(() =>
        {
            var handler = new HttpClientHandler()
            {
                ServerCertificateCustomValidationCallback = HttpClientHandler.DangerousAcceptAnyServerCertificateValidator,
            };

    return new(handler);
});

[roji]

I think the ideal thing here would be the new Cosmos Linux-based emulator, still in preview; that emulator seems ideal for this kind of thing. Unfortunately, it's still in early stages and does not yet support enough functionality to be used in the EF provider tests. Hopefully that'll change in the future.