dotnet-operator-sdk icon indicating copy to clipboard operation
dotnet-operator-sdk copied to clipboard

[bug]: UseCertificateProvider() not working very well

Open UTF-8x opened this issue 11 months ago • 1 comments

Describe the bug

When I use a CertificateGenerator and UseCertificateProvider() for local development with minikube, I get a TLS error when k8s tries to call a Webhook. There is nothing helpful in the logs, even in Debug.

Error from server (InternalError): error when creating ".\\Examples\\DnsBackend.yaml": Internal error occurred: failed calling webhook "mutate.dnsbackend.hades.bluecarbon.com.v1alpha1": failed to call webhook: Post "https://host.minikube.internal/mutate/v1dnsbackend?timeout=10s": x509: certificate signed by unknown authority
Program.cs
using KubeOps.Operator;
using KubeOps.Operator.Web.Builder;
using KubeOps.Operator.Web.Certificates;

var builder = WebApplication.CreateBuilder(args);

string ip = "host.minikube.internal";
ushort port = 443;

using var generator = new CertificateGenerator(ip);
var cert = generator.Server.CopyServerCertWithPrivateKey();

builder.WebHost.ConfigureKestrel(so =>
{
    so.Listen(System.Net.IPAddress.Any, port, lo =>
    {
        lo.UseHttps(cert);
    });
});

builder.Services
    .AddKubernetesOperator()
    .RegisterComponents()
    .UseCertificateProvider(port, ip, generator);
    
builder.Services.AddControllers();

var app = builder.Build();

app.UseRouting();
app.UseDeveloperExceptionPage();
app.MapControllers();

await app.RunAsync();
V1DnsBackendController.cs
[EntityRbac(typeof(V1DnsBackend), Verbs = RbacVerb.All)]
public class V1DnsBackendController(ILogger<V1DnsBackendController> log) : IEntityController<V1DnsBackend>
{
    public Task ReconcileAsync(V1DnsBackend entity, CancellationToken cancellationToken)
    {
        log.LogInformation("Reconciling entity {Entity}.", entity);
        return Task.CompletedTask;
    }

    public Task DeletedAsync(V1DnsBackend entity, CancellationToken cancellationToken)
    {
        log.LogInformation("Deleted entity {Entity}.", entity);
        return Task.CompletedTask;
    }
}
V1DnsBackendMutationWebhook.cs
[MutationWebhook(typeof(V1DnsBackend))]
public class V1DnsBackendMutationWebhook : MutationWebhook<V1DnsBackend>
{
    public override MutationResult<V1DnsBackend> Create(V1DnsBackend entity, bool dryRun)
    {
        entity.Status.Bound = true;
        
        return base.Create(entity, dryRun);
    }
}

To reproduce

  1. Create a new Webhook Operator project
  2. Set up a certificate generator and call UseCertificateProvider()
  3. Add a custom entity with a mutation webhook
  4. Create a new resource in k8s that will call this webhook

Expected behavior

The CA is correctly injected into the cluster

Screenshots

No response

Additional Context

  • .net9.0
  • KubeOps.* 9.2.0
  • Windows 10
  • Minikube in Docker Desktop
  • k8s 1.23.0

UTF-8x avatar Jan 29 '25 09:01 UTF-8x