Registry base URL is not the proper entry point for NuGet

[joelverhagen]

Describe the bug

Today, the registryBaseUrl serves as a pointer to which registry the package is stored in. This maybe makes sense for some package registries (i.e. base URL = entry point) but for NuGet the entry point is not a base URL, it is a URL to a JSON service index.

For example, NuGet clients configure "https://api.nuget.org/v3/index.json" NOT "https://api.nuget.org".

This line currently violates the service discovery protocol we have for NuGet: https://github.com/modelcontextprotocol/registry/blob/18cde88a7e1c7916f1433c1f131a8de2fec8fe65/internal/validators/registries/nuget.go#L51 (technically this should read the service index JSON document and get the README URL pattern with the ReadmeUriTemplate/6.13.0 service ID.

It is not correct to assume the /v3/index.json or /v3-flatcontainer/%s/%s/readme can be appended to a NuGet registry base URL to get a valid endpoint. This works for NuGet.org, but not other NuGet feed implementations.

Expected behavior

Ideally, for NuGet, the server.json property would be registryUrl in which case we would provide/default to https://api.nuget.org/v3/index.json.

Generally speaking, I think registryUrl is a little more generic and conceptually also supports a base URL if that is idiomatic for the given package registry type.

But the property name is already chosen and locked in the schema. Also, there is a data on PROD using that property name and value.

I propose:

1. Introduce an additional property registryUrl
2. For NuGet.org, recommend registryUrl but accept https://api.nuget.org and internally treat that as registryUrl = https://api.nuget.org/v3/index.json.
3. Enhance the NuGet validator code to do proper service discovery.

I am happy to implement all of this.