platform-compat
platform-compat copied to clipboard
dotnet
Deprecate HttpListener
System.Net.HttpListener does not support many modern protocols. It is compat-only (i.e. only critical fixes, no new improvements, enhancements). It's useful for low-volume basic server requests, but does not scale to modern requirements of HTTP servers (incl. performance).
Use KestrelHttpServer instead.
But Kestrel on Unix does not directly support the Windows authentication. https://github.com/aspnet/KestrelHttpServer/issues/2168 In my project, which I want to transfer to Unix, I use a HttpListener with Windows authentication.
@valeri2 Even Kestrel doesn't want to implement Windows authentication on Linux due to security concerns: https://github.com/aspnet/KestrelHttpServer/issues/2168#issuecomment-345033067. The less reason to implement it in HttpListener.
I would not consider HttpListener deprecated. Its directly tied to http.sys driver which constantly gets updated with newer IIS or OS versions since IIS itself heavily depends on it. Kestrel only can bind to entire ports while using http.sys/HttpListener many applications can simultaneously bind to port 80 as long as they use non-conflicting paths.
@springy76 not all HttpListener implementations are based on http.sys (e.g. UWP, Linux/Mac on .NET Core). We might eventually migrate even the Windows implementation out of http.sys to remove inconsistencies between platforms and to simplify maintenance - it is something we plan to consider.
Even if http.sys is maintained, it does not mean that new APIs will be exposed from HttpListener, or that HttpListener will opt-in into new options, settings and APIs in http.sys.
From .NET point of view, HttpListener is legacy without active work on it.
@karelz Wait a sec. Someone points out how being based on http.sys provides very useful functionality that they're relying on, that Kestrel can't duplicate, and your response is "meh, we're probably going to remove that anyway"? Seriously?!?
Also, how is it that you regard something that drags in a dependency on ASP.Net Core as a good alternative to a simple, lightweight server class with no dependencies outside of the standard library? Sorry, but no. From the perspective of a developer actually using it, that's simply insane!
@masonwheeler we need to weight in usefulness vs. maintenance cost. Is that particular feature so much more important to EVERY developer vs. other investments. Maintaining something, keeping it up to latest standards, etc. does cost something - see the cost in Kestrel. Using your arguments we would not be able to remove libcurl Http client ever, because it may have some useful corner-case features, we can't provide easily in SocketsHttpHandler.
If you want to help change things, I would recommend to focus on facts, and usage scenarios / commonality (e.g. how many customers truly depend on specific feature?)
Also note, that the fact we do not evolve HttpListener further, that it is compat-only API is a fact since its introduction into .NET Core 2.0. The deprecation is just clear public statement of where we are and where we have been for last year. It sets expectations. If you want something modern, it is not HttpListener.
@karelz
If you want to help change things, I would recommend to focus on facts, and usage scenarios / commonality
I chose HttpListener specifically because it does not drag dependencies on a massive, bloated ASP.Net framework into your project. Saying now that it's deprecated in favor of something that does is no good. Deprecation is supposed to mean "don't use this because we've got a better alternative." Providing a worse alternative is a very bad use of the concept. The Khronos Group is infamous for not understanding this; please don't be like them.
Also note, that the fact we do not evolve HttpListener further, that it is compat-only API is a fact since its introduction into .NET Core 2.0. The deprecation is just clear public statement of where we are and where we have been for last year. It sets expectations. If you want something modern, it is not HttpListener.
Yes, I understand that that's your position. I'm saying that this is a bad position and needs to be reconsidered. If there are new and better core HTTP server features available, what's stopping you from making them available to everyone even if they don't want to bloat up their project with unwanted frameworks and unnecessary IIS crud?
So, late to the party here. Please consider the magic that is happening over at Ooui, which allows you to pipe the rendering of .NET elements straight into a webpage. Additionally, you can perform this same magic in a native-hosted/store scenario (UWP as an example). This is all supported at present through the use of HttpListener. If I am understanding this correctly, this issue is saying that we would remove the use of System.Net and instead add a dependency to Microsoft.AspNetCore.Server.Kestrel in our native-hosted/store scenarios to get the same functionality? It would be incredibly valuable to know that what "just works" with HttpListener presently with Ooui also "just works" with KestrelHttpServer, too.
First a little bit of history: Long time ago, HttpListener was found to be insufficient in the areas of modern feature richness, modern API surface and performance characteristics. ASP.NET team forked the code base and evolved it into WebListener, which then evolved into Kestrel.
The moment this evolution path started, HttpListener became maintenance-only (aka non-modern aka obsolete).
If current HttpListener feature set is sufficient for your needs, you can still use it. We are just warning users to not expect modern evolution, APIs and performance improvements in the space. If you need any of that, we suggest to consider Kestrel or other alternatives.
@masonwheeler I chose HttpListener specifically because it does not drag dependencies on a massive ... Providing a worse alternative is a very bad use of the concept.
HttpListener may have fewer dependencies, for which you pay by limited capabilities (e.g. feature set, perf). Overall we are convinced that Kestrel is the better alternative from many point of views, especially those which are most important to majority of our customers.
@masonwheeler I'm saying that this is a bad position and needs to be reconsidered. If there are new and better core HTTP server features available, what's stopping you from making them available to everyone even if they don't want to bloat up their project with unwanted frameworks and unnecessary IIS crud?
As I suggested above, if you want us to reconsider, we will need some hard data on usage, limitations of Kestrel, etc. So far I am aware of only 1 feature which is not better in Kestrel. Its usefulness seems to be fairly limited. Maintaining multiple HTTP servers is cost, it does not come for free. The cost is high (you have to do everything twice at minimum), it also fractures ecosystem by providing multiple official ways of doing the same thing. Investing into Kestrel to fulfill all the needs of various HTTP server scenarios seems to be overall better investment. BTW: AFAIK the IIS dependency in Kestrel is just one of the options, it is not tied to it.
@Mike-EEE this issue is saying that we would remove the use of System.Net and instead add a dependency to Microsoft.AspNetCore.Server.Kestrel in our native-hosted/store scenarios to get the same functionality?
If your projects are satisfied with current limited HttpListener capabilities "as is", you can keep using it. If you need more, you might want to consider Kestrel (although I don't think it is available in UWP).
HttpListener is not going to disappear, however, you should not expect it to evolve further. It has reached its full potential.
It would be incredibly valuable to know that what "just works" with HttpListener presently with Ooui also "just works" with KestrelHttpServer, too.
That sounds like a very good idea. I am sure quite a few developers would appreciate that. I hope some Kestrel experts will be able to provide such guidance.
FYI: For context, check some of the discussion here: https://github.com/dotnet/designs/issues/9 (just please do not reopen the discussion there - it does not belong on the uber-issue).
If there are additional real-world usage patterns of HttpListener, I'll be happy to hear details.
Cool... thanks @karelz ... I am starting to dig and do some analysis around this space, myself. In my view it is incredibly important as at a glance it totally (and finally) satisfies the request for a Ubiquitous.NET. That is, using this magic bridge/connection, you can enable a world where it's one .NET codebase and one .NET viewbase total in your solution, rendered through a WebView (as presented/demonstrated above). That is a big deal and I do not want to jeopardize it. 😄
So it does sound like at the very least we will still be able to use HttpListener, but it will be deprecated? I am sure that will feature all the expected warnings in code as such.
If you need more, you might want to consider Kestrel (although I don't think it is available in UWP).
So I thought that UWP supported .NET Standard 2.0? That is the TargetFramework of Microsoft.AspNetCore.Server.Kestrel so it should "just work" right? Or am I missing something (again 😆)?
So it does sound like at the very least we will still be able to use HttpListener, but it will be deprecated? I am sure that will feature all the expected warnings in code as such.
Yes, I am not aware of any instance when we deprecate things and they immediately stop working :). The warnings are soft, easy to disable and opt-in currently - read more in this repo about the platform compat tooL; https://github.com/dotnet/platform-compat
So I thought that UWP supported .NET Standard 2.0? That is the
TargetFrameworkofMicrosoft.AspNetCore.Server.Kestrelso it should "just work" right?
I'd ask Kestrel folks where and how they are supported. IMO Kestrel needs to listen on ports which is not supported in UWP. Not sure if it can work without that (you have reached the limits of my knowledge).
BTW: Also note, that there are currently limitations on various platforms:
-
HttpListenerdoes not support HTTPS in UWP and on Linux - https://github.com/dotnet/corefx/issues/14691 (because it is C# implementation, not based on http.sys which is not available on the platforms). - WebSocket not supported on Win7 - https://github.com/dotnet/corefx/issues/13663#issuecomment-313783837.
Kestrel needs to listen on ports which is not supported in UWP
So to add a little more context here, the HttpListener in Ooui is listening on a port and works in UWP. This is because it isn't intended for any external connections, and it's only listening for the process in which it is running. This is a supported scenario in UWP, so it would be interesting to see if Kestrel works as well when listening within its own process.
and it's only listening for the process in which it is running
I wonder how this is being checked?
@springy76 It's probably better stated that it doesn't accept any connections from external requests, so as a result it only accepts requests/connections running from within the process from which the application is running. That is my understanding, at least. Also, while I have gotten this to work locally on UWP, it hasn't been officially verified as a deployed product.
As such, I do have an outstanding question on Xamarin's forums, but if I do not get an answer there, I will have to brogrammer-up and get some HelloWorld's deployed to get a definitive answer. Unfortunately, Droid development hasn't exactly been friendly to me, and I haven't even touched iOS.
If someone has some valuable knowledge around these scenarios it would be greatly appreciated if you could share. 👍
@masonwheeler
See discussion at: https://github.com/dotnet/designs/issues/9
@valeri2 , it seems to me that Kerberos (e.g. Active Directory) is one of the few reasons a developer would want to run an application on Windows vs. Linux. The Linux community by and large doesn't want to deal with Kerberos, whereas Windows has been pretty good at handling it for a long time. If Kerberos is a requirement (and tbh I would use if I could for an internal or enterprise app), you should stick with Windows.
I would also like it if you could keep the HttpListener as it is and maybe evolve it as a no-frills HTTP Api/Server alongside the other ones. Seeing how many times it has been ported by other people already there seems to be plenty of demand for it.
As for my own use: I like that in 3 lines of code I can listen for a connection and do almost anything with it without needing a lot of dependencies or learning a new paradigm.
There are some things that can easily be made better, like the certificate format it expects. And some other things I am sure you are already aware of.
In other words, I think you'd make a lot of people happy, including me, if you keep HttpListener around and Microsoft would still support it.
Hello @karelz , your recommendation is not to use HttpClient and use Kestrel that has been moved into ASP.NET Core. However, I don't fully understand how to go from one to the other. I have a desktop application that listens on port 443 and replies web requests, and not full blown ASP.NET Core project with all the configuration and setup that a web server requires, frameworks, paradigms and all its baggage. Is there a way to make a console application to respond to HTTP requests? or are we adding the constraint that only ASP.NET Core projects can do that?
@pablocar80 AFAIK you can write minimal Kestrel server with little overhead, baggage, etc. Check out their docs, I am not expert on the topic.
The docs are at https://docs.microsoft.com/en-us/aspnet/core/fundamentals/servers/kestrel?view=aspnetcore-2.2 . The minimal sample linked from the docs is not a full blown ASP.NET project, but it still carries some of the baggage with it: https://github.com/aspnet/Docs/blob/master/aspnetcore/fundamentals/servers/kestrel/samples/2.x/KestrelSample/Startup.cs#L4
cc @davidfowl
Would it make sense for the DotNetCore team to offer the HttpClient class as a thin wrapper that under the hood uses Kestrel? This would help with migration A LOT.
@jkotas thank you for the links. I checked out the documentation and the references needed are specific to NET Core. Ideally we should have HttpClient or its replacement compiling under NET Standard.
Nothing in the docs show a minimal example. There are different levels of minimal:
- Using the webhost/generic host
- With or without a startup class (the generic host has better startup performance as part of 3.0)
- Wire up the server instance yourself without a host.
The latter isn’t pretty but we could spend some cycles making a nicer API to get to the pre request callback if that was a concern.
Hello @davidfowl ! If I understood correctly then option 2 is what HttpListener is for.
Right now, if you have HttpListener, the Portability Analysis tool tells you that you're good to go.
Then you realize that not really -- the .NET Core implementation doesn't support HTTPS which is a reasonable requirement to put a system in production.
Then you look up the issue on why HTTPS isn't supported, and it's because there's this thread that says that HttpListener should be abandoned altogether.
So there's really a disconnect between the happy land of the portability analyzer that tells you you're ready to jump, and the reality that HttpListener in NET Core compiles yet isn't really implemented.
The latter isn’t pretty but we could spend some cycles making a nicer API to get to the pre request callback if that was a concern.
Suggestion:
Spend those cycles on an API that mimics the .NET HttpListner API. The API does not have to serve as a foundational API for upper ASP.NET layers, but rather be considered an optional slim "Adapter" class aimed for developers who feel comfortable with that API definition.
How does that sound?
Exposing the HttpListener API from Kestrel IMO doesn’t make any sense. It would make more sense to just “fix” HttpListener but to make it production ready would be a wasted effort (wasting cycles fixing hard networking and security issues I’m not places etc). Now you could argue that Kestrel and HttpListener could share the same underlying core but there’s tons of code in the upper layers that actually handle the protocol control flow that wouldn’t be shared. HttpListener is also a pull model and ASP.NET Core exposes a push model for handling requests.
There are other layering concerns with replatting one on the other (like namespace and assembly dependencies). Even the core of kestrel is pluggable and things like TLS are plugged in via this extensibility (also multiple transports) and I’m not sure there’s an appetite to push this down into the lower levels.
Layering the other way (building Kestrel on top of HttpListener) would be wasteful layering wise and it’s already possible plug server implementations into ASP.NET Core wholesale so this would be pointless IMO.
All that said I’m not sure what the best call is here because we don’t have true replacement that’s built into the “core” shared framework that is a “production ready” (whatever that means here) server. That’s a bigger strategic decision that needs to be discussed.
Exposing the HttpListener API from Kestrel IMO doesn’t make any sense.
Why?
It would make more sense to just “fix” HttpListener
Why?
Now you could argue that Kestrel and HttpListener could share the same underlying core but there’s tons of code in the upper layers that actually handle the protocol control flow that wouldn’t be shared.
The .NET HttpListener class does not implement any protocol control flow. Thats handeled by lower level Windows components which HttpListener “wraps”
HttpListener is also a pull model and ASP.NET Core exposes a push model for handling requests.
What is Kestrel “Push model” ?
There are other layering concerns with replatting one on the other (like namespace and assembly dependencies). Even the core of kestrel is pluggable and things like TLS are plugged in via this extensibility (also multiple transports) and I’m not sure there’s an appetite to push this down into the lower levels.
How do these concerns impact a user facing HttpListener API? What dependecies exist?
Why
It would be a performance regression to use another API which has a different header and HttpContext structure.
Why
If this is about API porting then HttpListener is compatible with what was there on .NET Framework.
The .NET HttpListener class does not implement any protocol control flow. Thats handeled by lower level Windows components which HttpListener “wraps”
Sure it is. On Linux the entire protocol is implemented. It’s uninteresting to discuss the windows http.sys implementation as we would never base our implementation on that since it is tied to the OS and requires a new OS to get fixes and features. It’s also inconsistent with the Linux behavior (see prefix registration and HTTPS). There’s a fork of HttpListener in ASP.NET Core called HttpSysServer that doesn’t pretend to be an abstraction. We also don’t pay for another abstraction as we gutted it and bind directly to the native http abstractions to avoid the indirection.
So yes the entire control flow for both HTTP/1 and HTTP/2 needs to be implemented in managed code.
What is Kestrel “Push model” ?
With kestrel and ASP.NET Core in general you register a callback (usually a piece of middleware) that gets invoked per request. Unlike HttpListener the caller is responsible for implementating the accept loop (PS accepting on connection at a time may not be optimal).
How do these concerns impact a user facing HttpListener API? What dependecies exist?
Not sure what you mean. The entire discussion is about supporting the existing API, not about a replacement right?
The entire discussion is about supporting the existing API, not about a replacement right?
I personally would be open to change my code as long as we get to a new standard that works everywhere. The class doesn't need to be HttpListener, it could even be a higher abstraction class that already receives requests and dispatches async calls to request controllers.
But in any case, it needs to (1) compile in NET Standard, (2) implement the HTTP protocol, (3) allow to read the header and content of requests, (4) allow to override and send custom response headers and write the content of the response.
What I would NOT want is a web server that reads the URL and expects to find files on the disk or programs with a matching name, or that requires the user to install IIS/Apache, or that I need to change the project type to an ASP.NET flavor, etc.