swift-nio
swift-nio copied to clipboard
apple
NTRIP support
Reinterprets NTRIP SOURCETABLE and ICY responses as HTTP/1.1 responses.
Motivation:
My goal is to use these changes to develop an NTRIP client (like a web browser) for iOS. These changes are being contributed in the hopes that other NTRIP developers use them to create more stable, safe and reliable NTRIP applications.
In the past I would accomplish this by manually encoding and decoding NTRIP requests and responses. This was error-prone, difficult to maintain, often lacked support (such as redirects, proxies and TLS), and was reinventing what many HTTP clients had already done.
What is NTRIP
Ntrip is a generic, stateless protocol based on the Hypertext Transfer Protocol HTTP/1.1.
It is intended for streaming real-time, high-accuracy positioning data over the internet.
At this time, no Swift-based HTTP package supports NTRIP. A similar pull request was proposed and accepted for OkHTTP (https://github.com/square/okhttp/pull/7320), which I am using on Android.
The next major version of NTRIP (revision 2) is fully HTTP-compliant but is not yet widely supported by a majority of NTRIP servers (called casters).
Modifications:
The http-parser (CNIOHTTPParser) main loop was modified to interpret messages starting with SOURCETABLE or ICY as HTTP responses with a major and minor version of 1.1.
Tests where added for both SOURCETABLE and ICY.
Future changes
Node.js, the apparent authors of the http-parser, no longer maintain it. It will no longer receive updates and the included update_and_patch_http_parser.sh script should be redirected to a forked repository, or modified to avoid overwriting these changes.
A pull request (https://github.com/apple/swift-nio/pull/2261) is needed to support a wider range of older NTRIP casters. Some casters produce responses that are not fully HTTP compliant. Most of these errors are tolerated. One error results in the body of the response being interpreted as a header, causing the execution to never produce a result or to produce an incorrect result (infinite loop, timeout, or invalid header and cut off body).
Result:
This results in NTRIP responses being properly interpreted as HTTP responses in tests. Addition improvements can be made directly by users (e.g. NTRIP suggests setting the Connection header to "close").
In this instance I think the place we'd like to get to is to do a two phase migration. The first is to migrate from http_parser to llhttp. The second step is to migrate from there to a Swift version. I appreciate this is quite a lot of work, but I'd like to get a sense of how much of it you feel like you're willing to tackle so we can balance the ideal evolution of swift-nio with your needs here.
The changes I need are the 2 HTTP version headers (SOURCETABLE and ICY) and the modification noted in https://github.com/apple/swift-nio/pull/2261#pullrequestreview-1095472261 (already in active use in the for-profit project). I may need additional error-tolerance towards HTTP requests and responses (custom headers, secondary status lines, bi-directional streaming bodies). On a broader scale, I need an abstraction of ntrip (revisions 1 and 2) and http that works on iOS 12 and above.
I'm not entirely sure what is being asked. The contribution was originally designed as part of a paid-project. I am interested in contributing further in my spare time, especially to replace the previous http_parser currently in use. I believe I am sufficiently proficient in TypeScript and C to do so, alone or as part of a team. I may not be able to contribute as much on the Swift side due to my lack of experience (started using Swift 2 weeks ago).
So the core problem is that I don't want to carry a patch to http_parser. Given that the http_parser upstream is not maintained any longer, it's going to be difficult for you to upstream it there, so we'd need to migrate off http_parser. The first step is to move to llhttp, as you note, and then the second step will be to move to a custom Swift version. There's no timeline for when we'd do the second step, but I cracked out the first step over the weekend: #2263.
In this instance, I think the next step for you is probably to try to get llhttp to accept the changes you need in their code, and then we can go from there. If they aren't interested, we may need to accelerate the move towards having a custom HTTP parser of our own in Swift, which will mean finding someone who wants to make that contribution.
