Character lost in HTTP post response body.

[PhyX-Meow]

Versions:

• Julia 1.11.0-beta1
• HTTP.jl 1.10.5
• MbedTLS.jl 1.1.9

I have a HTTP server running locally, but querying the server using HTTP.post give me this:

In [9]: HTTP.post("http://127.0.0.1:5700/get_login_info", ["Content-Type" => "application/json"], body = "{}")
HTTP.Messages.Response:
"""
HTTP/1.1 200 OK
Content-Type: application/json
Server: Microsoft-NetCore/2.0
Date: Thu, 11 Apr 2024 11:54:50 GMT
Content-Length: 113

"status":"ok","retcode":0,"data":{"user_id":3785797225,"nickname":"\u609F\u7406\u80DA\u80CE\u7403"},"echo":null}"""

The response body is missing the first character, in my case, the beginning { in json string is lost. Correct result should look like this curl result:

❯ curl http://127.0.0.1:5700/get_login_info
{"status":"ok","retcode":0,"data":{"user_id":3785797225,"nickname":"\u609F\u7406\u80DA\u80CE\u7403"},"echo":null}

The issue happens today when I updated julia version.

[PhyX-Meow]

This is not the only issue with julia 1.11-beta, running pkg>test HTTP results in a very long list of errors.

[marcelo-g-simas]

I am seeing a similar issue when processing the body of a POST request, see issue above on the Oxygen.jl repo for details.

[KristofferC]

Does anyone have a simple repro for this?

[marcelo-g-simas]

Server

using Oxygen
using HTTP
using StructTypes

struct Login
    user_name::String
end

@post "/login" function(req::HTTP.Request)
    @info req.body
    login_data = json(req, Login)
    @info login_data
    return "hello world!"
end

# start the web server
serve()

Client

using HTTP
HTTP.post("http://127.0.0.1:8080/login", body = "{ \"user_name\": \"test\" }")

[marcelo-g-simas]

You can see what the exception that Client call generates on the Server while running on Julia 1.11 here -> https://github.com/OxygenFramework/Oxygen.jl/issues/186#issue-2240239311

[nhz2]

I think the issue is from https://github.com/JuliaWeb/HTTP.jl/blob/efd74ea39a035bd849d1d225ed54ad00fae2a0d8/src/Messages.jl#L533 which calls: https://github.com/JuliaWeb/HTTP.jl/blob/efd74ea39a035bd849d1d225ed54ad00fae2a0d8/src/IOExtras.jl#L115-L124

Since https://github.com/JuliaLang/julia/pull/53896

calling String on a view of a Memory and then trying to use the Memory is unsafe.

Here is a example:

julia> a = Memory{UInt8}(undef, 10);

julia> fill!(a, 0x01);

julia> String(view(a, 1:2))
"\x01\x01"

julia> a
0-element Memory{UInt8}

[KristofferC]

Yeah, I think all the uses of the IOBuffer internals has to be removed.

https://github.com/JuliaWeb/HTTP.jl/pull/1145 and https://github.com/JuliaWeb/HTTP.jl/pull/1147 has started that work.

[vtjnash]

In theory, a normal readbyte call there could allocate a StringVector more directly here, avoiding the copy later to make a String from this, and avoid relying on implementation details too

[quinnj]

I agree we need to refactor a bunch of the internals to not abuse IOBuffer; either move everything to be Memory-based (my preference since we'll have full control and want to be efficient as possible w/ # of allocations), or just use IOBuffer's properly. In any case, I think https://github.com/JuliaWeb/HTTP.jl/pull/1170 is at least a stop-gap for the immediate issue posted here.