Feature request: output full HTTP request and response in a JSON structure

[hughpv]

I would love to capture the full HTTP request and response in a JSON structure so I can extract the various parts programmatically without parsing the plaintext output.

For example:

http --output-full-json ...

{
  "request": {
    "headers": "Accept: application/json\r\nContent-Type: application/json",
    "body": "<request body here, escaped/encoded as necessary to work as a string value>"
  },
  "response": {
    "headers": "201 Created\r\nContent-Type: application/json\r\nContent-Length: 42",
    "body": "<response body here, escaped/encoded as necessary to work as a string value>"
  }
}

[jkbrzt]

Thanks for the suggestion, @hughpv. I’ve been considering this for a while.

Initial thoughts

• Output options should be respected.
• With --all, an Array of exchanges is printed instead of a single object.
• What about binary bodies? Apply base64 encoding like httpbin.org? Let user choose to skip/base64-encode?
• JSON body, when valid, could be embedded into the structure directly (i.e., not as string).
• A structured object for headers would be nice, but we should support multiple headers of the same name somehow.
• UI — --output-format which could be raw (default) + json + potentially more in the future.
• User can always pipe to jq but consider supporting simple selectors directly. Something like --select=response.headers.content-type. These would work on the JSON structure so when used, then --output-format=json would be implied. Need to consider the multi-exchange scenario (where index might be needed).

Hypothetical example

$ http -v --output-format=json  example.org

// Exchange
{
  "request": {
    "headers": {
      "Foo": "Bar"
    },
    "body": "…"
  },
  "response": {
    "headers": {
      "Foo": "Bar"
    },
    "body": {
      "result": "ok"
    }
  }
}

[hughpv]

Awesome ideas; love them all. Let me just expand a little on my use case --

I have it in mind to build a test suite that, in addition to testing functionality, also captures output for use in documentation, e.g.:

= Example Request
----
include:request.http[]
----

= Example Response
----
include:response.http[]
----

... where request.http and response.http would be built from the httpie output.

Thus what I'm looking for is essentially a "verbatim" or "raw" option. All the other stuff makes sense but the raw output was the main idea I came here with.

Thanks!

[jkbrzt]

I see, so for your use case, the structured headers would be a complication, because you would have to manually un-structure them for the docs?

[hughpv]

Correct. :) But I could definitely see where some users would find structured headers quite useful.

[Ousret]

Hi,

I might have something to help with headers. https://github.com/Ousret/kiss-headers Should help with a potential JSON repr or pretty output.

[ahmadnassri]

HAR would be a great format to use for this

[morgen-peschke]

This would be extremely helpful for some testing use cases (I've currently got a really ugly shell function that mimics this with vanilla curl).

Some thoughts on the format:

• It's probably worth exposing the HTTP code as a number (makes success checks really easy)
• Wrapping the header values in an array may provide a low-friction way to deal with multiple array values. The default case only requires adding [0], which is a pretty low cost.
• It may be worth treating it as an opt-in sort of thing, as this format is more likely to be useful in scripts than interactively, so being a little more explicit is probably not going to be an issue.

Borrowing language from git, this could be an alternative to --print: --porcelain.

Examples

Raw

Command:

--porcelain=req.headers.raw,req.body.text,req.method,req.uri.raw,resp.headers.raw,resp.body.text,resp.code,resp.status

Output:

{
  "request": {
    "uri": {
      "raw": "http://example.com:8081/test?foo=bar"
    },
    "method": "POST",
    "headers": {
      "raw": "Accept: application/json\r\nContent-Type: application/json"
    },
    "body": {
      "text": "Lorem ipsum dolor sit amet"
    }
  },
  "response": {
    "code": 201,
    "status": "Created",
    "headers": {
      "raw": "Content-Type: application/json\r\nContent-Length: 42"
    },
    "body": {
      "text": "consectetur adipiscing elit"
    }
  }
}

Structured/Parsed

Command:

--porcelain=req.headers.json,req.body.base64,req.method,req.uri.json,resp.headers.text,resp.body.json,resp.code

Output:

{
  "request": {
    "uri": {
      "json": {
        "scheme": "http",
        "host": "example.com",
        "port": 8081,
        "path": "/test",
        "query": {
          "foo": [
            "bar"
          ]
        }
      }
    },
    "method": "POST",
    "headers": {
      "json": {
        "Accept": [
          "application/json"
        ],
        "Content-Type": [
          "application/json"
        ]
      }
    },
    "body": {
      "base64": "TG9yZW0gaXBzdW0gZG9sb3Igc2l0IGFtZXQ="
    }
  },
  "response": {
    "code": 201,
    "headers": {
      "text": [
        "Content-Type: application/json",
        "Content-Length: 42"
      ]
    },
    "body": {
      "json": "\"consectetur adipiscing elit\""
    }
  }
}

Mixed

Command:

--porcelain=req.method,req.uri.raw,resp.headers.raw,resp.headers.json,resp.body.base64,resp.code

Output:

{
  "request": {
    "uri": {
      "raw": "http://example.com:8081/test?foo=bar"
    },
    "method": "POST"
  },
  "response": {
    "code": 201,
    "headers": {
      "raw": "Accept: application/json\r\nContent-Type: application/json",
      "json": {
        "Accept": [
          "application/json"
        ],
        "Content-Type": [
          "application/json"
        ]
      }
    },
    "body": {
      "base64": "Y29uc2VjdGV0dXIgYWRpcGlzY2luZyBlbGl0"
    }
  }
}

Sorry for the edits, I kept hitting the key for "submit" instead of "newline+indent" 🤦🏻