xh icon indicating copy to clipboard operation
xh copied to clipboard

Allow to redirect parts of output to STDERR

Open andrew-grechkin opened this issue 1 month ago • 10 comments

For debug purpose, especially when using xh in a pipeline or result needs processing, it's quite nice to be able to collect headers (and request body too).

Currently this is not possible because all output assumes STDOUT.

One of the solutions would be to give ability to control (--stderr-print or --print-stderr) where separate parts are printed like:

if res=$(xhs jsonplaceholder.typicode.com/users --stderr-print=HBh 2>debug.out); then
    : # do something with res if everything is OK
else
   errno="$?"
   echo 'error occurred:' >&2
   cat debug.out >&2
   echo "$res" >&2
   exit "$errno"
fi

andrew-grechkin avatar Dec 05 '25 23:12 andrew-grechkin

It's possible to do this with the -d/--download flag:

$ xh -d https://example.org > /dev/null
HTTP/2.0 200 OK
accept-ranges: bytes
alt-svc: h3=":443"; ma=93600
cache-control: max-age=86000
content-length: 513
content-type: text/html
date: Sat, 06 Dec 2025 04:54:48 GMT
etag: "bc2473a18e003bdb249eba5ce893033f:1760028122.592274"
last-modified: Thu, 09 Oct 2025 16:42:02 GMT

Downloading 513 B to "<stdout>"
Done. 513 B in 0.00040s (1.22 MiB/s)

But that isn't obvious. At minimum we could document it better.

I'm not sure about a whole --print-stderr system... then you would be able to send the same information to both streams (IIRC this would require a bunch of refactoring for response bodies) and the --pretty option might have to be duplicated as well. I expect that most use cases are covered by sending the response body to stdout and everything else to stderr.

blyxxyz avatar Dec 06 '25 05:12 blyxxyz

Thanks for the answer. This definitely can help (in some cases) but it would still be a workaround.

For example this approach doesn't work not for HEAD nor for OPTIONS requests - I don't have anything at all on STDOUT if --download is used even though I would expect response headers there. Plus it writes meta in this case and this is not configurable.

I understand your concerns about refactoring but I think there is a solution to that:

just make sure stderr has higher priority if it's set.

for execution like this xh localhost --print=hb --stderr-print=HBh response headers should still go to stderr (because it always win)

I think this approach will not require any refactoring at all and as simple as adding command line option

andrew-grechkin avatar Dec 06 '25 07:12 andrew-grechkin

Another option is to extend --print/--history-print so they support --print=H:stderr,b:stdout (inspired by --format-options) in addition to --print=Hb.

Edit: Here is the closest httpie issue I could find https://github.com/httpie/cli/issues/1015#issuecomment-763625365

ducaale avatar Dec 06 '25 09:12 ducaale

Right, the headers... If you capture those, do you intend to parse them? Our format is intended for humans, so maybe it'd be better to have something more structured, like JSON output you can run through jq? That feels like a more principled solution for separating them than different streams.

e.g. | jq -r '.response.meta.headers["content-length"]'

(Though JSON isn't ideal. We'd have to be careful about non-Unicode headers, repeated headers, and case-insensitivity.)

blyxxyz avatar Dec 06 '25 10:12 blyxxyz

I like this a lot because it's backward compatible, gives full control and shouldn't be hard to implement (at least my experience tells me for such change there shouldn't be any refactoring required)

andrew-grechkin avatar Dec 06 '25 11:12 andrew-grechkin

@blyxxyz Parsing headers is fine, it's not hard to do (one jq command is enough to do that)

The biggest problem is to receive only headers when you need them or only body when you need that and redirect everything else to another output

andrew-grechkin avatar Dec 06 '25 11:12 andrew-grechkin

My thought was that JSON output would let you capture everything through one stream and then separate out the parts you care about afterward.

blyxxyz avatar Dec 06 '25 11:12 blyxxyz

My thought was that JSON output would let you capture everything through one stream and then separate out the parts you care about afterward.

I think this is quite hard to do if you would target generic approach.

This would normally work if body is a json. So app will output response headers as a json encoded object in one line and then response body as a json encoded line (object, array or something else doen't matter). This is easily consumable. If body is not a json - this becomes questionable because consumer will need to split the output somehow. Expect first line as headers and the rest of body? I'm not sure this is nice interface.

But I agree with you 100% ability to have headers as a json is quite nice feature to have.

I do think approach proposed above as --print=H:stderr,b:stdout is quite good and enough to cover all cases.

It gives full control. It even can be extended like --print=HB:3,h:4,b:stdout,m:stderr and just print data to the provided file handles, inherited from the shell.

This would be absolute killer feature, it's generic approach and it opens doors to all types of integration xh into shell pipelines.

andrew-grechkin avatar Dec 06 '25 14:12 andrew-grechkin

For reference, httpie has a feature request to output request and response as JSON https://github.com/httpie/cli/issues/1007

ducaale avatar Dec 06 '25 15:12 ducaale

This would normally work if body is a json. So app will output response headers as a json encoded object in one line and then response body as a json encoded line (object, array or something else doen't matter).

I was thinking it could output a JSON object like this:

{
    "request": {
        "method": "GET",
        "headers": {"..."},
        "..."
    },
    "response": {
        "status": 200,
        "headers": {"..."},
        "body": "{\"foo\": \"bar\"}"
    }
}

Of course this means that a JSON body will be doubly-encoded (like in the example), and it again is problematic with binary data...

But excepting those kinds of problems it would give fuller control than simply choosing what to send to stdout and what to stderr. And you could easily split it into more than two parts without resorting to file descriptor tricks.

(If we ever do this we should study hurl and HAR.)

blyxxyz avatar Dec 06 '25 17:12 blyxxyz