har-spec
har-spec copied to clipboard
ahmadnassri
Postdata changes
A note on timings.redirect
Since timings.tcpConnect = timings.connect - timings.ssl, and timings.total = entry.time, they may be omitted from the new spec in the interest of minimizing duplication, but I would like to make an argument for the new timings.redirect. There is an existing mechanism for grouping entries together called "pages" which represents a single request (e.g. of an HTML file), and after parsing that file, a collection of URLs are then requested from that file (e.g. fonts, images, scripts, styles, etc.) which are logically distinct. The concept I'm trying to get across is something like this, but when there is only a single HTTP resource being received, but when there are multiple requests and responses in order to receive it. On the one hand, this maps directly to the concept of "pages", but on the other hand, it is distinct, because "pages" group multiple distinct HTTP resources together. The way that python-requests handles this is via response.history[] property (in HAR, it would be entry.history[] but I have not added it to this proposal), which is a list of request-response pairs, or entries, which explain the redirection process, how the client got to the last response. In order explain why this might be necessary, I have created an example to keep in mind. And although I'm using HTTP/2 pseudo-header names, this kind of process could also happen over HTTP or HTTPS.
The most pathological "single entry" HTTP example I can think of:
Client sends Proxy (proxy.example.com) a Request with no body, but with:
-
:method: CONNECT -
:path: http://www.example.com/
Proxy sends Client a Response with:
-
:status: 407 -
proxy-authenticate: Basic
Client parses the user's $HOME/.netrc file to obtain credentials.
Client sends Proxy a Request with no body, but with:
-
:method: CONNECT -
:path: http://www.example.com/ -
proxy-authorization: Basic dXNlcjpwYXNz
Proxy (proxy.example.com) connects to Server (www.example.com) on behalf of Client. It is important to note that the traditional response happens on the TCP level, not the HTTP level, so technically speaking, there is no HTTP response.
Client sends Server (www.example.com) a Request with no body, but with:
-
:method: POST -
:path: / -
... headers ... -
expect: 100-continue
Server sends Client a Response with
-
:status: 403 -
www-authenticate: Basic
Client parses the user's $HOME/.netrc file to obtain credentials.
Client sends Server a Request with no body, but with:
-
:method: POST -
:path: / -
authorization: Basic dXNlcjpwYXNz -
... headers ... -
expect: 100-continue
Server sends Client a Response with
-
:status: 307 -
location: /upload
Client sends Server a Request with no body, but with:
-
:method: POST -
:path: /upload -
authorization: Basic dXNlcjpwYXNz -
... headers ... -
expect: 100-continue
Server sends Client a Response with
-
:status: 100
Client sends Server a Request without the expect header, and with:
-
:method: POST -
:path: /upload -
authorization: Basic dXNlcjpwYXNz -
... headers ... -
\r\n\r\n -
... body ...
Server sends Client a Response with
-
:status: 202
In the grand scheme of things, this might be logged as a single "POST" to "/" (entry.request.url) where the entry.redirectURL is set to "/upload", and all the time spent doing all of this is stored in timings.redirect. On the other hand, it is also possible that this is logged as 6 separate entries, which would require that the successful Proxy "CONNECT" response is null on the 2nd entry.
In order to reproduce this transaction faithfully, without using history[] as discussed above, would be:
{
"log": {
"version": "1.3",
"creator": {
"name": "com.example.creator",
"version": "0.0"
},
"pages": [],
"entries": [
{
"startedDateTime": "2016-01-01T01:01:01.000Z",
"time": 10,
"request": {
"method": "CONNECT",
"url": "http://www.example.com/",
"httpVersion": "HTTP/1.1",
"cookies": [],
"headers": [
{
"name": "connection",
"value": "keep-alive"
}
],
"queryString": [],
"headersSize": -1,
"bodySize": -1
},
"response": {
"status": 407,
"statusText": "Proxy Authentication Required",
"httpVersion": "HTTP/1.1",
"cookies": [],
"headers": [
{
"name": "proxy-authenticate",
"value": "Basic"
}
],
"content": {
"mimeType": "",
"size": 0
},
"redirectURL": "",
"headersSize": -1,
"bodySize": -1
},
"cache": {},
"timings": {
"send": -1,
"wait": 10,
"receive": -1
}
},
{
"startedDateTime": "2016-01-01T01:01:01.010Z",
"time": 10,
"request": {
"method": "CONNECT",
"url": "http://www.example.com/",
"httpVersion": "HTTP/1.1",
"cookies": [],
"headers": [
{
"name": "connection",
"value": "keep-alive"
},
{
"name": "proxy-authorization",
"value": "Basic dXNlcjpwYXNz"
}
],
"queryString": [],
"headersSize": -1,
"bodySize": -1
},
"response": {},
"cache": {},
"timings": {
"send": -1,
"wait": 10,
"receive": -1
}
},
{
"startedDateTime": "2016-01-01T01:01:01.020Z",
"time": 10,
"request": {
"method": "POST",
"url": "http://www.example.com/",
"httpVersion": "HTTP/1.1",
"cookies": [],
"headers": [
{
"name": "connection",
"value": "keep-alive"
},
{
"name": "expect",
"value": "100-continue"
}
],
"queryString": [],
"headersSize": -1,
"bodySize": -1
},
"response": {
"status": 403,
"statusText": "Forbidden",
"httpVersion": "HTTP/1.1",
"cookies": [],
"headers": [
{
"name": "www-authenticate",
"value": "Basic"
}
],
"content": {
"mimeType": "",
"size": 0
},
"redirectURL": "",
"headersSize": -1,
"bodySize": -1
},
"cache": {},
"timings": {
"send": -1,
"wait": 10,
"receive": -1
}
},
{
"startedDateTime": "2016-01-01T01:01:01.030Z",
"time": 10,
"request": {
"method": "POST",
"url": "http://www.example.com/",
"httpVersion": "HTTP/1.1",
"cookies": [],
"headers": [
{
"name": "connection",
"value": "keep-alive"
},
{
"name": "expect",
"value": "100-continue"
},
{
"name": "authorization",
"value": "Basic dXNlcjpwYXNz"
}
],
"queryString": [],
"headersSize": -1,
"bodySize": -1
},
"response": {
"status": 307,
"statusText": "Temporary Redirect",
"httpVersion": "HTTP/1.1",
"cookies": [],
"headers": [
{
"name": "location",
"value": "/upload"
}
],
"content": {
"mimeType": "",
"size": 0
},
"redirectURL": "",
"headersSize": -1,
"bodySize": -1
},
"cache": {},
"timings": {
"send": -1,
"wait": 10,
"receive": -1
}
},
{
"startedDateTime": "2016-01-01T01:01:01.040Z",
"time": 10,
"request": {
"method": "POST",
"url": "http://www.example.com/upload",
"httpVersion": "HTTP/1.1",
"cookies": [],
"headers": [
{
"name": "connection",
"value": "keep-alive"
},
{
"name": "expect",
"value": "100-continue"
},
{
"name": "authorization",
"value": "Basic dXNlcjpwYXNz"
}
],
"queryString": [],
"headersSize": -1,
"bodySize": -1
},
"response": {
"status": 100,
"statusText": "Continue",
"httpVersion": "HTTP/1.1",
"cookies": [],
"headers": [],
"content": {
"mimeType": "",
"size": 0
},
"redirectURL": "",
"headersSize": -1,
"bodySize": -1
},
"cache": {},
"timings": {
"send": -1,
"wait": 10,
"receive": -1
}
},
{
"startedDateTime": "2016-01-01T01:01:01.050Z",
"time": 50,
"request": {
"method": "POST",
"url": "http://www.example.com/upload",
"httpVersion": "HTTP/1.1",
"cookies": [],
"headers": [
{
"name": "connection",
"value": "keep-alive"
},
{
"name": "authorization",
"value": "Basic dXNlcjpwYXNz"
}
],
"queryString": [],
"postData": {
"text": "Hello, world"
},
"headersSize": -1,
"bodySize": -1
},
"response": {
"status": 202,
"statusText": "Accepted",
"httpVersion": "HTTP/1.1",
"cookies": [],
"headers": [
],
"content": {
"mimeType": "",
"size": 0
},
"redirectURL": "",
"headersSize": -1,
"bodySize": -1
},
"cache": {},
"timings": {
"send": -1,
"wait": 50,
"receive": -1
}
}
]
}
}
but if the client doesn't have access to all of these steps along the way, then it might be collapsed into a single entry:
{
"log": {
"version": "1.3",
"creator": {
"name": "com.example.creator",
"version": "0.0"
},
"pages": [],
"entries": [
{
"startedDateTime": "2016-01-01T01:01:01.000Z",
"time": 100,
"request": {
"method": "POST",
"url": "http://www.example.com/",
"httpVersion": "HTTP/1.1",
"cookies": [],
"headers": [
{
"name": "connection",
"value": "keep-alive"
},
{
"name": "expect",
"value": "100-continue"
},
{
"name": "authorization",
"value": "Basic dXNlcjpwYXNz"
},
{
"name": "proxy-authorization",
"value": "Basic dXNlcjpwYXNz"
}
],
"queryString": [],
"headersSize": -1,
"bodySize": -1
},
"response": {
"status": 202,
"statusText": "Accepted",
"httpVersion": "HTTP/1.1",
"cookies": [],
"headers": [
],
"content": {
"mimeType": "",
"size": 0
},
"redirectURL": "/upload",
"headersSize": -1,
"bodySize": -1
},
"cache": {},
"timings": {
"redirect": 50,
"send": -1,
"wait": 50,
"receive": -1
}
}
]
}
}
However, to combine the best of both worlds, using the entry.history[] extension discussed above, then we would put all entries of the first example (except the last entry, which is the same as the second example), in Example2.log.entries[0].history = Example1.log.entries[0...5]