varnishcache
Prepare Varnish-Cache for the age of AI
... where of course AI stands for Asynchronous Iterators ;)
On this PR
This PR might be more extensive than others, and I would like to ask reviewers to lean back and take some time for it. Thank you in advance to anyone who invests their time into looking at this.
The PR description has been written to give an overview; individual commit messages have more details.
While the code proposed here works, and I am already building on top of it, I am deliberately marking this PR as a draft to make clear that I am very interested in feedback and comments before committing to a particular interface. Also I expect sensible changes to arrive still, but the fundamentals are ripe, IMHO.
Intro
This PR suggests a new API to enable use of asynchronous I/O on the delivery side of Varnish-Cache. The design has been highly influenced by io_uring, because my prior experience has been that creating compatibility with it is fairly easy: The io_uring interface is extremely simple in that completions are identified by a single uint64_t value, so once calling code is prepared to work with this limited interface, adding other implementations is relatively easy.
Advantages of asynchronous I/O
In my mind, efficient asynchronous I/O interfaces primarily provides two main benefits:
a) using less threads
b) significantly lower context switching overhead when many I/O requests can be combined into a single system call (or, in the case of io_uring, optionally none at all)
a) has long been achieved with the traditional select() (poll() / epoll() / kqueue()) event loop model, which Varnish-Cache uses for the waiter facility. This technique saves on threads, but each I/O still requires >1 system call per I/O. As long as we used HTTP1 with TCP, this overhead was not too bad, because we could simply make our I/O system calls very big (using writev()). But already with HTTP2, handing large batches of data to the kernel becomes problematic, because the (questionable) benefits of the protocol require reacting to client events with lowest latency. And with HTTP3, we definitely need system calls to handle lots of small UDP datagrams, unless we want to map hardware into user address space and talk directly to it.
So b) becomes much more important now: More than ever, we will need to be able to batch I/O requests into as small a number of system calls as possible with a tight feedback loop to the client. If we want to stay efficient and not push too much data for each client at once, consequently we also need to handle many clients from a central instance (a thread or a few threads).
Or, summarizing even more, we need to use an event loop-ish architeture for client I/O.
Existing interfaces vs. asynchronous I/O
Our existing ObjIterate() interface calls into the storage engine with a delivery function to execute on every extent (pointer + length) of body data. Iteration can block at any point: The storage engine may issue blocking I/O, as certainly will the delivery function while sending data to the client.
For asynchronous I/O, we need to turn this inside out: Firstly, our iteration state can not live on the stack as it currently does, we need a data structure containing all we need to know about the iteration such that we can come back later and ask the storage engine for more data. Secondly, we need a guarantee from the storage engine to never block. And, following from that, we need some notification mechanism allowing the storage engine to tell client I/O code that it now has more data available. A special case of the last point is streaming a busy object, where it is not just the storage engine that blocks, but rather creation of the cache object itself.
Overview
This series of commits starts with the last aspect: ObjVAIGetExtend() in the second commit checks for more streaming data and, if none is available, registers a callback, allowing for asynchronous notification.
The third commit contains the proposed new API, which follows the idea that the storage engine hands out "leases" on segments of data to callers, which can then use them until they return the leases - where the storage engine might require leases to be returned before handing out new ones.
With an array of leases at hand, the caller can then do its asynchronous thing, also returning leases in an asynchronous fashion.
The last commit implements the new API for simple storage and reimplements the existing simple storage iterator using the new API as a PoC.
Performance
Performance of malloc storage with current varnish-cache head 508306fd06341c6e960353680245b373a8f406e9 has been measured against the reimplementation using the new API. The test has been set up such that the more relevant case of streaming data from small-ish extents is tested with varnish-cache as its own backend: A cache object is created from 64MB of random data, which is then streamed in pass mode by using a request from Varnish-Cache against itself. The test has also been run with debug.chkcrc32 from #4208 and also for much longer periods.
Details are given below.
Typical results for wrk -t100 -c400 -d100 http://localhost:8080 on my laptop are:
baseline 508306fd06341c6e960353680245b373a8f406e9
Running 2m test @ http://localhost:8080/
100 threads and 400 connections
Thread Stats Avg Stdev Max +/- Stdev
Latency 1.68s 306.07ms 2.00s 79.02%
Req/Sec 0.60 1.80 20.00 92.79%
4872 requests in 1.70m, 305.13GB read
Socket errors: connect 0, read 263, write 0, timeout 4729
Requests/sec: 47.82
Transfer/sec: 2.99GB
with VAI reimplementation 125dc28478bda5f759e991f930dd0e59adf863f8
Running 2m test @ http://localhost:8080/
100 threads and 400 connections
Thread Stats Avg Stdev Max +/- Stdev
Latency 1.61s 348.34ms 2.00s 75.63%
Req/Sec 0.56 1.68 20.00 93.12%
4897 requests in 1.70m, 306.35GB read
Socket errors: connect 0, read 327, write 0, timeout 4700
Requests/sec: 48.04
Transfer/sec: 3.01GB
baseline 508306fd06341c6e960353680245b373a8f406e9 with debug.chkcrc32
Running 2m test @ http://localhost:8080/
100 threads and 400 connections
Thread Stats Avg Stdev Max +/- Stdev
Latency 0.00us 0.00us 0.00us -nan%
Req/Sec 0.56 1.76 10.00 93.36%
1065 requests in 1.67m, 75.61GB read
Socket errors: connect 0, read 0, write 0, timeout 1065
Requests/sec: 10.64
Transfer/sec: 773.51MB
with VAI reimplementation 125dc28478bda5f759e991f930dd0e59adf863f8 with debug.chkcrc32
Running 2m test @ http://localhost:8080/
100 threads and 400 connections
Thread Stats Avg Stdev Max +/- Stdev
Latency 846.94ms 0.00us 846.94ms 100.00%
Req/Sec 0.67 1.91 10.00 92.55%
1128 requests in 1.67m, 81.11GB read
Socket errors: connect 0, read 0, write 0, timeout 1127
Requests/sec: 11.27
Transfer/sec: 829.73MB
Performance Test details
VCL with varnishd start command line in a comment (code in vcl_deliver commented out for no-crc test)
vcl 4.1;
# VCL to benchmark passes.
# A single object is created from a file, then varnish makes pass requests
# against itself which are hits on that file.
#
# Uses linux abstract sockets, but can easily be adjusted to using UDS
#
# example start command line:
#
# /tmp/sbin/varnishd -f $PWD/slink/bench_pass/bench_pass.vcl -a :8080 -a self=@varnish -n /tmp/t -p fetch_chunksize=4k -F
import debug;
import std;
import blob;
import blobsynth;
backend none none;
backend self {
.path = "@varnish";
}
sub vcl_init {
new data = blob.blob(BASE64,
std.fileread("/home/slink/Devel/varnish-git/slash/slink/misc/64mb.b64"));
}
sub vcl_recv {
if (local.socket == "self") {
set req.url = "/";
return (hash);
} else {
return (pass);
}
}
sub vcl_backend_fetch {
if (local.socket == "self") {
set bereq.backend = none;
} else {
set bereq.backend = self;
}
return (fetch);
}
sub vcl_backend_error {
set beresp.status = 200;
blobsynth.synthetic(data.get());
set beresp.ttl = 1y;
return (deliver);
}
sub vcl_deliver {
debug.chkcrc32(2561873268, panic_unless_error);
set resp.filters += " debug.chkcrc32";
}
I have re-run the mini benchmark. Because wrk does not provide individual numbers, I added varnishlog to get the Timestamp:Resp, so my start-script is now this[^1]. The outcome has been given to ministat.
As apparently I had not made this clear in the initial note, the goal at this point is to show that a reimplementation of sml_iterator() based on the new AI iterator does not show a relevant performance regression. Relevant gains of the AI interface are only expect with an asynchronous IO implementation actually making use of it, which I do not have yet, that is work in progress.
So here is an unfiltered comparison of two runs with vai vs. master.
These are response times, so lower is better.
crc32 check enabled
$ ministat -s -C 6 *varnishlog_Timestamp*
x master_varnishlog_Timestamp:Resp.20241021_112327.d7540e548dc2771a91a1a842949bb82904e06032
+ vai_varnishlog_Timestamp:Resp.20241021_112035.058edd04dedbde25d3193580fc9a63aba3d0c602
+------------------------------------------------------------------------------+
| + |
| + |
| + |
| + |
| + |
| + |
| + |
| + |
| + |
| + |
| + |
| + |
| + |
| + |
| + |
| + |
| + |
| + |
| + |
| + |
| + |
| + |
| + |
| + |
| + |
| + |
| + |
| + |
| + |
| + |
| + |
| + |
| + |
| + |
| + |
| + |
| + |
| + |
| + |
| + |
| + |
| + |
| + |
| + |
| + |
| + |
| + |
| + |
| + |
| + |
| + |
| + |
| + |
| + |
| + |
| + |
| + |
| + |
| + |
| + |
| + |
| + |
| + |
| + |
| + |
| + |
| + |
| + |
| + |
| + |
| + |
| + |
| + |
| + |
| + |
| + |
| + |
| + |
| * |
| * |
| * |
| * |
| * |
| * |
| * |
| * |
| * |
| * |
| * |
| * |
| * |
| * |
| * |
| * |
| * |
| * |
| * |
| * |
| * |
| * |
| * |
| +* |
| +* |
| +* |
| +* |
| +* |
| +* |
| +* |
| +* |
| +* |
| +* |
| +* |
| +* |
| +* |
| +* |
| +* |
| +* |
| +* |
| +* |
| +* |
| +* |
| +* |
| +* |
| +* |
| +* |
| +* |
| +* |
| +* |
| +* |
| +* |
| +* |
| +* |
| +* |
| +* |
| +* |
| +* |
| +* |
| +* |
| +* |
| +* |
| +* |
| +* |
| +* |
| +* |
| +* |
| +* |
| +* |
| +* |
| +* |
| +* |
| +* |
| +* |
| +* |
| +* |
| +* |
| +* |
| +* |
| +* |
| +* |
| +* |
| +* |
| +* |
| +* |
| +* |
| +* |
| +* |
| +* |
| +* |
| +* |
| +* |
| +* |
| +* |
| +* |
| +* |
| +* |
| +* |
| +* |
| +* |
| +* |
| +* |
| +* |
| +* |
| +* |
| +* |
| +* |
| +* |
| +* |
| +* |
| +* |
| +* |
| +* |
| +* |
| +* |
| +* |
| +* |
| +* |
| +* |
| +* |
| +* |
| +* |
| +* |
| +* |
| +* |
| +* |
| +* |
| +* |
| +* |
| +* |
| +* |
| +* |
| +* |
| +* |
| +* |
| +* |
| +* |
| +* |
| +* |
| +* |
| +* |
| +* |
| +* |
| +* |
| +* |
| +* |
| +* |
| +* |
| +* |
| +* |
| +* |
| +* |
| +* |
| +* |
| +* |
| +* |
| +* |
| +* |
| +* |
| +* |
| +* |
| +* |
| +* |
| +* |
| +* |
| +* |
| +* |
| +* |
| +* |
| +* |
| +* |
| +* |
| +* |
| +* |
| +* |
| +* |
| +* |
| ** |
| ** |
| ** |
| ** |
| ** |
| ** |
| ** |
| ** |
| ** |
| ** |
| ** |
| ** |
| ** |
| ** |
| ** |
| ** |
| ** |
| ** |
| ** |
| ** |
| ** |
| ** |
| ** |
| ** |
| ** |
| ** |
| ** |
| ** |
| ** |
| ** |
| ** |
| ** |
| ** |
| ** |
| ** |
| ** |
| ** |
| ** |
| ** |
| ** |
| ** |
| ** |
| ** |
| ** |
| ** |
| ** |
| ** |
| ** |
| ** |
| ** |
| ** |
| ** |
| ** |
| ** |
| ** |
| ** |
| ** |
| ** |
| ** |
| ** |
| ** |
| ** |
| ** |
| ** |
| ** |
| ** |
| ** |
| ** |
| ** |
| ** |
| ** |
| ** |
| ** |
| ** |
| ** |
| ** |
| ** |
| ** |
| ** |
| ** |
| ** |
| ** |
| ** |
| ** |
| ** |
| ** |
| ** |
| ** |
| ** |
| ** |
| ** |
| ** |
| ** |
| ** |
| ** |
| ** |
| ** |
| ** |
| ** |
| ** |
| ** |
| ** |
| ** |
| ** |
| ** |
| ** |
| ** |
| ** |
| ** |
| ** |
| **x |
| **x |
| **x |
| **x |
| **x |
| **x |
| **x |
| **x |
| **x |
| **x |
| **x |
| **x |
| **x |
| **x |
| **x |
| **x |
| **x |
| **x |
| **x |
| **x |
| **x |
| **x |
| **x |
| **x |
| **x |
| **x |
| **x |
| **x |
| **x |
| **x |
| **x |
| **x |
| **x |
| **x |
| **x |
| **x |
| **x |
| **x |
| **x |
| **x |
| **x |
| **x |
| **x |
| **x |
| **x |
| **x |
| **x |
| **x |
| **x |
| **x |
| **x |
| **x |
| **x |
| **x |
| **x |
| **x |
| **x |
| **x |
| **x |
| **x |
| **x |
| **x |
| **x |
| **x |
| **x |
| **x |
| **x |
| **x |
| **x |
| **x |
| **x |
| **x |
| **x |
| **x |
| **x |
| **x |
| **x |
| **x |
| **x |
| **x |
| **x |
| **x |
| **x |
| **x |
| **x |
| **x |
| **x |
| **x |
| **x |
| **x |
| **x |
| **x |
| **x |
| **x |
| **x |
| **x |
| **x |
| **x |
| **x |
| **x |
| **x |
| **x |
| **x |
| *** |
| *** |
| *** |
| *** |
| *** |
| *** |
| *** |
| *** |
| *** |
| *** |
| *** |
| *** |
| *** |
| *** |
| *** |
| *** |
| *** |
| *** |
| *** |
| *** |
| *** |
| *** |
| *** |
| *** |
| *** |
| *** |
| *** |
| *** |
| *** |
| *** |
| *** |
| *** |
| *** |
| *** |
| *** |
| *** |
| *** |
| *** |
| *** |
| *** |
| *** |
| *** |
| *** |
| *** |
| *** |
| *** |
| *** |
| *** |
| *** |
| *** |
| *** |
| *** |
| *** |
| *** |
| *** |
| *** |
| *** |
| *** |
| *** |
| *** |
| *** |
| *** |
| *** |
| *** |
| *** |
| *** |
| *** |
| *** |
| *** |
| *** |
| *** |
| *** |
| *** |
| *** |
| *** |
| *** |
| *** |
| *** |
| *** |
| *** |
| *** |
| *** |
| *** |
| *** |
| *** |
| *** |
| *** |
| *** |
| *** |
| *** |
| *** |
| *** |
| *** |
| *** |
| *** |
| *** |
| *** |
| *** |
| *** |
| *** |
| *** |
| *** |
| *** |
| *** |
| *** |
| *** |
| *** |
| *** |
| *** |
| *** |
| *** |
| *** |
| *** |
| *** |
| *** |
| *** |
| *** |
| *** |
| *** |
| *** |
| *** |
| *** |
| *** |
| *** |
| *** |
| *** |
| *** |
| *** |
| *** |
| *** |
| *** |
| *** |
| *** |
| *** |
| *** |
| *** |
| *** |
| *** |
| *** |
| *** |
| *** |
| *** |
| *** |
| *** |
| *** |
| *** |
| *** |
| *** |
| *** |
| *** |
| *** |
| *** |
| *** |
| *** |
| *** |
| *** |
| *** |
| *** |
| *** |
| *** |
| *** |
| *** |
| *** |
| *** |
| *** |
| *** |
| *** |
| *** |
| *** |
| *** |
| *** |
| *** |
| *** |
| *** |
| *** |
| *** |
| *** |
| *** |
| *** |
| *** |
| *** |
| *** |
| *** |
| *** |
| *** |
| *** |
| *** |
| *** |
| *** |
| *** |
| *** |
| *** |
| *** |
| *** |
| *** |
| *** |
| *** |
| *** |
| *** |
| *** |
| *** |
| *** |
| *** |
| *** |
| *** |
| *** |
| *** |
| *** |
| +*** |
| +*** |
| +*** |
| +*** |
| +*** |
| +*** |
| +*** |
| +*** |
| +*** |
| +*** |
| +*** |
| +*** |
| +*** |
| +*** |
| +*** |
| +*** |
| +*** |
| +*** |
| +*** |
| +*** |
| +*** |
| +*** |
| +*** |
| +*** |
| +*** |
| +*** |
| +*** |
| +*** |
| +*** |
| +*** |
| +*** |
| +*** |
| +*** |
| +*** |
| +*** |
| +*** |
| **** |
| **** |
| **** |
| **** |
| **** |
| **** |
| **** |
| **** |
| **** |
| **** |
| **** |
| **** |
| **** |
| **** |
| **** |
| **** |
| **** |
| **** |
| **** |
| **** |
| **** |
| **** |
| **** |
| **** |
| **** |
| **** |
| **** |
| **** |
| **** |
| **** |
| **** |
| **** |
| **** |
| **** |
| **** |
| **** |
| **** |
| **** |
| **** |
| **** |
| **** |
| **** |
| **** |
| **** |
| **** |
| **** |
| **** |
| **** |
| **** |
| **** |
| **** |
| **** |
| **** |
| **** |
| **** |
| **** |
| **** |
| **** |
| **** |
| **** |
| **** |
| **** |
| **** |
| **** |
| **** |
| **** |
| **** |
| **** |
| **** |
| **** |
| **** |
| **** |
| **** |
| **** |
| **** |
| **** |
| **** |
| **** |
| **** |
| **** |
| **** |
| **** |
| **** |
| **** |
| **** |
| **** |
| **** |
| **** |
| **** |
| **** |
| **** |
| **** |
| **** |
| **** |
| **** |
| **** |
| **** |
| **** |
| **** |
| **** |
| **** |
| **** |
| **** |
| **** |
| **** |
| **** |
| **** |
| **** |
| **** |
| **** |
| **** |
| **** |
| **** |
| **** |
| **** |
| **** |
| **** |
| **** |
| **** |
| **** |
| **** |
| **** |
| **** |
| **** |
| **** |
| ****x |
| ****x |
| ****x |
| ****x |
| ****x |
| ****x |
| ****x |
| ****x |
| ****x |
| ****x |
| ****x |
| ****x |
| ****x |
| ****x |
| ****x |
| ****x |
| ****x |
| ****x |
| ****x |
| ****x |
| ****x |
| ****x |
| ****x |
| ****x |
| ****x |
| ****x |
| ****x |
| ****x |
| ****x |
| ****x |
| ****x |
| ****x |
| ****x |
| ****x |
| ****x |
| ****x |
| ****x |
| ****x |
| ****x |
| ****x |
| ****x |
| ****x |
| ****x |
| ****x |
| ****x |
| ****x |
| ****x |
| ****x |
| ****x |
| ****x |
| ****x |
| ****x |
| ****x |
| ****x |
| ****x |
| ****x |
| ****x |
| ****x |
| ****x |
| ****x |
| ****x |
| ****x |
| ****x |
| ****x |
| ****x |
| ****x |
| ****x |
| ****x |
| ****x |
| ****x |
| ****x |
| ****x |
| ****x |
| ****x |
| ****x |
| ****x |
| ****x |
| ****x |
| ****x |
| ****x |
| ****x |
| ****x |
| ****x |
| ****x |
| ****x |
| ****x |
| ****x |
| ****x |
| ****x |
| ****x |
| ****x |
| ****x |
| ****x |
| ****x |
| ****x |
| ****x |
| ****x |
| ****x |
| ****x |
| ****x |
| ****x |
| ****x |
| ****x |
| ****x |
| ****x |
| ****x |
| ****x |
| ****x |
| ****x |
| ****x |
| ****x |
| ****x |
| ****x |
| ****x |
| ****x |
| ****x |
| ****x |
| ****x |
| ****x |
| ****x |
| ****x |
| ****x |
| ****x |
| ****x |
| ****x |
| ****x |
| ****x |
| ****x |
| ****x |
| ****x |
| ****x |
| ****x |
| ****x |
| ****x |
| ****x |
| ****x |
| ****x |
| ****x |
| ****x |
| ****x |
| ****x |
| ****x |
| ****x |
| ****x |
| ****x |
| ****x |
| ****x |
| ****x |
| ****x |
| ****x |
| ****x |
| ****x |
| ****x |
| ****x |
| ****x |
| x****x |
| x****x |
| x****x |
| x****x |
| x****x |
| x****x |
| x****x |
| x****x |
| x****x |
| x****x |
| x****x |
| x****x |
| x****x |
| x****x |
| x****x |
| x****x |
| x****x |
| x****x |
| x****x |
| x****x |
| x****x |
| x****x |
| x****x |
| x****x |
| *****x |
| *****x |
| *****x |
| *****x |
| *****x |
| *****x |
| *****x |
| *****x |
| *****x |
| *****x |
| *****x |
| *****x |
| *****x |
| *****x |
| *****x |
| *****x |
| *****x |
| *****x |
| *****x |
| *****x |
| ****** |
| ****** |
| ****** |
| ****** |
| ****** |
| ****** |
| ****** |
| ****** |
| ****** |
| ****** |
| ****** |
| ****** |
| ****** |
| ****** |
| ****** |
| ****** |
| ****** |
| ****** |
| ****** |
| ****** |
| ****** |
| ****** |
| ****** |
| ****** |
| ****** |
| ****** |
| ****** |
| ****** |
| ****** |
| ****** |
| ****** |
| ****** |
| ****** |
| ****** |
| +****** |
| +****** |
| +****** |
| +****** |
| +****** |
| +****** |
| +****** |
| +****** |
| +****** |
| +****** |
| +****** |
| +****** |
| +****** |
| +****** |
| +****** |
| +****** |
| +****** |
| +****** |
| +****** |
| +****** |
| +****** |
| +****** |
| +****** |
| +****** |
| +****** |
| +****** |
| +****** |
| +****** |
| +****** |
| +****** |
| +****** |
| +****** |
| +****** |
| +****** |
| +****** |
| +****** |
| +****** |
| +****** |
| +****** |
| +****** |
| +****** |
| +****** |
| +****** |
| ++****** |
| ++****** |
| ++****** |
| ++****** |
| ++****** |
| ++****** |
| ++****** |
| ++****** |
| ++****** |
| ++****** |
| ++****** |
| ++****** |
| ++****** |
| ++****** |
| ++****** |
| ++****** |
| ++****** |
| ++****** |
| ++****** |
| ++****** |
| ++****** |
| ++****** |
| ++****** |
| ++****** |
| ++****** |
| ++****** |
| ++****** |
| ++****** |
| ++****** |
| ++****** |
| ++****** |
| ++****** |
| ++****** |
| ++****** |
| ++****** |
| ++****** |
| ++****** |
| +******* |
| +******* |
| +******* |
| ++******* |
| ++******* |
| ++******* |
| ++******* |
| ++******* |
| ++******* |
| ++******* |
| ++******* |
| ++******* |
| ++******* |
| ++******* |
| ++******* |
| ++******* |
| ++******* |
| ++******* |
| ++******* |
| ++******* |
| ++******* |
| ++******* |
| ++******* |
| ++******* |
| ++******* |
| ++******* |
| ++******* |
| +******** |
| +******** |
| +******** |
| +******** |
| +******** |
| +******** |
| +******** |
| +********x |
| +********x |
| +********x |
| +********x |
| +********x |
| +********x |
| +********x |
| +********x |
| +********x |
| +********x |
| +********x |
| +********x |
| +********x |
| +********x |
| +********x |
| +********x |
| +********x |
| +********x |
| +********x |
| +********x |
| +********x |
| +********x |
| +********x |
| +********x |
| +********x |
| +********x |
| +********x |
| +********x |
| +********x |
| +********x |
| +********x |
| +********x |
| +********x |
| +********x |
| +********x |
| +********x |
| +********x |
| +********x |
| +********x |
| +********x |
| *********x |
| *********x |
| *********x |
| *********x |
| *********x |
| *********x |
| *********x |
| *********x |
| *********x |
| *********x |
| *********x |
| *********x |
| *********x |
| *********x |
| *********x |
| *********x |
| +*********x |
| +*********x |
| +*********x |
| +*********x |
| +*********x |
| +*********x |
| +*********x |
| +*********x |
| +*********x |
| +*********x |
| +*********x |
| +*********x |
| +*********x |
| ++*********x |
| ++*********x |
| ++*********x |
| ++*********x |
| ++*********x |
| ++********** |
| ++********** |
| ++********** |
| ++********** |
| ++********** |
| ++********** |
| ++********** |
| ++********** |
| ++********** |
| ++********** |
| +++********** |
| ++*********** |
| ++*********** |
| ++*********** |
| ++*********** |
| ++*********** |
| ++*********** |
| ++*********** |
| ++*********** |
| ++*********** |
| ++*********** |
| ++*********** |
| ++*********** |
| ++*********** |
| ++*********** |
| ++*********** |
| ++*********** |
| ++*********** |
| ++*********** |
| ++*********** |
| ++*********** |
| ++*********** |
| ++*********** |
| ++*********** |
| ++*********** |
| ++*********** |
| ++*********** |
| ++*********** |
| ++*********** |
| ++*********** |
| ++*********** |
| ++*********** |
| ++*********** |
| ++*********** |
| ++*********** |
| ++*********** |
| ++*********** |
| ++*********** |
| ++*********** |
| ++*********** |
| ++***********x |
| ++***********x |
| +************x |
| +************x |
| +************x |
| +************x |
| +************x |
| +************x |
| +************x |
| +************* |
| +************* |
| +************* |
| +************* |
| +************* |
| +************* |
| +************* |
| +************* |
| +************* |
| +************* |
| +************* |
| +*************x |
| +*************x |
| +*************x |
| +*************x |
| +*************x |
| +*************x |
| +*************x |
| +*************x |
| +*************x |
| +*************x |
| +************** |
| +************** |
| +************** |
| +************** |
| +************** |
| +************** |
| +************** |
| +************** |
| +************** |
|++************** |
|++************** |
|++************** |
|++************** |
|++************** |
|++**************x |
|++**************x |
|++**************x |
|+***************x |
|+***************x |
|+**************** |
|+**************** |
|+****************x+ |
|+****************x+ |
|+****************x+ |
|+****************x+ |
|+****************x* |
|+****************** + |
|+****************** ++x |
|+******************x ++* |
|+******************x ++* |
|+******************x ++* |
|+******************x+x ++* |
|*******************x+x +** |
|*******************x*x +**xx |
|*********************xx x ***xx |
|*********************xx x + ***xx |
|*********************** xx ++ +***xx |
|*********************** x + xx ++*x+***xx |
|*********************** ++ *+++ xxxx x ++++*******xxx|
| |_____A_____| |
| |_____A______| |
+------------------------------------------------------------------------------+
N Min Max Median Avg Stddev
x 6261 0.320022 81.947655 10.549157 10.777466 6.5456486
+ 6507 0.334026 78.256834 10.187824 10.159948 6.6247137
Difference at 95.0% confidence
-0.617518 +/- 0.228524
-5.72972% +/- 2.12038%
(Student's t, pooled s = 6.58606)
no crc32 check
$ ministat -s -C 6 *varnishlog_Timestamp*
x master_varnishlog_Timestamp:Resp.20241021_112820.d7540e548dc2771a91a1a842949bb82904e06032
+ vai_varnishlog_Timestamp:Resp.20241021_113109.058edd04dedbde25d3193580fc9a63aba3d0c602
+------------------------------------------------------------------------------+
| * |
| * |
| * |
| * |
| * |
| * |
| * |
| * |
| * |
| * |
| * |
| * |
| * |
| * |
| * |
| * |
| * |
| * |
| * |
| * |
| * |
| * |
| * |
| * |
| * |
| * |
| * |
| * |
| * |
| * |
| * |
| * |
| * |
| * |
| * |
| * |
| * |
| * |
| * |
| * |
| * |
| * |
| * |
| * |
| * |
| * |
| * |
| * |
| * |
| * |
| * |
| * |
| * |
| * |
| * |
| * |
| * |
| * |
| * |
| * |
| * |
| * |
| * |
| * |
| * |
| * |
| * |
| * |
| * |
| * |
| * |
| * |
| * |
| * |
| * |
| * |
| * |
| * |
| * |
| * |
| * |
| * |
| * |
| * |
| * |
| * |
| * |
| * |
| * |
| * |
| * |
| * |
| * |
| * |
| * |
| * |
| * |
| x* |
| x* |
| x* |
| x* |
| x* |
| x* |
| x* |
| x* |
| x* |
| x* |
| x* |
| x* |
| x* |
| x* |
| x* |
| x* |
| x* |
| x* |
| x* |
| x* |
| x* |
| x* |
| x* |
| x* |
| x* |
| x* |
| x* |
| x* |
| x* |
| x* |
| x* |
| x* |
| x* |
| x*x |
| x*x |
| x*x |
| x*x |
| x*x |
| x*x |
| x*x |
| x*x |
| x*x |
| x*x |
| x*x |
| x*x |
| x*x |
| x*x |
| x*x |
| x*x |
| x*x |
| x*x |
| x*x |
| x*x |
| x*x |
| x*x |
| x*x |
| x*x |
| x*x |
| x*x |
| x*x |
| x*x |
| x*x |
| x*x |
| x*x |
| x*x |
| x*x |
| x*x |
| x*x |
| x*x |
| x*x |
| x*x |
| x*x |
| x*x |
| x*x |
| x*x |
| x*x |
| x*x |
| x*x |
| x*x |
| x*x |
| x*x |
| x*x |
| x*x |
| x*x |
| x*x |
| x*x |
| x** |
| x** |
| x** |
| x** |
| x** |
| x** |
| x** |
| x** |
| x** |
| x** |
| x** |
| x** |
| x** |
| x** |
| x** |
| x** |
| x** |
| x** |
| x** |
| x** |
| x** |
| x** |
| x** |
| x** |
| x** |
| x** |
| x** |
| x** |
| x** |
| x** |
| x** |
| x** |
| x** |
| x** |
| x** |
| x** |
| x** |
| x** |
| x** |
| x** |
| x** |
| x** |
| x** |
| x** |
| x** |
| x** |
| x** |
| x** |
| x** |
| x** |
| x** |
| x** |
| x** |
| x** |
| x** |
| x** |
| x** |
| x** |
| x** |
| x** |
| x** |
| x** |
| x** |
| x** |
| x** |
| x** |
| x** |
| x** |
| x** |
| x** |
| x** |
| x** |
| x** |
| x** |
| x** |
| x** |
| x** |
| x** |
| x** |
| x** |
| x** |
| x** |
| x** |
| x** |
| x** |
| x** |
| x** |
| x** |
| x** |
| x** |
| x** |
| x** |
| x** |
| x** |
| x** |
| x** |
| x** |
| x** |
| x** |
| x** |
| x** |
| x** |
| x** |
| x** |
| x** |
| x** |
| x** |
| x** |
| x** |
| x** |
| x** |
| x** |
| x** |
| x** |
| x** |
| x** |
| x** |
| x** |
| x** |
| x** |
| x** |
| x** |
| x** |
| x** |
| x** |
| x** |
| x** |
| x** |
| x** |
| x** |
| x** |
| x** |
| x** |
| x** |
| x** |
| x** |
| x** |
| x** |
| x** |
| x** |
| x** |
| x** |
| x** |
| x** |
| x** |
| x** |
| x** |
| x** |
| x** |
| x** |
| x** |
| x** |
| x** |
| x** |
| x** |
| x** |
| x** |
| x** |
| x** |
| x** |
| x** |
| x** |
| x** |
| x** |
| x** |
| x** |
| x** |
| x** |
| x** |
| x** |
| x** |
| x** |
| x** |
| x** |
| x** |
| x** |
| x** |
| x** |
| x** |
| x** |
| x** |
| x** + |
| x** + |
| x** ++ |
| x** ++ |
| x** ++ |
| x** ++ |
| x** ++ |
| x** ++ |
| x** ++ |
| x** ++ |
| x** ++ |
| x** ++ |
| x** ++ |
| x**+ ++ |
| x**+ ++ |
| x**+ ++ |
| x**+ ++ |
| x**+ ++ |
| x**+ ++ |
| x**+ ++ |
| x**+ ++ |
| x**+ ++ |
| x**+ ++ |
| x**+ ++ |
| x**+ ++ |
| x**+ ++ |
| x**+ ++ |
| x**+ ++ |
| x**+ ++ |
| x**+ ++ |
| x**+ ++ |
| x**+ ++ |
| x**+ ++ |
| x**+ ++ |
| x**+ ++ |
| x**+ ++ |
| x**+ ++ |
| x**+ ++ |
| x**+ ++ |
| x**+ ++ |
| x**+ ++ |
| x**+ ++ |
| x**+ ++ |
| ***+ ++ |
| ***+ ++ |
| ***+ ++ |
| ***+ ++ |
| ***+ ++ |
| ***+ ++ |
| ***+ ++ |
| ***+ ++ |
| ***+ ++ |
| ***+ ++ |
| ***+ ++ |
| ***+ ++ |
| ***+ ++ |
| ***+ ++ |
| ***+ ++ |
| ***+ ++ |
| ***+ ++ |
| ***+ ++ |
| ***+ ++ |
| **** ++ |
| **** ++ |
| **** ++ |
| **** ++ |
| **** ++ |
| **** ++ |
| **** ++ |
| **** ++ |
| **** ++ |
| **** ++ |
| **** ++ |
| **** ++ |
| **** *+ |
| **** *+ |
| **** *+ |
| **** *+ |
| **** *+ |
| **** *+ |
| **** *+ |
| **** *+ |
| **** *+ |
| **** *+ |
| **** *+ |
| **** *+ |
| **** *+ |
| **** *+ |
| **** *+ |
| **** *+ |
| **** *+ |
| **** *+ |
| **** *+ |
| **** *+ |
| **** *+ |
| **** *+ |
| **** *+ |
| **** *+ |
| **** *+ |
| **** *+ |
| **** *+ |
| **** *+ |
| **** *+ |
| **** *+ |
| **** *+ |
| **** *+ |
| **** *+ |
| **** *+ |
| **** *+ |
| **** *+ |
| **** *+ |
| **** *+ |
| **** *+ |
| **** *+ |
| **** *+ |
| **** *+ |
| **** *+ |
| **** *+ |
| **** x*+ |
| **** x*+ |
| **** x*+ |
| **** x*+ |
| **** x*+ |
| **** x*+ |
| **** x*+ |
| **** x*+ |
| **** x*+ |
| **** x*+ |
| **** x*+ |
| **** **+ |
| **** **+ |
| **** **+ |
| **** **+ |
| **** **+ |
| **** **+ |
| **** **+ |
| **** **+ |
| **** **+ |
| **** **+ |
| **** **+ |
| **** **+ |
| **** **+ |
| **** **+ |
| **** *** |
| **** *** |
| **** *** |
| **** *** |
| **** +*** |
| **** +*** |
| **** +*** |
| **** +*** |
| **** +*** |
| **** +*** |
| **** +*** |
| **** +*** |
| **** +*** |
| **** +*** |
| **** +*** |
| **** +***+ |
| **** +***+ |
| **** +***+ |
| **** +***+ |
| **** +***+ |
| **** +***+ |
| **** +***+ |
| **** +***+ |
| **** +***+ |
| **** +***+ |
| **** +***+ |
| **** +***+ |
| **** +***+ |
| **** +***+ |
| **** +***+ |
| **** +***+ |
| **** +***+ |
| **** +***+ |
| **** +***+ |
| **** +***+ |
| **** +***+ |
| **** +***+ |
| **** +***+ |
| **** +***+ |
| **** +***+ |
| **** +***++ |
| **** +***++ |
| **** +***++ |
| ****++***++ |
| ****++***++ |
| ****++***++ |
| ****++***++ |
| ****++***++ |
| ****++***++ |
| ****++***++ |
| ****++***++ |
| ****++***++ |
| ****++***++ |
| ****++***++ |
| ****++***+++ |
| ****++***+++ |
| ****++***+++ |
| ****++***+++ |
| ****++***+++ |
| ****++***+++ |
| ****++***+++ |
| ****++***+++ |
| ****++***+++ |
| ****++***+++ |
| ****++***+++ |
| ****++***+++ |
| ****++***+++ |
| *****+***+++ |
| *****+***+++ |
| *****+***+++ |
| *****+***+++ |
| *****+***+++ |
| *****+***+++ |
| *****+***+++ |
| *****+***+++ |
| *****+***+++ |
| *****+***+++ |
| *****+***+++ |
| *****+***+++ |
| *****+***+++ |
| *****+***+++ |
| *****+***+++ |
| *****+***+++ |
| *****+***+++ |
| *****+***+++ |
| *****+***+++ |
| *****+***+++ |
| *****+***+++ |
| *****+***+++ |
| *****+***+++ |
| *****+***+++ |
| *****+***+++ |
| *****+***+++ |
| *****+***+++ |
| *****+***+++ |
| *****+***+++ |
| *****+***+++ |
| *****+***+++ |
| *****+***+++ |
| *****+***++++ |
| *****+***++++ |
| *****+***++++ |
| *****+***++++ |
| *****+***++++ |
| *****+***++++ |
| *****+***++++ |
| *****+***++++ |
| *****+***++++ |
| *****+***++++ |
| *****+***++++ |
| *****+***++++ |
| *****+***++++ |
| *****+***++++ |
| *****+***++++ |
| *****+***++++ |
| *****+***+++* |
| *****+***+++* |
| *****+***+++* |
| *****+***+++* |
| *****+***+++* |
| *****+***+++* |
| *****+***+++* |
| *****+***+++* |
| *****+***+++* |
| *****+***+++* |
| *****+***+++* |
| *****+***+++* |
| *****+***+++* |
| *********+++* |
| *********+++* |
| *********+++* |
| *********++** |
| *********++** |
| *********++** |
| *********++** |
| *********++** |
| *********++** |
| *********++** |
| *********++** |
| *********+*** |
| *********+*** |
| *********+*** |
| *********+*** |
| *********+*** |
| *********+*** |
| *********+*** |
| *********+*** |
| *********+*** |
| *********+*** |
| *********+*** |
| *********+*** |
| *********+*** |
| *********+*** |
| ************* |
| ************* |
| ************* |
| ************* |
| ************* |
| ************* |
| ************* |
| ************* |
| ************* |
| ************* |
| ************* |
| ************* |
| ************* |
| ************* |
| ************* |
| ************* |
| ************* |
| ************* |
| ************* |
| ************* |
| ************* |
| ************* |
| ************* |
| ************* |
| ************* |
| ************* |
| ************* |
| ************* |
| ************* |
| ************* |
| ************* |
| ************* |
| ************* |
| ************* |
| ************* |
| *************+ |
| *************+ |
| *************+ |
| *************+ |
| *************+ |
| *************+ |
| *************+ |
| *************+ |
| *************+ |
| *************+ |
| *************+ |
| *************+ |
| *************+ |
| *************+ |
| *************+ |
| *************+ |
| *************+ |
| *************+ |
| *************+ |
| *************+ |
| *************+ |
| *************+ |
| *************+ |
| *************+ |
| *************+ |
| *************+ |
| *************+ |
| *************+ |
| *************+ |
| *************+ |
| *************+ |
| *************+ |
| *************+ |
| *************+ |
| *************+ |
| *************+ |
| *************+ |
| *************+ |
| *************+ |
| *************+ |
| *************+ |
| *************+ |
| *************+ |
| *************+ |
| *************+ |
| *************+ |
| *************+ |
| *************+ |
| *************+ |
| *************+ |
| *************+ |
| *************+ |
| *************+ |
| *************+ |
| *************+ |
| *************+ |
| *************+ |
| *************+ |
| *************+ |
| *************+ |
| *************+ |
| *************+ |
| *************+ |
| *************+ |
| *************+ |
| *************++ |
| *************++ |
| *************++ |
| *************++ |
| *************++ |
| *************++ |
| *************++ |
| *************++ |
| *************++ |
| *************++ |
| *************++ |
| *************++ |
| *************++ |
| *************++ |
| *************++ |
| *************++ |
| *************++ |
| *************++ |
| *************++ |
| *************++ |
| *************++ |
| *************++ |
| *************++ |
| *************++ |
| *************++ |
| *************++ |
| *************++ |
| *************++ |
| *************++ |
| *************++ |
| *************++ |
| *************++ |
| *************++ |
| **************+ |
| **************+ |
| **************+ |
| **************+ |
| **************+ |
| **************+ |
| **************+ |
| **************++ |
| **************++ |
| **************++ |
| **************++ |
| **************++ |
| **************++ |
| **************++ |
| **************++ |
| **************++ |
| **************++ |
| **************++ |
| **************++ |
| **************++ |
| **************++ |
| **************++ |
| **************++ |
| **************++ |
| **************++ |
| **************++ |
| **************++ |
| **************++ |
| **************++ |
| **************++ |
| **************++ |
| **************++ |
| **************++ |
| x**************++ |
| x**************++ |
| x**************++ |
| x**************++ |
| x**************++ |
| x**************++ |
| x**************++ |
| x**************++ |
| x**************++ |
| x**************++ |
| x**************++ |
| x**************++ |
| x**************++ |
| x**************++ |
| x**************++ |
| x**************++ |
| x**************++ |
| x**************++ |
| x**************++ |
| x**************++ |
| x**************++ |
| x**************++ |
| x***************+ |
| x***************+ |
| x***************+ |
| x***************+ |
| x***************+ |
| x***************+ |
| x***************+ |
| x***************+ |
| x***************+ |
| x***************+ |
| x***************+ |
| x***************+ |
| x***************+ |
| x***************+ |
| x***************+ |
| x***************+ |
| x***************+ |
| x***************+ |
| x**************** |
| x**************** |
| x**************** |
| x**************** |
| x**************** |
| x**************** |
| x**************** |
| x**************** |
| x**************** |
| x**************** |
| x**************** |
| x**************** |
| x**************** |
| x**************** |
| x**************** |
| x**************** |
| x**************** |
| x**************** |
| x**************** |
| x**************** |
| x**************** |
| x**************** |
| x****************+ |
| x****************+ |
| *****************+ |
| *****************+ |
| *****************+ |
| *****************+ |
| *****************+ |
| *****************+ |
| *****************+ |
| *****************+ |
| *****************+ |
| *****************+ |
| *****************+ |
| *****************+ |
| *****************+ |
| *****************+ |
| *****************+ |
| ****************** |
| ****************** |
| ****************** |
| ****************** |
| ****************** |
| ****************** |
| ****************** |
| ****************** |
| ****************** |
| ****************** |
| ****************** |
| ****************** |
| ****************** |
| ****************** |
| ******************+ |
| ******************+ |
| ******************+ |
| ******************+ |
| ******************+ |
| ******************+ |
| ******************+ |
| ******************+ |
| ******************+ |
| ******************+ |
| ******************+ |
| ******************+ |
| ******************+ |
| ******************+ |
| ******************+ |
| ******************+ |
| ******************+ |
| ******************+ |
| ******************+ |
| ******************* |
| ******************* |
| ******************* |
| ******************* |
| ******************* |
| ******************* |
| ******************* |
| ******************* |
| ******************* |
| ******************* |
| ******************* |
| ******************* |
| ******************* |
| *******************+ |
| *******************+ |
| ******************** |
| ******************** |
| ******************** |
| ******************** |
| ******************** |
| ******************** |
| ******************** |
| ******************** |
| ******************** |
| ******************** |
| ******************** |
| ******************** |
| ******************** |
| ******************** |
| ******************** |
| ******************** |
| ******************** |
| ******************** |
| ******************** |
| ********************+ |
| ********************+ |
| ********************* |
| ********************* |
| ********************* |
| ********************* |
| ********************* |
| ********************* |
| ********************* |
| ********************* |
| ********************* |
| ********************* |
| ********************* |
| ********************* |
| ********************* |
| ********************* |
| ********************* |
| ********************* |
| ********************* |
| ********************* |
| ********************* |
| ********************* |
| ********************* |
| ********************* |
| ********************* |
| ********************* |
| ********************* |
| *********************x |
| *********************x |
| *********************x |
| *********************x |
| *********************x |
| *********************x |
| *********************x |
| *********************x |
| *********************x |
| *********************x |
| *********************x |
| *********************xx |
| **********************x |
| **********************x |
| **********************x |
| **********************x |
| **********************x |
| **********************x |
| **********************x x |
| **********************xxx |
| **********************xxx |
| **********************xxx |
| **********************xxx |
| **********************xxx |
| **********************xxx |
| **********************xxxx |
| **********************xxxx |
| **********************xxxx |
| **********************xxxx |
| **********************xxxx |
| **********************xxxx |
| **********************xxxx x |
| **********************xxxx x |
|x**********************xxxx x |
|x**********************xxxx x |
|x**********************xxxx x |
|x***********************xxx x |
|x************************xx x |
|x************************xx x |
|x************************xx x |
|x************************xx x |
|x************************xx x |
|x************************xxxx |
|x************************xxxx |
|x************************xxxx |
|x************************xxxx |
|x************************xxxxxx |
|**************************xxxxx |
|**************************xxxxx x |
|**************************xxxxx x |
|**************************xxxxx x |
|***************************xxxx x |
|***************************xxxx x |
|****************************xxx xx |
|****************************xxx xx |
|****************************xxx xx |
|****************************xxx xx |
|****************************xxx xx |
|*****************************xx xx |
|*****************************xxxxx |
|*****************************xxxxx |
|*****************************xxxxx |
|*****************************xxxxx |
|*****************************xxxxx |
|*****************************xxxxx |
|*****************************xxxxx |
|******************************xxxxx |
|******************************xxxxxx x |
|*******************************xxxxx x |
|*******************************xxxxx x |
|*******************************xxxxxx x |
|********************************xxxxxxx |
|*********************************xxxxxx x |
|*********************************xxxxxx x x x |
|**********************************xxxxxx x x x xx xx |
|***********************************xxxxx xxxxxx xxx xx xx x x |
|****************************************xxxxxxx+x x xxxx xx xx xxx xx x x|
| |_____MA_______| |
| |_____A_____| |
+------------------------------------------------------------------------------+
N Min Max Median Avg Stddev
x 10671 0.044828 32.76001 3.286472 3.9256969 3.1341722
+ 10551 0.037735 20.157598 3.679551 3.9293176 2.4154246
No difference proven at 95.0% confidence
[^1]: start script
v=$(git rev-parse HEAD)
d=$(date +%Y%m%d_%H%M%S)
/tmp/bin/varnishlog -n /tmp/t -g raw -I Timestamp:Resp >varnishlog_Timestamp:Resp.$d.$v &
make -j 20 install && pushd ../*bsynth && make -j 20 install && popd && time /tmp/sbin/varnishd -f $PWD/slink/bench_pass/bench_pass.vcl -a :8080 -a self=@varnish -n /tmp/t -p fetch_maxchunksize=64k -p vsl_mask=+debug -F
pkill varnishlog
With the force push as of just now I fixed a misnomer of struct vaiov and renamed it to struct viov in preparation of a suggestion for more general use. And also the vanishified io vector has nothing to do with the asynchronous iterator, so while it was funny (at least to me) that the letter i stood for two words, viov really is the better name
With the force-push as of just now (226d9ebfbe1a81ac0564da8f4c1b77a465f5e1c8), I have changed the implementation to use a "flexible array of things" for scatter arrays and return of leases which I worked on in November and December - mostly in preparation of bringing VAI also to VDPs. I have edited the commit messages (please let me know if you see anything inconsistent now) and added an explanation to the commit message of 928a388ca1fff4f40b87665cf321e63f7c8fff92, which I am going to copy here (with minor edits) as an introduction to the changes:
Batching with scatter arrays (VSCARAB)
The existing objiterate_f works on one buffer at a time, yet even before asynchronous I/O, issuing one system call for each buffer would be inefficient. So, for the case of HTTP/1, the V1L layer collects buffers into an array of io vectors (struct iovec), which are handed over to the kernel using writev(). These arrays of io vectors seem to have no established name even after decades of existence, elsewhere they are called siov or sarray, so in this API, we are going to call them scatter arrays.
With the new API, we (will) use scatter arrays for all the processing steps: The goal is that storage fills a scatter array, which then gets processed and maybe replaced by filters, until finally some transport hands many I/Os at once to the kernel.
Established interfaces follow the signature of writev(), they have a pointer to an array of struct iovec and a count (struct iovec *iov, int iovcnt).
Yet for our plans, we want to have something which can be passed around in a single unit, to ensure that the array is always used with the right count, something which can vary in size and live on the heap or the stack.
This is the VSCARAB, the Varnish SCatter ARAy of Buffers, basically a container struct with a flexible array member (fam). The VSCARAB has a capacity, a used count, and is annotated with v_counted_by_() such that, when support for bounds checking is improved by compilers, we get additional sanity checks (and possibly optimizations).
We add macros to work on VSCARABs for allocation (on the stack and heap), initialization, checking (magic and limits), iterating, and adding elements.
VSCARET and VFLA
Managing scatter arrays is one side of the coin, when we are done using buffers, we need to return them to storage, such that storage can do LRU things or reuse memory. As before, we want to batch these operations for efficiency.
As an easy to use, flexible data structure, we add VSCARABs sibing VSCARET. And, because both are basically the same, we generalize macros as VFLA, Varnish Flexible Arrays.
force-push: ae38cd68126ca393b02b4c0ebcccc1a70738bd08 updates to current master and has minor polishings (edit: I had overlooked a fixup)
With the 5992d534e0df4b2c42f4acef50ff87fc8f658de0 push, support for VDPs has been added and the "prepare for AI" task should be getting somehow near completion. Details from the main commit 38722a39c353db6ed8d37057a67b200b54028cae. Before continuing here, please read the previous notes of this PR.
Why?
The VAI interface lay ground for asynchronous iteration, but did not yet address filters.
The existing VDP model is a "push" design: VDP_ObjIterate() calls VDP_bytes() with data from storage. VDP_bytes() calls the first VDP, which does its processing, calls VDP_bytes() for the next VDP etc until the last VDP sends data out somewhere.
This is a good model for our existing "synchronous I/O from threads" design, but it is, again, fundamentally incompatible with async I/O (see Context): Any filter in the chain can assume that VDP_bytes(..., VDP_FLUSH) will, when it returns, be done with the buffer, such that the filter may issue the next VDP_byte(), potentially on the same buffer.
For async I/O, we need a model where buffers are handed out and explicitly returned when done with.
Discarded prior work
Before ending up at the model in this patch, a "push" design had been attempted where buffers would be handed from filter to filter, with the option for each filter to say "first I need more input" or "I have more output after this". This model turned out overly complicated, so it was discarded.
How?
The model in this patch takes VDP from "push" to "pull": The top level delivery code asks the filter chain for buffers like it would ask the ObjVAI API for data. An example is coming up in patch "vmod_debug: Switch transport_vai to VDPIO Upgrade", it basically looks like this:
do {
nbufs = vdpio_pull(req->vdc, NULL, scarab);
send_data(scarab);
} while ((scarab->flags & VSCARAB_F_END) == 0)
Similarly to VDP_bytes(), vdpio_pull() calls into the VDP layer, but this time in the other direction, from last VDP to first to storage. Each VDP now returns buffers it has ready, and when it needs more data, it calls vdpio_pull() to get more input buffers from the next layer and ultimately from the storage engine.
API Usage
The VDP filter API is similar to the existing API in that it consists of an initializer, a finalizer and a "bytes" function, which is now called "lease" to match the lease concept introduced with VAI. The lease function is called from vdpio_pull(). It is responsible for vdpio_pull()'ing data from the previous layer, processing it and putting buffers into a provided output vscarab.
Any data which the VDP creates needs to put into buffers allocated from storage via ObjVAIbuffer(). Any buffers which the VDP receives from a previous layer and does not emit in the output vscarab need to be returned with ObjVAIreturn(). To batch these returns, a VSCARET is kept in the VDP context and managed by these helpers:
-
vdpio_return_lease()to return a single lease to the batch -
vdpio_return_vscarab()to return a full vscarab to the batch -
vdpio_consolidate_vscarab()to return all leases with a zero size, where the zero size marks them being consumed. This is intended to facilitate input vscarabs.
Naturally, this API is likely to still evolve.
VDPIO management & transitional interface
The basic model for VDPIO is kept from VDP: There is VDPIO_Push() to match VDP_Push() and VDPIO_Close() to match VDP_Close(). Yet, for the time being, we need to have VDP and VDPIO co-exist: Not all filters will be ready for VDPIO and there will be bugs, so we will want the option to go back to the old interface.
This is why VDPIO_Push() is #ifdef'ed out, and VDPIO_Upgrade() used: It works an already initialized VDP filter list and retuns if it can be upgraded to VDPIO. To do so, it calls the vdpio_upgrade function of each VDP. If a vdpio_upgrade function is missing for any filter, all of the upgrade fails and the caller is expected to fall back to traditional VDP.
What's new in 30b6f62a713bf017bdb0fad7abac784ea5b8993f since https://github.com/varnishcache/varnish-cache/pull/4209#issuecomment-2668690440 (5992d534e0df4b2c42f4acef50ff87fc8f658de0)
- Various polishing and minor fixes
- Generalize the magic lease value
VAI_LEASE_NORETfor static data and lease fragments - Added
VSCARAB_ADD_IOV_NORETfor the common case of "Here's some static data to add to the vscarab" - Added
VDPIO_Push()for real: This is useful to add a VDPIO-enabled VDP after aVDPIO_Upgrade() - Added
VDPIO_Close1()to allow a VDP to remove itself from the filter list - storage_simple: Always add
VSCARAB_F_ENDto flags once we're done
What's new in 15d660ddde8a632126fec2da57d0056d4aa1b5b6 since https://github.com/varnishcache/varnish-cache/pull/4209#issuecomment-2693709610 (30b6f62a713bf017bdb0fad7abac784ea5b8993f)
- Clarify that
vdpio_lease_f()now maintainsstruct vdp_entry.bytes_inand.callsmembers - Clarify what a zero return value of
vai_lease_f()/vdpio_lease_f()means ("not enough space in vscarab"), and in particular that it does not mean EOF. - Clarify that EOF is only signalled through
VSCARAB_F_ENDin the scarab flags - Clarify that
vdpio_lease_f()needs to handle partially vscarabs and why that is not the case for the transitional V1L implementation
What's new in fbe0ed39659c29bccde259f51a988450db393d94 since https://github.com/varnishcache/varnish-cache/pull/4209#issuecomment-2696924765 (15d660ddde8a632126fec2da57d0056d4aa1b5b6)
- Added VDPIO wrappers for VAI because VDPIO maintains VAI state through
struct vdp_ctx - Polished dbg_vai_lease() of the demo transport
And another round : fbe0ed39659c29bccde259f51a988450db393d94 -> 1d8d295e4b35d9333a0c89c35bdfa2a757ce5f01
- Various Flexelint polishing, constification and various minor fixes
- Test/Demo
VDPIO_Close1()
Fixes 1d8d295e4b35d9333a0c89c35bdfa2a757ce5f01 -> 760f20e261030fb5b26b0160174e890e84efedb0
- Fix a double lease return via
sml_ai_return()lastmarker - Make
sml_ai_return()a noop for emptyscaret - Fix
vdpio_pullfor no filters ((struct vdp_ctx).vdplist empty)
760f20e261030fb5b26b0160174e890e84efedb0 -> 6f91629df02e8e456fec0fb4faf27a7563b70efa:
- Add VDPIO Support to the range VDP
As this implementation passes all b and c tests with an out-of-tree transport implementation, I consider it ready for merge.
@mbgrydeland bugwash participants were not sure if you still wanted to have a final look, so we put this on "last call".
@mbgrydeland bugwash participants were not sure if you still wanted to have a final look, so we put this on "last call".
I'm not planning on spending more time with this before the release, so please consider this as LGTM from me.
