brefphp
Performance benchmarks
So I did some testing on a small app. I tested with #17 included (that PR does not have any performance impact).
A normal request: 10ms First request: 1805ms First request with BrefKernel: 510ms
BrefKernel spent 88ms preparing cache directory. It spent 77 ms out of those 88ms to copy the "pools" directory.
So thanks so #17, I disabled copying "pools" and enabled opcache.preload.
A normal request: 7.5ms First request: 1144ms First request with BrefKernel: 160ms
Yes. The first (cold) request is still 20 times slower. (Or 51x slower without preloading) But it is better than 180x slower as we began with.
Note: These sample data may not be reliable. I only did 2 samples of each. To get better data I should take 5-10 samples and show the average. However, I did deploy over CI so I know the process of warmup etc is the same.
I'm confused as to why preloading improves the first request time.
Preload does load files on boot so they are available within the request right? So shouldn't the first request be slower because it loads stuff it does not need? Or is the preload time within the init time and therefore not counted?
Actually no. Preload loads stuff before the first request. Ie, when PHP-Fpm is starting. That does technically happen before we get the "request data" from AWS.
So my "initialisation" should be a little bit slower when using preload. But that was nothing I noticed.
Preload is btw also faster as the init process has more CPU power: https://medium.com/@hichaelmart/shave-99-93-off-your-lambda-bill-with-this-one-weird-trick-33c0acebb2ea
First, awesome job man @Nyholm . Thank you.
Just some future prospects thoughts here.
With every day, and especially with bref, we are moving more and more towards pre-packaged PHP releases. Something that is quite unusual in PHP world - we are packaging compiled assets, compiled/installed vendors, soon compiled code, and one day i hope precompiled container itself. We might as well look into how other ecosystems do it and why. I briefly asked Nikita about the theorethical possibility of precompiling the whole thing into a binary release, but he said that apart from some little CPU benefit he did not see any real value.
Back to topic at hand...
There might be a future possibility of compiling and baking the preloaded opcodes as well as symfony cache into the distribution layer.
-
As far as opcodes go, at the moment i do not know of a way to forcibly specify the output location, which would allow us to distribute precompiled code along with the lambda code. Similarly to how we would treat a binary release. But if we look into it, there might be a way to compile on one lambda and bake the whole layer into a new lambda, or something similar. I think the exact same process is called compiler bootstrapping in C-world.
-
As for baking and including a precompiled Symfony container, caches and whatever else comes with it - i did do it on non-bref instances a year ago, and it kind of worked. But it was just a quick local experiment, and i never dwelled deeper into it, to prove that it actually worked and if it had any performance benefit. I hope @Nyholm could tell us more about the symfony side of things, as i beleive that the same principle as above could be employed. Just run the first request on a lambda, copy the whole container and caches, and make a new lambda. Unless there are some drawbacks or blockers i do not know about.
@markomitranic what I usually do is compile the Symfony cache in a Bref docker image -> the paths are all good then.
Out of curiosity I benched my project too with different php versions and preload on/off.
I now used the configuration i mentioned in https://github.com/brefphp/symfony-bridge/issues/21#issuecomment-612695723 even though that is irrelevant for my test request since it uses no validators or property-info. It is therefore similar to @Nyholm's test since his test had not copied the pools directory (what this library currently mainly does)
I always ran 2 requests to check for cold start and warm run. I didn't repeat runs but the performance in lambda is really consistent so I don't worry too much about it. The page I requested is a login page so there is a session to dynamodb in there and symfony form is loaded including lots of very small twig templates.
php 7.3
REPORT RequestId: 7cc167fb-5d41-4d59-b301-baf1358b63df Duration: 1174.78 ms Billed Duration: 1600 ms Memory Size: 1024 MB Max Memory Used: 120 MB Init Duration: 337.25 ms
REPORT RequestId: 320c114e-d88a-49b4-9f6c-237ca224eeab Duration: 56.56 ms Billed Duration: 100 ms Memory Size: 1024 MB Max Memory Used: 120 MB
php 7.4
REPORT RequestId: b4c1a445-fc15-46d3-a73a-cdb93e5372cd Duration: 1072.20 ms Billed Duration: 1500 ms Memory Size: 1024 MB Max Memory Used: 125 MB Init Duration: 332.41 ms
REPORT RequestId: a8473aa9-0f1d-4800-872d-f9d22f55f9b7 Duration: 56.21 ms Billed Duration: 100 ms Memory Size: 1024 MB Max Memory Used: 125 MB
php 7.4 with container.dumper.inline_class_loader set to true because it defaults to false in php 7.4 even tough it is needed for the preload feature. It is true by default in 7.3.
REPORT RequestId: 7d9e1292-dadd-4e01-a99f-2e9ce2424c0f Duration: 1094.20 ms Billed Duration: 1500 ms Memory Size: 1024 MB Max Memory Used: 125 MB Init Duration: 336.00 ms
REPORT RequestId: 3e55912c-40ea-4958-a4f3-995ed54963ab Duration: 49.77 ms Billed Duration: 100 ms Memory Size: 1024 MB Max Memory Used: 125 MB
php 7.4 with preload configured like opcache.preload=${LAMBDA_TASK_ROOT}/var/cache/prod/srcApp_KernelProdContainer.preload.php
REPORT RequestId: 363c7125-1255-4362-8239-99d5c2865e02 Duration: 609.20 ms Billed Duration: 1200 ms Memory Size: 1024 MB Max Memory Used: 131 MB Init Duration: 570.07 ms
REPORT RequestId: cf862c09-ddd9-49f4-8057-a740566138de Duration: 59.27 ms Billed Duration: 100 ms Memory Size: 1024 MB Max Memory Used: 131 MB
So php 7.4 is faster in any way. Although you should enable the inline_class_loader option which defaults to false starting with 7.4.
The preload feature does improve cold start time from 1430.20 ms to 1179.27 ms (duration + init time) but it seams to slightly decrease performance for warm requests.
Not this is for symfony 4.4 which is the first version to include the preload feature. There are improvements in 5.1.