wordpress-develop icon indicating copy to clipboard operation
wordpress-develop copied to clipboard

#59269 Add support for adding metadata in bulk

Open johnbillion opened this issue 2 years ago • 5 comments

Trac ticket: https://core.trac.wordpress.org/ticket/59269

This introduces a bulk_add_metadata() function along with wrapper functions for post, user, site, term, and comment meta. This function accepts an array of key value pairs for inserting metadata in bulk. The metadata is inserted using a single database query and a single flush of the term cache, which is more performant than multiple calls to add_{type}_meta(). The performance benefit increases linearly as the number of meta fields increases.

Example

 $meta = [
     'key1' => 'value1',
     'key2' => 'value2',
     'key3' => 'value3',
 ];

- foreach ( $meta as $key => $value ) {
-     add_user_meta( $user_id, $key, $value );
- }
+ bulk_add_user_meta( $user_id, $meta );

Performance

The below table compares the peak memory usage and time taken reported by PHPBench to add various sets of data using multiple calls to add_post_meta() versus one call to bulk_add_post_meta(). Absolute times vary depending on the system being used to run the tests (eg. my local Docker environment is slower than the GitHub Actions workflow runs), but the relative changes remain quite consistent.

Various data sizes have been tested and can be seen in the provideMetaData() method in the tests/phpbench/BulkAddMetaBench.php file. PHPBench is configured to run each test set 30 times and report the peak memory usage and the mode time taken.

Memory usage remains unchanged in the majority of cases. It peaks somewhat at the extreme end for multiple extra large data sets where meta values are between 10kb and 200kb in size. I have yet to investigate what causes this, but it's not a big concern.

Timing decreases significantly for all test cases, as expected.

set mem_peak mode
3 metas, 10-30 bytes 30.383mb -0.00% 928.586μs -61.37%
20 metas, 10-200 bytes 30.393mb -0.00% 1.401ms -91.85%
50 metas, 5-250 bytes 30.405mb -0.00% 2.018ms -95.30%
100 metas, 5-500 bytes 30.688mb -0.00% 3.642ms -96.11%
10 metas, 0.1-1 Kb 30.396mb -0.00% 1.193ms -86.82%
10 metas, 1-10 Kb 30.506mb -0.00% 2.393ms -86.66%
3 metas, 30-90 Kb 30.934mb +1.99% 4.162ms -51.67%
10 metas, 10-100 Kb 32.193mb +9.70% 10.563ms -62.09%
20 metas, 10-200 Kb 37.426mb +35.22% 36.750ms -45.43%

Notes

  • The PHPBench code and its workflow won't be included when this gets committed. It's there to demonstrate the performance improvements while working on this PR.
  • Naming is hard. Perhaps add_multiple_metadata() is a better name, there are a few other functions in core that use multiple in their name.
  • The same actions and filters are fired as get fired when calling add_{type}_meta() multiple times:
    • add_{$meta_type}_metadata is called once for every key
    • add_{$meta_type}_meta and added_{$meta_type}_meta are called once for every key when the above filter hasn't short-circuited the corresponding value for a key

johnbillion avatar Sep 01 '23 22:09 johnbillion