ElasticPressLabs icon indicating copy to clipboard operation
ElasticPressLabs copied to clipboard

ep_user_pre_query_db_results per page?

Open yarovikov opened this issue 2 months ago • 3 comments

Describe your question

Hi there

Trying new filter ep_user_pre_query_db_results

Any chance to set ep_indexing_last_processed_object_id for correct sync? Or other way to resolve the problem with sync?

add_filter('ep_user_pre_query_db_results', function($results, $args) {

error_log(print_r($args, true));

$users = [...] // my array of ids

return [
            'objects' => $users,
            'total_objects' => count($users),
        ];
});

LOG

[10-Nov-2025 09:49:09 UTC] Array
(
    [number] => 300
    [offset] => 0
    [orderby] => ID
    [order] => desc
    [ep_indexing_advanced_pagination] => 1
    [per_page] => 300
    [ep_sync_id] => 6911b5159bbbd
)

[10-Nov-2025 09:49:23 UTC] Array
(
    [number] => 300
    [offset] => 0
    [orderby] => ID
    [order] => desc
    [ep_indexing_advanced_pagination] => 1
    [per_page] => 300
    [ep_sync_id] => 6911b523a628a
    [ep_indexing_last_processed_object_id] => 4395918 // this
)

[10-Nov-2025 09:49:40 UTC] Array
(
    [number] => 300
    [offset] => 0
    [orderby] => ID
    [order] => desc
    [ep_indexing_advanced_pagination] => 1
    [per_page] => 300
    [ep_sync_id] => 6911b534b0e31
    [ep_indexing_last_processed_object_id] => 4395918  // the same
)

Elasticsearch 9.2.0 ElasticPress 5.3.1 ElasticPress Labs 2.5.0

Thanks

Code of Conduct

  • [x] I agree to follow this project's Code of Conduct

yarovikov avatar Nov 10 '25 10:11 yarovikov

Probably this is not needed

https://github.com/10up/ElasticPressLabs/blob/develop/includes/classes/Indexable/User/User.php#L675-L677

if ( isset( $args['include'] ) || isset( $args['exclude'] ) || 0 < $args['offset'] ) {
  $args['ep_indexing_advanced_pagination'] = false;
}

yarovikov avatar Nov 10 '25 14:11 yarovikov

Hi @yarovikov,

The idea behind the ep_user_pre_query_db_results filter is that it returns a limited number of users, allowing ElasticPress to sync all of them in one go. In this case, pagination isn’t required. However, I believe in your situation, the number of users is much higher. Could you please share approximately how many users are being returned by the ep_user_pre_query_db_results filter?

Regards, Burhan

burhandodhy avatar Nov 11 '25 09:11 burhandodhy

Hi @burhandodhy

For example i need ~9000 users of ~1200000

I'v fixed my problem adding some code from the plugin

add_filter('ep_user_pre_query_db_results', function($results, $args) {
        // ... some code

        // the part from the plugin
        $offset = (int) $args['offset'];
        $number = (int) $args['number'];

        $placeholders_exclude = implode(',', array_fill(0, count($exclude_ids), '%d'));
        $placeholders_types = implode(',', array_fill(0, count($post_types), '%s'));

        $requested_lower_limit_id = $args['ep_indexing_lower_limit_object_id'] ?? 0;
        $requested_upper_limit_id = $args['ep_indexing_upper_limit_object_id'] ?? PHP_INT_MAX;
        $last_processed_id = $args['ep_indexing_last_processed_object_id'] ?? null;

        $upper_limit_range_id = is_numeric($last_processed_id) ? $last_processed_id - 1 : $requested_upper_limit_id;

        $range = [
            'u.ID <= ' . (int) $upper_limit_range_id,
            'u.ID >= ' . (int) $requested_lower_limit_id,
        ];

        $where_clause = implode(' AND ', $range);

       // ....
        
       $users = $wpdb->get_results($sql);

        return [
            'objects' => $users,
            'total_objects' => (int) get_option('total_engineers_count'),
        ];
});

yarovikov avatar Nov 11 '25 09:11 yarovikov