Synchronize replicas's settings locally with scout:sync

[ghost]

We can define the index settings in its dedicated settings file, but there is no way to define replicas's settings. (at least not in documentation).

This following is example of Article Model setting which was generated by php artisan scout:optimize. The following settings are synced with remote configuration when I run php artisan scout:sync.

// conifg/scout-articles.php
<?php
return [
    'searchableAttributes' => [ 'title',  'content', ],
    'customRanking' => ['desc(published_at)'],
    'removeStopWords' => null,
    'disableTypoToleranceOnAttributes' => ['title', 'description'],
    'attributesForFaceting' => null,
    'unretrievableAttributes' => null,
    'ignorePlurals' => null,
    'queryLanguages' => null,
    'distinct' => null,
    'attributeForDistinct' => null,
    'replicas' => ['articles_asc', 'articles_desc'],
];

It would be nice to have similar settings for replicas locally.

@nunomaduro

[rgehan]

That would be nice, especially when you have many replicas and you don't want to forget updating one of them.

[chrisbbreuer]

I would love to see something like this introduced, too. Seeing that Nuno added the "enhancement" tag, is there any sort of timeline for this feature?

[nunomaduro]

No ETA on this feature, @Chris1904 do you have a proposition for this feature?

[chrisbbreuer]

Just this weekend I dove into this package and remember from using it before, that replicas can't be controlled via version control yet. I have been collecting a few ideas to improve this package which I would gladly share soon (dynamic index names, maybe allowing for utilizing _tags natively), but need to fully understand the Algolia ecosystem before I'd waste anyone's time.

At this moment, I am not fully sure yet how exactly my Algolia data structure will look like and how much I will rely on replicas etc.. Currently, it's manageable with just 2 replicas.

Regarding this feature, with my current knowledge and for ease of use, I would add replicas to the config array, e.g.

// config/scout-articles.php
'replicas' => [
    'articles_asc' => [
        'customRanking' => 'asc(published_at)',
        'some_other_allowed_config' => 'etc.'
    ],
    'articles_desc' => [
        'customRanking' => 'desc(published_at)',
        'some_other_allowed_config' => 'etc.'
    ],
    // etc.
],

This way it would allow for making sure replicas then stay in sync with your codebase (very useful if you have several devs in your team and you're using prefixes for your indices) and you the config options are exposed there as well.

[felixwatzka]

Are there any considerations yet to implement this feature to scout-extended? This actually existed in the older algoliasearch-laravel package, see: https://github.com/algolia/algoliasearch-laravel#primaryreplica

I'm currently working on a project where we need 20 or more virtual replicas for different sortings. Additionally, every environment has its own indices. This means that we have to manually set the settings for 20 * (num of envs) replicas via the UI. So this feature would be amazing to have!

[DevinCodes]

Unfortunately there's currently no plan to work on this in the near future from our side, but we're definitely open for contributions, as this can be a valuable addition to Scout Extended!

[JamesHemery]

I put in place a temporary solution, the time to do a PR to have a permanent solution.

Create a command:

<?php

namespace App\Console\Commands;

use Algolia\ScoutExtended\Facades\Algolia;
use Illuminate\Console\Command;

class SyncAlgoliaReplicas extends Command
{
    protected $signature = 'scout:sync-replicas';

    protected $description = 'Sync Algolia replicas.';

    public function handle()
    {
        $indexes = config('scout-replicas');
        $client = Algolia::client();

        foreach ($indexes as $indexKey => $replicas) {
            $index = $client->initIndex($indexKey);
            $indexConfig = config("scout-$indexKey", []);

            $index->setSettings([
                ...$indexConfig,
                'replicas' => array_keys($replicas)
            ]);

            foreach ($replicas as $replicaKey => $replicaConfig) {
                $replica = $client->initIndex($replicaKey);
                $replica->setSettings([
                    ...$indexConfig,
                    ...$replicaConfig
                ]);
            }
        }
    }
}

And create a config file for replicas:

<?php

return [
  'ads' => [
      'ads_created_at_asc' => [
          'customRanking' => ['asc(created_at)']
      ],
      'ads_created_at_desc' => [
          'customRanking' => ['desc(created_at)']
      ],
  ],
];

I didn't put the replicas in the scout- config file because it gets overwritten during the remote sync.

[goellner]

I put in place a temporary solution, the time to do a PR to have a permanent solution.

Create a command:

<?php

namespace App\Console\Commands;

use Algolia\ScoutExtended\Facades\Algolia;
use Illuminate\Console\Command;

class SyncAlgoliaReplicas extends Command
{
    protected $signature = 'scout:sync-replicas';

    protected $description = 'Sync Algolia replicas.';

    public function handle()
    {
        $indexes = config('scout-replicas');
        $client = Algolia::client();

        foreach ($indexes as $indexKey => $replicas) {
            $index = $client->initIndex($indexKey);
            $indexConfig = config("scout-$indexKey", []);

            $index->setSettings([
                ...$indexConfig,
                'replicas' => array_keys($replicas)
            ]);

            foreach ($replicas as $replicaKey => $replicaConfig) {
                $replica = $client->initIndex($replicaKey);
                $replica->setSettings([
                    ...$indexConfig,
                    ...$replicaConfig
                ]);
            }
        }
    }
}

And create a config file for replicas:

<?php

return [
  'ads' => [
      'ads_created_at_asc' => [
          'customRanking' => ['asc(created_at)']
      ],
      'ads_created_at_desc' => [
          'customRanking' => ['desc(created_at)']
      ],
  ],
];

I didn't put the replicas in the scout- config file because it gets overwritten during the remote sync.

I tried to use your code, but keep getting This index is a replica, you cannot set replicas parameter. Does this workaround still work for you?

[goellner]

Nevermind, i added my replicas in the scout-xxx.config and had to remove this first:

$settings = [
    ...$indexConfig,
    ...$replicaConfig,
];
// remove replicas from settings array, else the algolia api throws an error
unset($settings['replicas']);
$replica->setSettings($settings);