craft-scout

craft-scout copied to clipboard

Odd question, but we have all of our sections in one large index, but our Solspace Calendar events are a different element type. Since I can't add to an index more than once apparently, what are my options, if any for merging craft\elements\Entry and Solspace\Calendar\Elements\Event into one index?

I haven't tested this, but I think you could use craft\base\Element as the element type, you'll have to experiment a bit with the criteria then to get everything in there.

With the new feature in 1.2.1 you can also just do your checks in the transformer and if you don't want to index them return an empty array.

Okay, I'll give it a look and see if anything can come of it!

I am a bit late to this. There was a way to map different elementTypes to the same index. You would just map to the same index name. It worked for me.

I recently started upgrading to the new version and have noticed that the unintended feature actually does not work in v2.

@riasvdv there is no way one could use the same index name, but use two different element types? Say one for categories and another for entries?

You should be able to set craft\base\Element as the ElementType, you'll then get an ElementQuery in your criteria on which you can set some extra constraints, then in your transformer you should receive a craft\base\Element object, where you can check which element you got back and how to transform it.

I haven't tested any of this yet, but I don't see a reason why it couldn't work

@riasvdv thank you for the reply.

I saw your comment earlier on about using craft\base\Element but unfortunately that doesn't make a whole lot of sense to me.

Do you have any resources you could share? Not resources to Craft, but how to write it inside of the index configuration? I am not familiar with the -> writing where you chain functions.

My understanding is I define the elementType and then the criteria is based off of that elementType. But if I wanted to do say a category and entry elementType, that is where I am confused.

@riasvdv I just wanted to let you know that I tested using the craft\base\Element class, however it's an abstract class and therefore fails when the transformer tries to instantiate it.

Didn't realise you couldn't start a query from the base Element class, I'll reopen this, but I don't know if I'll be able to find a solution for this.

As a (temporary) solution, Algolia does allow searching multipe indices at the same time: https://www.algolia.com/doc/api-reference/api-methods/multiple-queries/

@riasvdv Is there a good reason for validating the index names are unique? I have config for each section based on my criteria that I am passing over that configures the values going into the index. I don't really think the unique check needs to happen, and I can't think of any negative side effects if it were to be removed.

If you think removing it will have no negative side effects, and you can add some tests that prove multiple index configurations do the expected things, I'd gladly accept a PR that implements this.

It would be nice if you could use CraftCMS manually and pass in an array or something, rather then using elementType, criteria, etc. So something like

$getDataFromCraftCMSYourself = \craft...

$indices[] = \rias\scout\ScoutIndex::create('index')->data($getDataFromCraftCMSYourself)->transformer(function ($data) {} );

Or something?

Obviously this would be a high/advanced user not the default, but this would allow us to basically do anything.

Found myself in the same scenario after needing to implement a global search that included both craft\elements\Entry and Solspace\Calendar\Elements\Event.

I got around Scout's validateConfig method by installing cweagans/composer-patches and using this patch:

diff -rupN a/src/Scout.php b/src/Scout.php
--- a/src/Scout.php	2020-01-27 12:13:35.148078307 -0500
+++ b/src/Scout.php	2020-01-27 12:14:38.294046990 -0500
@@ -133,6 +133,7 @@ class Scout extends Plugin
 
     private function validateConfig()
     {
+        return;
         $indices = $this->getSettings()->getIndices();
 
         if ($indices->unique('indexName')->count() !== $indices->count()) {

@riasvdv maybe you could add a validate_config boolean property that defaults to true in Scout's conifg?

@nikolowry

Are you able to provide a full example of how you patched and or what needs to be done? It looks like you copied some git diff in your messagfe.

@besimhu a patch is produced by diffing. To replicate what I did:

• composer require cweagans/composer-patches
• Save the patch contents I posted above in your project's root at patches/Scout-Indices-No-Validation.patch
• In your composer.json add the following to extra:

{
  "extra": {
    "patches": {
      "rias/craft-scout": {
        "No Scout Indices Validation": "patches/Scout-Indices-No-Validation.patch"
      }
    }
  }
}

• Now any time you run composer install or composer update, cweagans/composer-patches will automatically apply the patch to Scout. You can run composer update nothing to manually trigger it for the first time

@nikolowry thank you, it worked as you described and I was able to finally upgrade to the latest version.

@riasvdv what type of tests would you need to convince you multiple elements on one index works as expected?

I've been indexing two elements -- Entries and Solspace\Calendar\Event -- on the same indices in two separate environments (dev/stage) for over a month without an issue.

Or thoughts on adding a boolean config setting to allow users to index multiple elements on one index? You could explicitly state it's an unsupported/experimental feature and devs should use at their own risk.

I'm advocating for this because full-site search is such a core functionality of most CMS's and Craft is refusing to implement, https://github.com/craftcms/cms/issues/878.

Using Scout with the patch above, along with https://github.com/trendyminds/algolia, was the only way I could pull off Craft's traditional pagination in Twig.

I agree with @nikolowry ... I am relying on the instantsearch.js and that does not allow for combining multiple indecies. I have some content that falls under categories, and while they are categories, they do contain website content. So when I am wanting to do a website search, I need to include those as part of the website index.

@timkelty thoughts?

Reading through this, my inclination is to keep indexName unique, but allow users to manually tap into a getElements method, like @verbeeksteven https://github.com/riasvdv/craft-scout/issues/69#issuecomment-548764036 suggested.

So for @onebrightlight it might look something like this:

ScoutIndex::create()
    ->indexName('myIndex')
    ->getElements(function() {
      $entries = craft\elements\Entry::find(…);
      $events = Solspace\Calendar\Elements\Event::find(…);

      return array_merge($entries, $events);
    })
    ->transform(function (craft\base\Element $element) {
            'lorem' => $element->lorem,
            'ipsum' => $element->ipsum,
        ];
    });

It seems like this would fit people's use-cases, while maintaining the 1-1 relationship of ScoutIndex to Algolia index.

@timkelty

I'm confused: is this already implemented or will this be implemented in the future?

@martinhellwagner not yet implemented, but thinking it should be a quick addition.

We're running into the same issue. A solution as described in https://github.com/riasvdv/craft-scout/issues/69#issuecomment-616563983 would be a very welcome feature! ❤️

I also would love this functionality. Any progress or update on it?

@elivz No progress, but I'm really hoping to carve out some time for some long standing Scout issues in the coming week.

Does what I outline here fit your use-case? https://github.com/riasvdv/craft-scout/issues/69#issuecomment-616563983

@timkelty Yes! I think being able to pass in an array of items in place of the ElementType/Criteria query rules would open up a lot of possibilities. Not just combining different element types, but potentially even indexing non-Element data from a raw SQL query or pulling from an external API or whatever (not sure if that would cause other complications & I currently have no need for it...just thinking).

Before I start implementing a workaround for this, just wanted to check & make sure you're just just about to release an official solution 😁

@elivz pinged you on Discord

I am also curious about the status of this. @timkelty your proposed solution above looks like it would work great for my use case. Any plans to implement?

I also need to combine entries and categories into the same index.

@timkelty did you make any progress with the following feature? https://github.com/riasvdv/craft-scout/issues/69#issuecomment-616563983

Looking forward to use it!