How do you set index_analyzer and search_analyzer ?

[twisted1919]

Hi guys,

The example given at http://elastica.io/getting-started/storing-and-indexing-documents.html:

...
mapping = new \Elastica\Type\Mapping();
$mapping->setType($elasticaType);
$mapping->setParam('index_analyzer', 'indexAnalyzer');
$mapping->setParam('search_analyzer', 'searchAnalyzer');
...

Doesn't work anymore, if i try it i get:

Exception 'Elastica\Exception\ResponseException' with message 'Root mapping definition has unsupported parameters:  [index_analyzer : indexAnalyzer] [search_analyzer : searchAnalyzer] [_boost : {name=_boost, null_value=1}]'

Any hints on this? I have been stuck with an ancient version of this library and elastic search and now i finally got the chance to upgrade and things break :)

[kersten]

This one is also not working:

$mapping->setParam( '_boost', array( 'name' => '_boost', 'null_value' => 1.0 ) );

[ruflin]

I remember there were some changes to analyzer in elasticsearch but I would the example still expect to work. What did the creation of the index return?

@twisted1919 Can you share the full code? @kersten What is the error message you get?

Didn't have time yet to test the example myself.

[twisted1919]

@ruflin - here's my current code:

class JobSchema 
{
    public static function build()
    {
        $client = elastica()->getClient();
        $index  = $client->getIndex(Job::index());
        $force  = true;

        $index->create([
            'number_of_shards'   => 4,
            'number_of_replicas' => 1,
            'analysis' => [
                'analyzer' => [
                    'indexAnalyzer' => [
                        'type'      => 'custom',
                        'tokenizer' => 'standard',
                    ],
                    'searchAnalyzer' => [
                        'type'      => 'custom',
                        'tokenizer' => 'standard',
                    ],
                    'stringLowercase' => [
                        'type'      => 'custom',
                        'tokenizer' => 'keyword',
                        'filter'    => 'lowercase'
                    ],
                    'autocomplete' => [
                        'type' => 'custom',
                        'tokenizer' => 'standard',
                        'filter' => ['standard', 'lowercase', 'stop', 'kstem', 'myNgram'],
                    ],
                ],
                'filter' => [
                    'myNgram' => [
                        'type' => 'ngram',
                        'min_gram' => 3,
                        'max_gram' => 15
                    ],
                ],
            ],
        ],
            $force
        );

        $type = $index->getType(Job::type());
        $mapping = new \Elastica\Type\Mapping();
        $mapping->setType($type);
        // $mapping->setParam('index_analyzer', 'indexAnalyzer');
        // $mapping->setParam('search_analyzer', 'searchAnalyzer');

        // Define boost field
        // $mapping->setParam('_boost', ['name' => '_boost', 'null_value' => 1.0]);

        // mappings
        $mappingProperties = [];

        // these are used for display purposes only
        $job = new Job();
        foreach ($job->attributes() as $attr) {
            $mappingProperties[$attr] = [
                'type'           => 'string',
                'include_in_all' => true,
            ];
        }

        // these are used for search only.
        $mappingProperties = array_merge($mappingProperties, [
            'searchPosition'    => [
                'type'              => 'string',
                'include_in_all'    => true,
                'analyzer'          => 'stringLowercase',
            ],
            'searchDescription' => [
                'type'              => 'string',
                'include_in_all'    => true,
                'analyzer'          => 'stringLowercase',
            ],

            '_boost'  => [
                'type'              => 'float',
                'include_in_all'    => false
            ],
        ]);

        // Set mapping properties
        $mapping->setProperties($mappingProperties);

        // Send mapping to type
        $mapping->send();
    }
}

Everything works now. If i uncomment the setParam calls, then i get the error above. Let me know if you need anything from me.

[kersten]

@ruflin this is my error:

Root mapping definition has unsupported parameters: [_boost : {name=_boost, null_value=1}]

[ruflin]

@kersten I digged into the issue for the _boost field and mapping, and the answer is quite simple: It is not supported anymore in 2.x: https://www.elastic.co/guide/en/elasticsearch/reference/2.x/mapping-boost-field.html We need to remove that from the docs.

[ruflin]

@twisted1919 Since elasticsearch 2.x it is not possible anymore to set default analyzer per type, only per index: https://www.elastic.co/guide/en/elasticsearch/reference/current/analysis-analyzers.html So also this example is unfortunately out of date :-(

[Numkil]

Hey @ruflin. Since those mapping parameters are not supported anymore what is the current way of using your analyzers then? Naming them "default" and "default_search" when creating the index?

Edit: Using those names seems to create the exact same results as using the now deprecated parameters did so I guess I was right about that.

[ruflin]

@Numkil As far as I understand your comment, with the above changes it is working as expected?

[Numkil]

Yes it does. It was a bit confusing because before I could name the analyzers whatever I wanted and then use the mentioned mappings to define which analyzer was used to index everything and which one was used to search the index.

Now I just have to make sure they get named correctly("default", "default_search") so ElasticSearch can guess which one he needs to use for both situations.

Thanks for asking anyway.

[ruflin]

@Numkil As a next step, do you suggest to "only" update the guide or additional changes?

[Numkil]

I'll make a PR in the weekend for the documentation page "Storing-and-indexing-documents". I'm pretty sure most people will be helped by this small change.

Some sort of analysis interface in the code would be cool but I don't know how much extra value it would add to the code. Something so you could do something like this maybe? This way you could enforce good practices and maybe hide some of the more "magic" like things in elasticsearch like how the name of your analyzer can alter the results of your indexing/queries.

$analysis = new \Elastica\Analysis();
$analysis->setNumberOfShards(4);
$analysis->setNumberOfReplicas(1);
//Do stuff here with your analysis to create/configure analyzers

// Load index
$elasticaIndex = $elasticaClient->getIndex('twitter');

// Create the index new
$elasticaIndex->create($analysis, true);

[ruflin]

I definitively it would not hurt to extend the Mapping with an Analysis object to make the handling easier. I haven't check in detail yet what the best way would be but it is definitively worth investigating.

Looking forward to the docs PR. Thanks.

[veloper]

Adding more info to what @Numkil said about setting default analyzers:

I found this SO question to be helpful: http://stackoverflow.com/questions/34894671/set-default-analyzer-of-index