MageOS 1.0.4 is not compatible with ES 7, and shoudl drop ES7 modules and support

[ProxiBlue]

Summary

Building a new project, using MageOS 1.0.4, I ran into the problem that certain category views would fail. This was duplicated in a clean luma 2.4.7 install with sample data, as well as a clean mageOS 1.0.4 install with sample data.

Exception #0 (Elasticsearch\Common\Exceptions\BadRequest400Exception): {"error":{"root_cause":[{"type":"x_content_parse_exception","reason":"[1:625] [bool] failed to parse field [must]"}],"type":"x_content_parse_exception","reason":"[1:625] [bool] failed to parse field [filter]","caused_by":{"type":"x_content_parse_exception","reason":"[1:625] [bool] failed to parse field [must]","caused_by":{"type":"illegal_argument_exception","reason":"field name is null or empty"}}},"status":400}

The issue can be tracked down to the fact that 2.4.7 (1.0.4) uses 'must' in the query, and that is an ES 8 convention

So, this is not at all compatible with ES7, and ES 8 is required.

However, once one upgrades ES to v8, a new error is presented:

Catalog Search indexer process unknown error:
{"error":"no handler found for uri [/magento2_product_1_v3/document/_mapping?include_type_name=true] and method [PUT]"}

This can be fixed by setting magento to use 'elasticsearch8' as the needed engine, BUT, that is not possible out-the-box.

Only ES 7 modules are included on base install, and one is required to manually install the ES 8 packages via composer. This in itself presented a new issue, in the fact that there seems to no mageOS version of the needed package (I know this will be easy to solve, and will need to be solved for the proposed solution)

composer require mage-os/module-elasticsearch-8

results in:

 [InvalidArgumentException]                                                                                                                                                                                      
  Could not find a matching version of package mage-os/module-elasticsearch-8. Check the package spelling, your version constraint and that the package is available in a stability which matches your minimum-s  
  tability (stable).       

so, to get around this for now, I had to add magento repos back into composer

"magento": {
            "type": "composer",
            "url": "https://repo.magento.com/",
            "only": [
                "magento/module-elasticsearch-8"
            ]
        }

however, this then presented a whole new problem:

Your requirements could not be resolved to an installable set of packages.

  Problem 1
    - Only one of these can be installed: mage-os/module-elasticsearch-7[1.0.4], magento/module-elasticsearch-8[100.4.0, 100.4.1]. They all replace magento/module-elasticsearch-7 and thus cannot coexist.
    - mage-os/product-community-edition 1.0.4 requires mage-os/module-elasticsearch-7 1.0.4 -> satisfiable by mage-os/module-elasticsearch-7[1.0.4].
    - mage-os/product-community-edition is locked to version 1.0.4 and an update of this package was not requested.
    - Root composer.json requires magento/module-elasticsearch-8 ^100.4 -> satisfiable by magento/module-elasticsearch-8[100.4.0, 100.4.1].

which then requires one to replace the magento/module-elasticsearch-7 module in composer

"replace": {
        "mage-os/module-elasticsearch-7": "*"
    },

and then only can one install the needed ES 8 module, and then setup the needed config to use, and overcome the error

This is extremely convoluted to get ES 8 to work.

Examples

See description

Proposed solution

I propose that the module mage-os/module-elasticsearch-7 be removed permanently from mageOS-1.0.4+ and replaced with the v8 version.

End users should not need to go through the whole issue. It should work out-th-box with ES8, as that is the minimum requirement There should be no need to keep the ES7 module code in the system. It is defunct.

Release note

No response

Triage and priority

• [ ] Severity: S0 - Affects critical data or functionality and leaves users without workaround.
• [X] Severity: S1 - Affects critical data or functionality and forces users to employ a workaround.
• [ ] Severity: S2 - Affects non-critical data or functionality and forces users to employ a workaround.
• [ ] Severity: S3 - Affects non-critical data or functionality and does not force users to employ a workaround.
• [ ] Severity: S4 - Affects aesthetics, professional look and feel, “quality” or “usability”.