Add Arch Linux artifacts

[hashworks]

Description

Adds artifacts for the official Arch Linux OpenSearch packages.

As discussed with @stockholmux by mail.

The Arch Linux packages I created do stuff a bit differently than the archives you normally provide. The opensearch package is minimal by default, without any plugins. Those are available as additional packages. Therefore, I would provide the installation instructions like this:

Same goes for OpenSearch Dashboards:

If you think this doesn't suit you we can provide the Arch Linux platform on the "Minimal Distributions" list only.

[inglor]

I don't this older versions should be included here. Only the version in the official repositories (the latest) is the one Arch Linux will support.

[hashworks]

That sounds reasonable. I guess every merge request should remove the old (unsupported) version. Since we then provide only one version we should also link to the package pages.

[stockholmux]

@hashworks This amazing! I'm working on some refactoring of the downloads page so I'm holding on merging this one for a little bit (merging makes it go live in the deployment pipeline).

[nateynateynate]

Hi @hashworks - first, I wanted to thank you for your contribution! I was quite pleased to see this PR come in. I was wondering if you could help me here.

I'm going through an install on my own Archlinux box just for a sanity check, and I didn't have any luck getting dashboards to run (it installed fine). I kept getting this error:

When clicking through the dependencies at https://archlinux.org/packages/community/x86_64/opensearch-dashboards/, I noticed that it has a requirement for https://archlinux.org/packages/community/x86_64/nodejs-lts-erbium/ which is at 12.22.9, but OpenSearch Dashboards requires 12.22.8 to run.

Are you seeing this same issue at all?

[hashworks]

I'm sorry, this must have slipped by me. Please try out the new 1.2.0-2.

Since we only package the LTS versions of node and current (thus, minimum 12.22.9) we don't provide the currently listed 10.24.1. Since in my tests v12 worked just fine and there are plans to move to v14 I replaced the version requirement in the package.json with our current nodejs-lts-erbium version. This works fine for builds, but may fail in runtime since the nodejs-lts-erbium patch version (8 -> 9) might change and node_version_validator.js checks for the exact version.

For now, I've removed require('./node_version_validator'); (and thus the runtime check). This shouldn't be required once v14 is used, but I would still advise to only check major and minor versions, not patch versions, like:

var isVersionValid = requiredVersion.match(/^v(\d+\.\d+)/)[1] === currentVersion.match(/^v(\d+\.\d+)/)[1]

But let's discuss that in another issue.

Do you want people to be notified about the adjusted NodeJS version on the download page?

[nateynateynate]

Thanks for the update @hashworks.

I think we might have an opportunity for collaboration here.

We'd really love to see the post-install experience to match what is currently distributed by the project, but I can't help but notice a bit of divergence.

It might be good to explore together how we can fill in the gaps for the Arch users so they can get up and running in a way that matches the OpenSearch documentation more closely in the areas around security and configuration. Glad to work with you on this or hear your thoughts on what would be idiomatic for Arch users.

[hashworks]

I can't help but notice a bit of divergence

If you list the divergences you noticed I would happily address them or provide reasoning for the adjustments :)

Glad to work with you on this or hear your thoughts on what would be idiomatic for Arch users.

I guess the basic idiom is simplicity. Arch users expect the installed packages to be as close to upstream as possible without unnecessary additions or modifications, including configuration files. All adjustments should be distro related, not something the packager "prefers". We also want to ensure flexibility – if users decide that they don't need a security plugin they should be able to do so, which is why the plugins are provided as separate packages.

Therefore, the upstream configuration file is used without any modifications. For opensearch-dashboards a single adjustment is made since we expect users to run it using the provided systemd service.

sed -r 's|#(logging.quiet:) false|\1 true|' -i config/opensearch_dashboards.yml

[stockholmux]

@hashworks OK. I get where you're coming from with regards to packaging & thanks for explaining the Arch philosophy.

I think the rub is with the packaging - individual plugins vs together as a single unit. Currently, OpenSearch does distribute a -min package but with a serious warning before download. And the plugins aren't distributed individually on the downloads page. This is intentional. From the project perspective, the way this is split up is largely historical and will likely change radically at version 2 and through version 3. So, that leads Arch users to be put at some pretty serious disadvantages:

1. Most of the documentation will not apply to those who install OpenSearch through Arch. The documentation assumes SSL and authentication, as an example.
2. As the project changes and the constituent units (plugins), which are now represented as separate packages on Arch, will go away or change function, so if someone is using Arch and expecting their install scripts to behave the same way in, say, 6-8 months, they're going to be pretty disappointed.

Point 1 is easier to bridge. One thing that comes to mind is making the downloadable artifact on the website a script that uses pacman to install and setup an equivalent experience to what someone would get if they installed from the .tar.gz or using docker. Someone could independently install OpenSearch with pacman but at least if they get something from the opensearch.org download page, it will be like-for-like. This is something I'm sure I can find some folks to collaborate on.

Point 2 is tougher. While breaking changes happen all the time across a distro - I'd anticipate the combination of how OpenSearch is broken up now vs where it's heading causing more headaches that love for arch users. I'd be very interested how there could be collaboration in making this a smooth transition for everyone.

Thoughts on any of this?

[hashworks]

Sorry for the silence, I've been a bit busy.

One thing that comes to mind is making the downloadable artifact on the website a script that uses pacman to install and setup an equivalent experience to what someone would get if they installed from the .tar.gz or using docker.

I don't think this is something Arch users expect. I also suspect most will refer to the official documentation for specific needs, not for an installation guide.

However, I get your point – if we want to unify the experience I guess it would be best to install all plugins when one calls pacman -Sy opensearch. One way to do this would be a meta package (like base), which only has dependencies - in our case opensearch-min and all plugins. This way the experience would be streamlined and the alternative (-min and only selective plugins) is still available. If a manual install "breaks" in later releases that would be fine, since it isn't the "recommended" way.

But: Doing so now might break existing installations. If someone isn't using the security plugin, installing it now would break their setup:

Feb 17 14:28:36 hive.hashworks.net opensearch[1814058]: java.lang.IllegalStateException: failed to load plugin class [org.opensearch.security.OpenSearchSecurityPlugin]
Feb 17 14:28:36 hive.hashworks.net opensearch[1814058]: Likely root cause: OpenSearchException[plugins.security.ssl.transport.keystore_filepath or plugins.security.ssl.transport.server.pemcert_filepath and plugins
.security.ssl.transport.client.pemcert_filepath must be set if transport ssl is requested.]

The .tar.gz installation method solves this by providing a script, which sets up an initial configuration – again, not something Arch users would use or expect. I wonder if it would make sense for the security plugin to just disable TLS when it isn't configured and provide a warning instead.

Do you think it would be reasonable to only provide an Arch option in the "OpenSearch Minimum" section? Alternatively I could add a warning to the current state.

[dblock]

@peternied WDYT ^?

[peternied]

First off - this is amazingly cool!

I wonder if it would make sense for the security plugin to just disable TLS when it isn't configured and provide a warning instead.

I'm not sure if disabling the feature is correct vs making TLS configuration have better defaults; however, this is a great issue for us to tackle. I've created https://github.com/opensearch-project/security/issues/1631

One way to do this would be a meta package

I like this idea, a logical framework that can be consistent even if we reshape the 'physical' layout of plugins.

[dblock]

Can this be merged (soon)? We're talking about Arch Linux in https://github.com/opensearch-project/project-website/pull/670

[hashworks]

One way to do this would be a meta package

Since #1631 has been closed, I won't add such a drastic change (which might break current installations). Any other way we could merge this? Adjustments to the text, additional warnings, wiki-notes, …

[hashworks]

What should we do with this PR? If this will never be merged I can stop adding the new versions.

[krisfreedain]

Hello @hashworks - I've recently taken over the site maintenance from @stockholmux and need to look into this. OK if I read through and work to get back to you soon? Your work on this is greatly appreciated.

[hashworks]

OK if I read through and work to get back to you soon?

Sure!

[bbarani]

Anyone looking in to this PR? @krisfreedain @stockholmux

[nateynateynate]

Can I get an assist on whomever is involved here on what the next steps are? @dblock, @peternied, @hashworks, @bbarani ?

It would seem that Arch Linux has some requirements for their packages being as minimal as possible. I'd sure like to get this one merged some day.

[dblock]

Apologies for the ... delay. Looks like archlinux artifacts continue being released (awesome!) but we are still failing to publish links to the on the website. @krisfreedain could you please take a look?

[Catch All Triage - Attendees 1, 2, 3, 4, 5]

[krisfreedain]

@nateynateynate @kaimmej - can we look at this one this week? I think we're at a place where this distro needs to be included in the download each time - looks like the team is doing a build each time - thanks

[nateynateynate]

https://github.com/opensearch-project/project-website/pull/3107

[nateynateynate]

Ok it looks like you're live from #3117 Enjoy! :-)