wazuh-packages
wazuh-packages copied to clipboard
wazuh
Include ARM support to indexer and dashboard
Hello all,
Description
We need to include a procedure and scripts to build ARM packages from indexer and dashboard base package. Some users are requesting ARM support for our Wazuh indexer and dashboard products. We want to include a way to manage the ARM package build and include an automatic procedure to build and release them.
Tasks
- [ ] Research ARM OpenSearch packages
- [ ] Build ARM OpenSearch base for Wazuh indexer
- [ ] Build ARM OpenSearch base for Wazuh dashboard
- [ ] Build ARM package for Wazuh indexer
- [ ] Build ARM package for Wazuh dashboard
- [ ] Install in testing environment
- [ ] Research about Filebeat options
- [ ] Apply Filebeat solution
- [ ] Generate ARM Filebeat packages
Validation
- [ ] Base builds correctly
- [ ] Package builds correctly
- [ ] Package installs
- [ ] Package uninstalls
- [ ] Package purge
- [ ] Package upgrade
- [ ] Test in recommended systems
- [ ] Amazon Linux 2
- [ ] CentOS 7
- [ ] CentOS 8
- [ ] Red Hat Enterprise Linux 7
- [ ] Red Hat Enterprise Linux 8
- [ ] Red Hat Enterprise Linux 9
- [ ] Ubuntu Xenial
- [ ] Ubuntu Bionic
- [ ] Ubuntu Focal
- [ ] Ubuntu Jammy
- [ ] Check Filebeat connectivity and right working
I found a bug in the Debian 8 repositories for arm, the build worked with Debian 9, I have to confirm that this is valid
Sending build context to Docker daemon 5.12kB
Step 1/6 : FROM arm64v8/debian:8
8: Pulling from arm64v8/debian
559afc254be4: Pull complete
Digest: sha256:410257f511e4fe5cf234148cff2fddf5fc8586f4bb8a9a2d4c56e98190928e7c
Status: Downloaded newer image for arm64v8/debian:8
---> 89af0bd7c556
Step 2/6 : ENV DEBIAN_FRONTEND noninteractive
---> Running in 86595267fd11
Removing intermediate container 86595267fd11
---> d4f781a9fac4
Step 3/6 : RUN apt-get update && apt-get install -y apt-utils && apt-get install -y curl sudo wget expect gnupg build-essential devscripts equivs selinux-basics procps gawk
---> Running in ea3213046e05
Get:1 http://security.debian.org jessie/updates InRelease [44.9 kB]
Ign http://deb.debian.org jessie InRelease
Get:2 http://deb.debian.org jessie-updates InRelease [16.3 kB]
Get:3 http://deb.debian.org jessie Release.gpg [1652 B]
Get:4 http://deb.debian.org jessie Release [77.3 kB]
Fetched 140 kB in 0s (1134 kB/s)
W: Failed to fetch http://security.debian.org/debian-security/dists/jessie/updates/InRelease Unable to find expected entry 'main/binary-arm64/Packages' in Release file (Wrong sources.list entry or malformed file)
W: Failed to fetch http://deb.debian.org/debian/dists/jessie-updates/InRelease Unable to find expected entry 'main/binary-arm64/Packages' in Release file (Wrong sources.list entry or malformed file)
W: Failed to fetch http://deb.debian.org/debian/dists/jessie/Release Unable to find expected entry 'main/binary-arm64/Packages' in Release file (Wrong sources.list entry or malformed file)
E: Some index files failed to download. They have been ignored, or old ones used instead.
The command '/bin/sh -c apt-get update && apt-get install -y apt-utils && apt-get install -y curl sudo wget expect gnupg build-essential devscripts equivs selinux-basics procps gawk' returned a non-zero code: 100
I managed to create the first Wazuh dashboard and indexer 4.3.5 packages for ARM64, I only managed to successfully test the installation of dashboard on both Ubuntu 20.04 and Amazon Linux 2.
It remains to test the installation of indexer and then the operation of the complete stack on these systems.
Another thing that I have to solve is the creation of these packages through the automatic process of Jenkins, I still need to create the necessary images for these processes.
Packages:
Dashboard:
- deb: https://packages-dev.wazuh.com/trash/apt/pool/main/w/wazuh-dashboard/wazuh-dashboard_4.3.5-1_arm64.deb
- rpm: https://packages-dev.wazuh.com/trash/yum/wazuh-dashboard-4.3.5-1.aarch64.rpm
Indexer:
- deb: https://packages-dev.wazuh.com/trash/apt/pool/main/w/wazuh-indexer/wazuh-indexer_4.3.5-1_arm64.deb
- rpm: https://packages-dev.wazuh.com/trash/yum/wazuh-indexer-4.3.5-1.aarch64.rpm
We do not have filebeat 7.10.2 packages for arm64 architecture, so this requires further investigation, the options are:
- Create own filebeat 7.10.2 packages for arm64
- Use filebeat 7.12.0 which already has its own arm64 packages
- Use Logstash 7.10.2 which has support for arm64
In the branch https://github.com/wazuh/wazuh-packages/tree/1637-RPMSuportfoStack-to4.3 are the changes made for the creation of the arm64 packages
@c-bordon I'm currently trying to test-compile your branch-changes on an arm64 bullseye node (debian 11) ...
How exactly do the commands look like to build the dashboard and indexer packages?
Following https://documentation.wazuh.com/current/development/packaging/generate-deb-package.html there's only the
manager and client
options for the shellscript in debs subdir
(https://github.com/wazuh/wazuh-packages/blob/1637-RPMSuportfoStack-to4.3/debs/generate_debian_package.sh)
Edit I recon these are in here?
https://github.com/wazuh/wazuh-packages/tree/master/stack
Do they get used by the above shell script or are they unrelated?
Hi @thiscantbeserious, I share with you some steps that you should execute to build the packages.
-
You must work on the https://github.com/wazuh/wazuh-packages/tree/1637-RPMSuportfoStack-to4.3 branch since this development is not completed
-
You must run the scripts on an arm64 host
-
You must create the bases for both indexer and dashboard by executing the script as follows:
./generate_base.sh -a arm64 -s /tmp
This script is located at:
wazuh-packages/stack/dashboard/base
wazuh-packages/stack/indexer/base
The -s is to put the base file in /tmp, you can change the path or omit this parameter and an output directory will be created in the directory where the script is located.
- You must create the packages by executing the following script:
./build_package.sh -a arm64 -s /tmp --base local --base-path /tmp
This script is located at:
wazuh-packages/stack/dashboard/rpm
wazuh-packages/stack/dashboard/deb
wazuh-packages/stack/indexer/rpm
wazuh-packages/stack/indexer/deb
the -s works the same as for step 3
--base-path has to contain the directory where you put the base in step 3
With this, you should be able to build your Wazuh indexer and Wazuh dashboard arm64 packages
By the way, if it helps, I think I got filebeat 7.10.2 to package for ARM64 with this patch (its long lol):
Show patch content
diff --git a/dev-tools/mage/build.go b/dev-tools/mage/build.go
index 2efe61502a..1b616f8db2 100644
--- a/dev-tools/mage/build.go
+++ b/dev-tools/mage/build.go
@@ -83,6 +83,24 @@ func DefaultGolangCrossBuildArgs() BuildArgs {
return args
}
+// DefaultGolangNativeBuildArgs returns the default BuildArgs for use in
+// native-builds.
+func DefaultGolangNativeBuildArgs() BuildArgs {
+ args := DefaultBuildArgs()
+ args.Name += "-" + Platform.GOOS + "-" + Platform.Arch
+ args.OutputDir = filepath.Join("build", "golang-nativebuild")
+ if bp, found := BuildPlatforms.Get(Platform.Name); found {
+ args.CGO = bp.Flags.SupportsCGO()
+ }
+
+ // Enable DEP (data execution protection) for Windows binaries.
+ if Platform.GOOS == "windows" {
+ args.LDFlags = append(args.LDFlags, "-extldflags=-Wl,--nxcompat")
+ }
+
+ return args
+}
+
// GolangCrossBuild invokes "go build" inside of the golang-crossbuild Docker
// environment.
func GolangCrossBuild(params BuildArgs) error {
@@ -95,6 +113,11 @@ func GolangCrossBuild(params BuildArgs) error {
return Build(params)
}
+// GolangNativeBuild invokes "go build" natively.
+func GolangNativeBuild(params BuildArgs) error {
+ return Build(params)
+}
+
// Build invokes "go build" to produce a binary.
func Build(params BuildArgs) error {
fmt.Println(">> build: Building", params.Name)
diff --git a/dev-tools/mage/godaemon.go b/dev-tools/mage/godaemon.go
index b580bc3ed5..6e5227d215 100644
--- a/dev-tools/mage/godaemon.go
+++ b/dev-tools/mage/godaemon.go
@@ -79,6 +79,46 @@ func BuildGoDaemon() error {
return RunCmds(compileCmd)
}
+// BuildGoDaemon builds the go-deamon binary.
+func NativeBuildGoDaemon() error {
+ if GOOS != "linux" {
+ return errors.New("go-daemon only builds for linux")
+ }
+
+ // Test if binaries are up-to-date.
+ godaemonDir, err := gotool.ListModuleCacheDir("github.com/tsg/go-daemon")
+ if err != nil {
+ return err
+ }
+ input := filepath.Join(godaemonDir, "src", "god.c")
+ output := MustExpand("build/golang-nativebuild/god-{{.Platform.GOOS}}-{{.Platform.Arch}}")
+ if IsUpToDate(output, input) {
+ log.Println(">>> nativeBuildGoDaemon is up-to-date for", Platform.Name)
+ return nil
+ }
+
+ // Determine what compiler to use based on CC that is set by golang-crossbuild.
+ cc := os.Getenv("CC")
+ if cc == "" {
+ cc = "cc"
+ }
+
+ compileCmd := []string{
+ cc,
+ input,
+ "-o", createDir(output),
+ "-lpthread", "-static",
+ }
+ switch Platform.Name {
+ case "linux/amd64":
+ compileCmd = append(compileCmd, "-m64")
+ case "linux/386":
+ compileCmd = append(compileCmd, "-m32")
+ }
+
+ return RunCmds(compileCmd)
+}
+
// CrossBuildGoDaemon cross-build the go-daemon binary using the
// golang-crossbuild environment.
func CrossBuildGoDaemon(options ...CrossBuildOption) error {
diff --git a/dev-tools/mage/pkg.go b/dev-tools/mage/pkg.go
index 4ecdec89d3..5a7366e85a 100644
--- a/dev-tools/mage/pkg.go
+++ b/dev-tools/mage/pkg.go
@@ -106,6 +106,87 @@ func Package() error {
return nil
}
+// Package native packages the Beat for distribution. It generates packages based on
+// the current platform and registered packaging specifications.
+func NativePackage() error {
+ if len(Packages) == 0 {
+ return errors.New("no package specs are registered. Call " +
+ "UseCommunityBeatPackaging, UseElasticBeatPackaging or USeElasticBeatWithoutXPackPackaging first.")
+ }
+
+ var tasks []interface{}
+
+ currentPlatform := BuildPlatform{
+ Name: Platform.GOOS + "/" + Platform.GOARCH,
+ }
+
+ os.Setenv("PLATFORMS", currentPlatform.Name)
+
+ currentPlatform.Flags = BuildPlatforms.filter(func(p BuildPlatform) bool {
+ return p.Name == currentPlatform.Name
+ })[0].Flags
+
+ for _, pkg := range Packages {
+ if pkg.OS != currentPlatform.GOOS() {
+ continue
+ }
+
+ for _, pkgType := range pkg.Types {
+ if pkgType == DMG && runtime.GOOS != "darwin" {
+ log.Printf("Skipping DMG package type because build host isn't darwin")
+ continue
+ }
+
+ packageArch, err := getOSArchName(currentPlatform, pkgType)
+ if err != nil {
+ log.Printf("Skipping arch %v for package type %v: %v", currentPlatform.Arch(), pkgType, err)
+ continue
+ }
+
+ agentPackageType := TarGz
+ if pkg.OS == "windows" {
+ agentPackageType = Zip
+ }
+
+ agentPackageArch, err := getOSArchName(currentPlatform, agentPackageType)
+ if err != nil {
+ log.Printf("Skipping arch %v for package type %v: %v", currentPlatform.Arch(), pkgType, err)
+ continue
+ }
+
+ agentPackageDrop, _ := os.LookupEnv("AGENT_DROP_PATH")
+
+ spec := pkg.Spec.Clone()
+ spec.OS = currentPlatform.GOOS()
+ spec.Arch = packageArch
+ spec.Snapshot = Snapshot
+ spec.evalContext = map[string]interface{}{
+ "GOOS": currentPlatform.GOOS(),
+ "GOARCH": currentPlatform.GOARCH(),
+ "GOARM": currentPlatform.GOARM(),
+ "Platform": currentPlatform,
+ "AgentArchName": agentPackageArch,
+ "PackageType": pkgType.String(),
+ "BinaryExt": binaryExtension(currentPlatform.GOOS()),
+ "AgentDropPath": agentPackageDrop,
+ }
+
+ spec.packageDir, err = pkgType.PackagingDir(packageStagingDir, currentPlatform, spec)
+ if err != nil {
+ log.Printf("Skipping arch %v for package type %v: %v", currentPlatform.Arch(), pkgType, err)
+ continue
+ }
+
+ spec = spec.Evaluate()
+
+ tasks = append(tasks, packageBuilder{currentPlatform, spec, pkgType}.Build)
+ }
+ }
+
+ Parallel(tasks...)
+ return nil
+}
+
type packageBuilder struct {
Platform BuildPlatform
Spec PackageSpec
diff --git a/dev-tools/mage/pkgspecs.go b/dev-tools/mage/pkgspecs.go
index 558b8e37ae..b656c7cb52 100644
--- a/dev-tools/mage/pkgspecs.go
+++ b/dev-tools/mage/pkgspecs.go
@@ -68,6 +68,12 @@ func UseElasticBeatOSSPackaging() {
MustUsePackaging("elastic_beat_oss", packageSpecFile)
}
+// UseElasticBeatOSSPackaging configures the package target to build OSS
+// packages.
+func UseElasticNativeBeatOSSPackaging() {
+ MustUsePackaging("elastic_native_beat_oss", packageSpecFile)
+}
+
// UseElasticBeatXPackPackaging configures the package target to build Elastic
// licensed (X-Pack) packages.
func UseElasticBeatXPackPackaging() {
diff --git a/dev-tools/packaging/packages.yml b/dev-tools/packaging/packages.yml
index ce48eb2209..7172b9af01 100644
--- a/dev-tools/packaging/packages.yml
+++ b/dev-tools/packaging/packages.yml
@@ -352,6 +352,53 @@ shared:
template: '{{ elastic_beats_dir }}/dev-tools/packaging/templates/{{.PackageType}}/init.sh.tmpl'
mode: 0755
+ # Deb/RPM native spec for community beats.
+ - &deb_rpm_native_spec
+ <<: *common
+ post_install_script: '{{ elastic_beats_dir }}/dev-tools/packaging/files/linux/systemd-daemon-reload.sh'
+ files:
+ /usr/share/{{.BeatName}}/bin/{{.BeatName}}{{.BinaryExt}}:
+ source: build/golang-nativebuild/{{.BeatName}}-{{.GOOS}}-{{.Platform.Arch}}{{.BinaryExt}}
+ mode: 0755
+ /etc/{{.BeatName}}/fields.yml:
+ source: fields.yml
+ mode: 0644
+ /usr/share/{{.BeatName}}/LICENSE.txt:
+ source: '{{ repo.RootDir }}/LICENSE.txt'
+ mode: 0644
+ /usr/share/{{.BeatName}}/NOTICE.txt:
+ source: '{{ repo.RootDir }}/NOTICE.txt'
+ mode: 0644
+ /usr/share/{{.BeatName}}/README.md:
+ template: '{{ elastic_beats_dir }}/dev-tools/packaging/templates/common/README.md.tmpl'
+ mode: 0644
+ /usr/share/{{.BeatName}}/.build_hash.txt:
+ content: >
+ {{ commit }}
+ mode: 0644
+ /etc/{{.BeatName}}/{{.BeatName}}.reference.yml:
+ source: '{{.BeatName}}.reference.yml'
+ mode: 0644
+ /etc/{{.BeatName}}/{{.BeatName}}.yml:
+ source: '{{.BeatName}}.yml'
+ mode: 0600
+ config: true
+ /usr/share/{{.BeatName}}/kibana:
+ source: _meta/kibana.generated
+ mode: 0644
+ /usr/share/{{.BeatName}}/bin/{{.BeatName}}-god:
+ source: build/golang-nativebuild/god-{{.GOOS}}-{{.Platform.Arch}}
+ mode: 0755
+ /usr/bin/{{.BeatName}}:
+ template: '{{ elastic_beats_dir }}/dev-tools/packaging/templates/linux/beatname.sh.tmpl'
+ mode: 0755
+ /lib/systemd/system/{{.BeatServiceName}}.service:
+ template: '{{ elastic_beats_dir }}/dev-tools/packaging/templates/linux/systemd.unit.tmpl'
+ mode: 0644
+ /etc/init.d/{{.BeatServiceName}}:
+ template: '{{ elastic_beats_dir }}/dev-tools/packaging/templates/{{.PackageType}}/init.sh.tmpl'
+ mode: 0755
+
# MacOS pkg spec for community beats.
- &macos_beat_pkg_spec
<<: *common
@@ -428,12 +475,49 @@ shared:
source: _meta/kibana.generated
mode: 0644
+ - &binary_native_files
+ '{{.BeatName}}{{.BinaryExt}}':
+ source: build/golang-nativebuild/{{.BeatName}}-{{.GOOS}}-{{.Platform.Arch}}{{.BinaryExt}}
+ mode: 0755
+ fields.yml:
+ source: fields.yml
+ mode: 0644
+ LICENSE.txt:
+ source: '{{ repo.RootDir }}/LICENSE.txt'
+ mode: 0644
+ NOTICE.txt:
+ source: '{{ repo.RootDir }}/NOTICE.txt'
+ mode: 0644
+ README.md:
+ template: '{{ elastic_beats_dir }}/dev-tools/packaging/templates/common/README.md.tmpl'
+ mode: 0644
+ .build_hash.txt:
+ content: >
+ {{ commit }}
+ mode: 0644
+ '{{.BeatName}}.reference.yml':
+ source: '{{.BeatName}}.reference.yml'
+ mode: 0644
+ '{{.BeatName}}.yml':
+ source: '{{.BeatName}}.yml'
+ mode: 0600
+ config: true
+ kibana:
+ source: _meta/kibana.generated
+ mode: 0644
+
# Binary package spec (tar.gz for linux/darwin) for community beats.
- &binary_spec
<<: *common
files:
<<: *binary_files
+ # Binary package spec (tar.gz for linux/darwin) for community beats.
+ - &binary_native_spec
+ <<: *common
+ files:
+ <<: *binary_native_files
+
# Binary package spec (zip for windows) for community beats.
- &windows_binary_spec
<<: *common
@@ -459,6 +543,19 @@ shared:
mode: 0600
config: true
+ - &docker_native_spec
+ <<: *binary_native_spec
+ extra_vars:
+ from: 'centos:7'
+ buildFrom: 'centos:7'
+ user: '{{ .BeatName }}'
+ linux_capabilities: ''
+ files:
+ '{{.BeatName}}.yml':
+ source: '{{.BeatName}}.docker.yml'
+ mode: 0600
+ config: true
+
- &docker_ubi_spec
extra_vars:
image_name: '{{.BeatName}}-ubi8'
@@ -591,6 +688,53 @@ specs:
<<: *apache_license_for_binaries
name: '{{.BeatName}}-oss'
+
+ # Elastic Native Beat with Apache License (OSS) and binary taken the current
+ # directory.
+ elastic_native_beat_oss:
+ - os: windows
+ types: [zip]
+ spec:
+ <<: *windows_binary_spec
+ <<: *apache_license_for_binaries
+ name: '{{.BeatName}}-oss'
+
+ - os: darwin
+ types: [tgz]
+ spec:
+ <<: *binary_spec
+ <<: *apache_license_for_binaries
+ name: '{{.BeatName}}-oss'
+
+ - os: darwin
+ types: [dmg]
+ spec:
+ <<: *macos_beat_pkg_spec
+ <<: *apache_license_for_macos_pkg
+ name: '{{.BeatName}}-oss'
+
+ - os: linux
+ types: [tgz]
+ spec:
+ <<: *binary_native_spec
+ <<: *apache_license_for_binaries
+ name: '{{.BeatName}}-oss'
+
+ - os: linux
+ types: [deb, rpm]
+ spec:
+ <<: *deb_rpm_native_spec
+ <<: *apache_license_for_deb_rpm
+ name: '{{.BeatName}}-oss'
+
+ - os: linux
+ types: [docker]
+ spec:
+ <<: *docker_native_spec
+ <<: *elastic_docker_spec
+ <<: *apache_license_for_binaries
+ name: '{{.BeatName}}-oss'
+
# Elastic Beat with Elastic License and binary taken the current directory.
elastic_beat_xpack:
###
diff --git a/filebeat/magefile.go b/filebeat/magefile.go
index 9b90cc68d9..1ce395eeec 100644
--- a/filebeat/magefile.go
+++ b/filebeat/magefile.go
@@ -15,6 +15,7 @@
// specific language governing permissions and limitations
// under the License.
+//go:build mage
// +build mage
package main
@@ -75,6 +76,11 @@ func GolangCrossBuild() error {
return devtools.GolangCrossBuild(devtools.DefaultGolangCrossBuildArgs())
}
+// GolangNativeBuild build the Beat binary natively;
+func NativeBuild() error {
+ return devtools.GolangNativeBuild(devtools.DefaultGolangNativeBuildArgs())
+}
+
// BuildGoDaemon builds the go-daemon binary (use crossBuildGoDaemon).
func BuildGoDaemon() error {
return devtools.BuildGoDaemon()
@@ -90,6 +96,11 @@ func CrossBuildGoDaemon() error {
return devtools.CrossBuildGoDaemon()
}
+// NativeBuildGoDaemon builds the go-daemon binary.
+func NativeBuildGoDaemon() error {
+ return devtools.NativeBuildGoDaemon()
+}
+
// Package packages the Beat for distribution.
// Use SNAPSHOT=true to build snapshots.
// Use PLATFORMS to control the target platforms.
@@ -107,6 +118,20 @@ func Package() {
mg.SerialDeps(devtools.Package, TestPackages)
}
+// Use VERSION_QUALIFIER to control the version qualifier.
+func NativePackage() {
+ start := time.Now()
+ defer func() { fmt.Println("package ran for", time.Since(start)) }()
+
+ devtools.UseElasticNativeBeatOSSPackaging()
+ devtools.PackageKibanaDashboardsFromBuildDir()
+ filebeat.CustomizePackaging()
+
+ mg.Deps(Update)
+ mg.Deps(NativeBuild, NativeBuildGoDaemon)
+ mg.SerialDeps(devtools.NativePackage, TestPackages)
+}
+
// TestPackages tests the generated packages (i.e. file modes, owners, groups).
func TestPackages() error {
return devtools.TestPackages(devtools.WithModules(), devtools.WithModulesD())
Edit: It packaged! The package test also worked!
Also, while untested, the Wazuh indexer and dashboard package as DEBs successfully with this patch:
Show patch content
diff --git a/.gitignore b/.gitignore
index 822b7b01..6bbbbe55 100644
--- a/.gitignore
+++ b/.gitignore
@@ -21,6 +21,8 @@ ova/Config_files/filebeat.yml
.java
stack/dashboard/base/output
stack/indexer/base/output
+stack/dashboard/tmp
+stack/indexer/tmp
.cache
wpk/versions
solaris/packer/builds/
diff --git a/VERSION b/VERSION
index 6016e8ad..cbe06cdb 100644
--- a/VERSION
+++ b/VERSION
@@ -1 +1 @@
-4.6.0
+4.4.4
diff --git a/stack/dashboard/base/docker/Dockerfile b/stack/dashboard/base/docker/Dockerfile
index 184d894e..90601fa7 100644
--- a/stack/dashboard/base/docker/Dockerfile
+++ b/stack/dashboard/base/docker/Dockerfile
@@ -23,7 +23,7 @@ RUN yum install -y \
RUN git clone https://github.com/google/brotli.git
-RUN cd brotli && chmod +x ./bootstrap && ./bootstrap && ./configure --prefix=/usr --bindir=/usr/bin --sbindir=/usr/sbin --libexecdir=/usr/lib64/brotli --libdir=/usr/lib64/brotli --datarootdir=/usr/share --mandir=/usr/share/man/man1 --docdir=/usr/share/doc \
+RUN cd brotli && chmod +x ./bootstrap && ./bootstrap && ./configure --prefix=/usr --bindir=/usr/bin --sbindir=/usr/sbin --libexecdir=/usr/lib/brotli --libdir=/usr/lib/brotli --datarootdir=/usr/share --mandir=/usr/share/man/man1 --docdir=/usr/share/doc \
&& make && make install
# Add the scripts to build the RPM package
diff --git a/stack/dashboard/base/generate_base.sh b/stack/dashboard/base/generate_base.sh
index e1a665e8..f81bd8e0 100755
--- a/stack/dashboard/base/generate_base.sh
+++ b/stack/dashboard/base/generate_base.sh
@@ -73,6 +73,7 @@ help() {
echo "Usage: $0 [OPTIONS]"
echo
echo " --app-url <url> [Optional] Set the repository from where the Wazuh plugin should be downloaded. By default, will be used pre-release."
+ echo " -a, --arch <arch> [Optional] Set the architecture to build for. By default ${architecture}"
echo " -s, --store <path> [Optional] Set the destination path of package. By default, an output folder will be created."
echo " --reference <ref> [Optional] wazuh-packages branch or tag"
echo " --future [Optional] Build test future package 99.99.0 Used for development purposes."
@@ -107,6 +108,14 @@ main() {
help 1
fi
;;
+ "-a"|"--arch")
+ if [ -n "${2}" ]; then
+ architecture="${2}"
+ shift 2
+ else
+ help 1
+ fi
+ ;;
"--reference")
if [ -n "${2}" ]; then
reference="${2}"
diff --git a/stack/dashboard/deb/build_package.sh b/stack/dashboard/deb/build_package.sh
index e75e9aa1..63d36a3a 100755
--- a/stack/dashboard/deb/build_package.sh
+++ b/stack/dashboard/deb/build_package.sh
@@ -14,11 +14,12 @@ outdir="${current_path}/output"
revision="1"
build_docker="yes"
deb_amd64_builder="deb_dashboard_builder_amd64"
+deb_arm64_builder="deb_dashboard_builder_arm64"
deb_builder_dockerfile="${current_path}/docker"
future="no"
base_cmd=""
-url=""
-build_base="yes"
+base="s3"
+base_path="${current_path}/../base/output"
trap ctrl_c INT
@@ -46,20 +47,6 @@ build_deb() {
# Copy the necessary files
cp ${current_path}/builder.sh ${dockerfile_path}
- if [ "${build_base}" == "yes" ];then
- # Base generation
- if [ "${future}" == "yes" ];then
- base_cmd+="--future "
- fi
- if [ "${reference}" ];then
- base_cmd+="--reference ${reference}"
- fi
- if [ "${url}" ];then
- base_cmd+="--app-url ${url}"
- fi
- ../base/generate_base.sh -s ${outdir} -r ${revision} ${base_cmd}
- fi
-
# Build the Docker image
if [[ ${build_docker} == "yes" ]]; then
docker build -t ${container_name} ${dockerfile_path} || return 1
@@ -67,15 +54,19 @@ build_deb() {
# Build the Debian package with a Docker container
volumes="-v ${outdir}/:/tmp:Z"
- if [ "${reference}" ];then
+ if [ "${reference}" ]; then
docker run -t --rm ${volumes} \
${container_name} ${architecture} ${revision} \
- ${future} ${url} ${reference} || return 1
+ ${future} ${base} ${architecture_base} ${reference} || return 1
else
+ if [ "${base}" = "local" ];then
+ volumes="${volumes} -v ${base_path}:/root/output:Z"
+ fi
+
docker run -t --rm ${volumes} \
-v ${current_path}/../../..:/root:Z \
${container_name} ${architecture} ${revision} \
- ${future} ${url} || return 1
+ ${future} ${base} ${architecture_base} || return 1
fi
echo "Package $(ls -Art ${outdir} | tail -n 1) added to ${outdir}."
@@ -86,14 +77,22 @@ build_deb() {
build() {
build_name=""
file_path=""
+
if [ "${architecture}" = "x86_64" ] || [ "${architecture}" = "amd64" ]; then
architecture="amd64"
+ architecture_base="x64"
build_name="${deb_amd64_builder}"
file_path="${deb_builder_dockerfile}/${architecture}"
+ elif [ "${architecture}" = "arm64" ] || [ "${architecture}" = "aarch64" ]; then
+ architecture="arm64"
+ architecture_base="arm64"
+ build_name="${deb_arm64_builder}"
+ file_path="${deb_builder_dockerfile}/${architecture}"
else
- echo "Invalid architecture. Choose: amd64 (x86_64 is accepted too)"
+ echo "Invalid architecture. Choose: amd64 | arm64 | x86_64 | aarch64"
return 1
fi
+
build_deb ${build_name} ${file_path} || return 1
return 0
@@ -104,13 +103,13 @@ help() {
echo "Usage: $0 [OPTIONS]"
echo
echo " -a, --architecture <arch> [Optional] Target architecture of the package [amd64]."
- echo " --app-url <url> [Optional] Set the repository from where the Wazuh plugin should be downloaded. By default, will be used pre-release."
- echo " -b, --build-base <yes/no> [Optional] Build a new base or use a existing one. By default, yes."
echo " -r, --revision <rev> [Optional] Package revision. By default: 1."
echo " -s, --store <path> [Optional] Set the destination path of package. By default, an output folder will be created."
echo " --reference <ref> [Optional] wazuh-packages branch to download SPECs, not used by default."
echo " --dont-build-docker [Optional] Locally built docker image will be used instead of generating a new one."
echo " --future [Optional] Build test future package 99.99.0 Used for development purposes."
+ echo " --base <s3/local> [Optional] Base file location, can be \"local\" or \"s3\". Default: s3"
+ echo " --base-path [Optional] If base is local, you can indicate the full path where the base is located. Default: stack/dashboard/base/output"
echo " -h, --help Show this help."
echo
exit $1
@@ -132,22 +131,6 @@ main() {
help 1
fi
;;
- "--app-url")
- if [ -n "$2" ]; then
- repository="$2"
- shift 2
- else
- help 1
- fi
- ;;
- "-b"|"--build-base")
- if [ -n "${2}" ]; then
- build_base="${2}"
- shift 2
- else
- help 1
- fi
- ;;
"-r"|"--revision")
if [ -n "${2}" ]; then
revision="${2}"
@@ -172,6 +155,22 @@ main() {
future="yes"
shift 1
;;
+ "--base")
+ if [ -n "${2}" ]; then
+ base="${2}"
+ shift 2
+ else
+ help 1
+ fi
+ ;;
+ "--base-path")
+ if [ -n "${2}" ]; then
+ base_path="${2}"
+ shift 2
+ else
+ help 1
+ fi
+ ;;
"-s"|"--store")
if [ -n "${2}" ]; then
outdir="${2}"
diff --git a/stack/dashboard/deb/builder.sh b/stack/dashboard/deb/builder.sh
index c3fd6107..6c095518 100755
--- a/stack/dashboard/deb/builder.sh
+++ b/stack/dashboard/deb/builder.sh
@@ -14,8 +14,9 @@ target="wazuh-dashboard"
architecture=$1
revision=$2
future=$3
-repository=$4
-reference=$5
+base_location=$4
+architecture_base=$5
+reference=$6
directory_base="/usr/share/wazuh-dashboard"
if [ -z "${revision}" ]; then
@@ -32,21 +33,6 @@ else
fi
fi
-if [ "${repository}" ];then
- valid_url='(https?|ftp|file)://[-[:alnum:]\+&@#/%?=~_|!:,.;]*[-[:alnum:]\+&@#/%=~_|]'
- if [[ $repository =~ $valid_url ]];then
- url="${repository}"
- if ! curl --output /dev/null --silent --head --fail "${url}"; then
- echo "The given URL to download the Wazuh plugin zip does not exist: ${url}"
- exit 1
- fi
- else
- url="https://packages-dev.wazuh.com/${repository}/ui/dashboard/wazuh-${version}-${revision}.zip"
- fi
-else
- url="https://packages-dev.wazuh.com/pre-release/ui/dashboard/wazuh-${version}-${revision}.zip"
-fi
-
# Build directories
build_dir=/build
pkg_name="${target}-${version}"
@@ -64,13 +50,13 @@ else
cp -r /root/stack/dashboard/deb/debian/* ${source_dir}/debian/
fi
-
# Generating directory structure to build the .deb package
cd ${build_dir}/${target} && tar -czf ${pkg_name}.orig.tar.gz "${pkg_name}"
# Configure the package with the different parameters
sed -i "s:VERSION:${version}:g" ${source_dir}/debian/changelog
sed -i "s:RELEASE:${revision}:g" ${source_dir}/debian/changelog
+sed -i "s:ARCHITECTURE:${architecture_base}:g" ${source_dir}/debian/rules
sed -i "s:export INSTALLATION_DIR=.*:export INSTALLATION_DIR=${directory_base}:g" ${source_dir}/debian/rules
# Installing build dependencies
@@ -78,7 +64,7 @@ cd ${source_dir}
mk-build-deps -ir -t "apt-get -o Debug::pkgProblemResolver=yes -y"
# Build package
-debuild --no-lintian -eINSTALLATION_DIR="${directory_base}" -eVERSION="${version}" -eREVISION="${revision}" -eURL="${url}" -b -uc -us
+debuild --no-lintian -eINSTALLATION_DIR="${directory_base}" -eBASE="${base_location}" -eBASE_VERSION="${version}" -eBASE_REVISION="${revision}" -b -uc -us
deb_file="${target}_${version}-${revision}_${architecture}.deb"
diff --git a/stack/dashboard/deb/debian/rules b/stack/dashboard/deb/debian/rules
index 22f2c14c..3e923eab 100644
--- a/stack/dashboard/deb/debian/rules
+++ b/stack/dashboard/deb/debian/rules
@@ -28,9 +28,10 @@ export TARGET_DIR=${CURDIR}/debian/wazuh-dashboard
export NAME=wazuh-dashboard
export CONFIG_DIR=/etc/$(NAME)
export INSTALLATION_DIR=$(INSTALLATION_DIR)
+export ARCH=ARCHITECTURE
export USER=$(NAME)
export GROUP=$(NAME)
-export DASHBOARD_FILE=wazuh-dashboard-base-$(VERSION)-$(REVISION)-linux-x64.tar.xz
+export DASHBOARD_FILE=wazuh-dashboard-base-$(BASE_VERSION)-$(BASE_REVISION)-linux-$(ARCH).tar.xz
# -----------------------------------------------------------------------------
@@ -52,7 +53,11 @@ override_dh_auto_install:
# -----------------------------------------------------------------------------
override_dh_install:
- cp /tmp/$(DASHBOARD_FILE) ./
+ if [ "$(BASE)" = "s3" ]; then \
+ curl -kOL https://packages-dev.wazuh.com/stack/dashboard/base/$(DASHBOARD_FILE) ;\
+ else \
+ cp /root/output/$(DASHBOARD_FILE) ./ ;\
+ fi
groupadd $(GROUP)
useradd -g $(GROUP) $(USER)
@@ -79,11 +84,7 @@ override_dh_install:
chown -R $(USER):$(GROUP) $(TARGET_DIR)$(INSTALLATION_DIR)
chown -R $(USER):$(GROUP) $(TARGET_DIR)$(CONFIG_DIR)
- if [ "$(VERSION)" = "99.99.0" ]; then \
- runuser $(USER) --shell="/bin/bash" --command="$(TARGET_DIR)$(INSTALLATION_DIR)/bin/opensearch-dashboards-plugin install https://packages-dev.wazuh.com/futures/ui/dashboard/wazuh-99.99.0-$(REVISION).zip" ;\
- else \
- runuser $(USER) --shell="/bin/bash" --command="$(TARGET_DIR)$(INSTALLATION_DIR)/bin/opensearch-dashboards-plugin install $(URL)" ;\
- fi
+ runuser $(USER) --shell="/bin/bash" --command="$(TARGET_DIR)$(INSTALLATION_DIR)/bin/opensearch-dashboards-plugin install https://packages-dev.wazuh.com/pre-release/ui/dashboard/wazuh-$(BASE_VERSION)-${BASE_REVISION}.zip" ;\
find $(TARGET_DIR)$(INSTALLATION_DIR)/plugins/wazuh/ -exec chown $(USER):$(GROUP) {} \;
diff --git a/stack/indexer/base/generate_base.sh b/stack/indexer/base/generate_base.sh
index 20a224e9..914a2955 100755
--- a/stack/indexer/base/generate_base.sh
+++ b/stack/indexer/base/generate_base.sh
@@ -66,7 +66,8 @@ help() {
echo
echo "Usage: $0 [OPTIONS]"
echo
- echo " -s, --store <path> [Optional] Set the destination path of package. By default, an output folder will be created."
+ echo " -a, --arch <arch> [Optional] Set the architecture to build for. By default ${architecture}"
+ echo " -s, --store <path> [Optional] Set the destination path of package. By default, an output folder will be created."
echo " --reference <ref> [Optional] wazuh-packages branch or tag"
echo " --future [Optional] Build test future package 99.99.0 Used for development purposes."
echo " -r, --revision <rev> [Optional] Package revision. By default ${revision}"
@@ -84,6 +85,14 @@ main() {
"-h"|"--help")
help 0
;;
+ "-a"|"--arch")
+ if [ -n "${2}" ]; then
+ architecture="${2}"
+ shift 2
+ else
+ help 1
+ fi
+ ;;
"-s"|"--store")
if [ -n "${2}" ]; then
outdir="${2}"
diff --git a/stack/indexer/deb/build_package.sh b/stack/indexer/deb/build_package.sh
index b5032d07..ef3e6760 100755
--- a/stack/indexer/deb/build_package.sh
+++ b/stack/indexer/deb/build_package.sh
@@ -14,10 +14,11 @@ outdir="${current_path}/output"
revision="1"
build_docker="yes"
deb_amd64_builder="deb_indexer_builder_amd64"
+deb_arm64_builder="deb_indexer_builder_arm64"
deb_builder_dockerfile="${current_path}/docker"
future="no"
-base_cmd=""
-build_base="yes"
+base="s3"
+base_path="${current_path}/../base/output"
trap ctrl_c INT
@@ -41,34 +42,26 @@ build_deb() {
# Copy the necessary files
cp ${current_path}/builder.sh ${dockerfile_path}
- if [ "${build_base}" == "yes" ];then
- # Base generation
- if [ "${future}" == "yes" ];then
- base_cmd+="--future "
- fi
- if [ "${reference}" ];then
- base_cmd+="--reference ${reference}"
- fi
- ../base/generate_base.sh -s ${outdir} -r ${revision} ${base_cmd}
- fi
-
# Build the Docker image
if [[ ${build_docker} == "yes" ]]; then
docker build -t ${container_name} ${dockerfile_path} || return 1
fi
-
# Build the Debian package with a Docker container
volumes="-v ${outdir}/:/tmp:Z"
if [ "${reference}" ];then
docker run -t --rm ${volumes} \
${container_name} ${architecture} ${revision} \
- ${future} ${reference} || return 1
+ ${future} ${base} ${architecture_base} ${reference} || return 1
else
+ if [ "${base}" = "local" ];then
+ volumes="${volumes} -v ${base_path}:/root/output:Z"
+ fi
+
docker run -t --rm ${volumes} \
-v ${current_path}/../../..:/root:Z \
${container_name} ${architecture} \
- ${revision} ${future} || return 1
+ ${revision} ${future} ${base} ${architecture_base} || return 1
fi
echo "Package $(ls -Art ${outdir} | tail -n 1) added to ${outdir}."
@@ -81,12 +74,19 @@ build() {
file_path=""
if [ "${architecture}" = "x86_64" ] || [ "${architecture}" = "amd64" ]; then
architecture="amd64"
+ architecture_base="x64"
build_name="${deb_amd64_builder}"
file_path="${deb_builder_dockerfile}/${architecture}"
+ elif [ "${architecture}" = "arm64" ] || [ "${architecture}" = "aarch64" ]; then
+ architecture="arm64"
+ architecture_base="arm64"
+ build_name="${deb_arm64_builder}"
+ file_path="${deb_builder_dockerfile}/${architecture}"
else
- echo "Invalid architecture. Choose: amd64 (x86_64 is accepted too)"
+ echo "Invalid architecture. Choose: amd64 | arm64 | x86_64 | aarch64"
return 1
fi
+
build_deb ${build_name} ${file_path} || return 1
return 0
@@ -96,13 +96,14 @@ help() {
echo
echo "Usage: $0 [OPTIONS]"
echo
- echo " -a, --architecture <arch> [Optional] Target architecture of the package [amd64]."
- echo " -b, --build-base <yes/no> [Optional] Build a new base or use a existing one. By default, yes."
+ echo " -a, --architecture <arch> [Optional] Target architecture of the package [amd64, arm64]."
echo " -r, --revision <rev> [Optional] Package revision. By default: 1."
echo " -s, --store <path> [Optional] Set the destination path of package. By default, an output folder will be created."
echo " --reference <ref> [Optional] wazuh-packages branch to download SPECs, not used by default."
echo " --dont-build-docker [Optional] Locally built docker image will be used instead of generating a new one."
echo " --future [Optional] Build test future package 99.99.0 Used for development purposes."
+ echo " --base <s3/local> [Optional] Base file location, can be \"local\" or \"s3\". Default: s3"
+ echo " --base-path [Optional] If base is local, you can indicate the full path where the base is located. Default: stack/indexer/base/output"
echo " -h, --help Show this help."
echo
exit $1
@@ -124,14 +125,6 @@ main() {
help 1
fi
;;
- "-b"|"--build-base")
- if [ -n "${2}" ]; then
- build_base="${2}"
- shift 2
- else
- help 1
- fi
- ;;
"-r"|"--revision")
if [ -n "${2}" ]; then
revision="${2}"
@@ -156,6 +149,22 @@ main() {
future="yes"
shift 1
;;
+ "--base")
+ if [ -n "${2}" ]; then
+ base="${2}"
+ shift 2
+ else
+ help 1
+ fi
+ ;;
+ "--base-path")
+ if [ -n "${2}" ]; then
+ base_path="${2}"
+ shift 2
+ else
+ help 1
+ fi
+ ;;
"-s"|"--store")
if [ -n "${2}" ]; then
outdir="${2}"
diff --git a/stack/indexer/deb/builder.sh b/stack/indexer/deb/builder.sh
index fa349531..3fd5cb36 100755
--- a/stack/indexer/deb/builder.sh
+++ b/stack/indexer/deb/builder.sh
@@ -15,7 +15,9 @@ target="wazuh-indexer"
architecture=$1
revision=$2
future=$3
-reference=$4
+base_location=$4
+architecture_base=$5
+reference=$6
directory_base="/usr/share/wazuh-indexer"
if [ -z "${revision}" ]; then
@@ -55,13 +57,14 @@ cd ${build_dir}/${target} && tar -czf ${pkg_name}.orig.tar.gz "${pkg_name}"
# Configure the package with the different parameters
sed -i "s:VERSION:${version}:g" ${source_dir}/debian/changelog
sed -i "s:RELEASE:${revision}:g" ${source_dir}/debian/changelog
+sed -i "s:ARCHITECTURE:${architecture_base}:g" ${source_dir}/debian/rules
# Installing build dependencies
cd ${source_dir}
mk-build-deps -ir -t "apt-get -o Debug::pkgProblemResolver=yes -y"
# Build package
-debuild --no-lintian -eINSTALLATION_DIR="${directory_base}" -eVERSION="${version}" -eREVISION="${revision}" -b -uc -us
+debuild --no-lintian -eINSTALLATION_DIR="${directory_base}" -eBASE="${base_location}" -eBASE_VERSION="${version}" -eBASE_REVISION="${revision}" -b -uc -us
deb_file="${target}_${version}-${revision}_${architecture}.deb"
diff --git a/stack/indexer/deb/debian/rules b/stack/indexer/deb/debian/rules
index 9759cd19..71787f4b 100644
--- a/stack/indexer/deb/debian/rules
+++ b/stack/indexer/deb/debian/rules
@@ -36,7 +36,8 @@ export LIB_DIR=/var/lib/${NAME}
export PID_DIR=/run/${NAME}
export SYS_DIR=/usr/lib
export BASE_DIR=${NAME}-*
-export INDEXER_FILE=wazuh-indexer-base-$(VERSION)-$(REVISION)-linux-x64.tar.xz
+export ARCH=ARCHITECTURE
+export INDEXER_FILE=wazuh-indexer-base-$(BASE_VERSION)-$(BASE_REVISION)-linux-$(ARCH).tar.xz
export REPO_DIR=/root/unattended_installer
# -----------------------------------------------------------------------------
@@ -60,7 +61,12 @@ override_dh_auto_install:
override_dh_install:
rm -rf $(INSTALLATION_DIR)/
- cp /tmp/$(INDEXER_FILE) ./
+
+ if [ "$(BASE)" = "s3" ]; then \
+ curl -kOL https://packages-dev.wazuh.com/stack/indexer/base/$(INDEXER_FILE) ;\
+ else \
+ cp /root/output/$(INDEXER_FILE) ./ ;\
+ fi
tar -xf $(INDEXER_FILE)
@@ -1208,10 +1214,13 @@ override_dh_fixperms:
chmod 640 $(TARGET_DIR)$(INSTALLATION_DIR)/jdk/jmods/jdk.incubator.vector.jmod
chmod 640 $(TARGET_DIR)$(INSTALLATION_DIR)/jdk/jmods/jdk.jpackage.jmod
chmod 640 $(TARGET_DIR)$(INSTALLATION_DIR)/jdk/jmods/jdk.random.jmod
- chmod 640 $(TARGET_DIR)$(INSTALLATION_DIR)/jdk/lib/libjsvml.so
chmod 640 $(TARGET_DIR)$(INSTALLATION_DIR)/jdk/lib/libsyslookup.so
chmod 640 $(TARGET_DIR)$(INSTALLATION_DIR)/jdk/lib/security/blocked.certs
+ if [ -f "$(TARGET_DIR)$(INSTALLATION_DIR)/jdk/lib/libjsvml.so" ]; then \
+ chmod 640 $(TARGET_DIR)$(INSTALLATION_DIR)/jdk/lib/libjsvml.so; \
+ fi
+
# -----------------------------------------------------------------------------
override_dh_strip:
Note that RPMs do not package with this, as I have not modified the build scripts for them since I don't know how RPM packaging works xD
Update: Just a note that this was done on the master branch, but I had to change the contents of the VERSION file since the version listed does not exist on the packages server, causing the builds to throw an error.
Hello, any update on that? Do you know when Wazuh will officially be supported for ARM devices?
I am waiting for this, the indexer and dashboard is not working on ARM. i have a five node ARM cluster in my lab. I really do not want to introduce x64 nodes for Wazuh
I might try to work on this again - I'll have to see if I have the time. I also wanted to say that the patches I made above are pretty outdated, likely won't work with current Wazuh code, and, well, are pretty garbage to be honest. I did all of this remotely with a web-based VS Code instance on an arm-based server, so I had many limitations. I'll try to make some better patches for the future, and even open a PR, but as far as filebeat may go, that might have to be an extra build step on Wazuh's part. I don't think that the version of filebeat Wazuh uses is still maintained. (I'm going off of memory, sorry if some of the details are wrong)
Currently, we have no plans on supporting ARM architectures on wazuh-dashboard and wazuh-indexer
We might revisit this in the future.
