Hardcoded path to store dependent files causes issues in Yocto setups

[AnilChoudhury-Eaton]

Environment:

• eKuiper version (e.g. 1.3.1):
• Hardware configuration (e.g. lscpu): KVM x86_64
• OS (e.g. cat /etc/os-release): Linux 5.4.x
• Others:

What happened and what you expected to happen: When we are building eKuiper along with EdgeX components using Dunfell Yocto recipe, we are facing issues as many installable files are conflicting with files already installed due to other recipes. eKuiper try to put its files into /etc/services path and /etc/functions path. it has hardcoded this all over. in our case systemd is adding '/etc/services' as a file with network port information. some of our recipe is adding '/etc/functions' as a file.

How to reproduce it (as minimally and precisely as possible): We are building Yocto build of Linux with Dunfell and systemd enabled.

Error: Transaction check error:
  file /etc/services conflicts between attempted installs of netbase-2:6.1-r0.core2_64 and rules-engine-1.3.1-r1.core2_64
  file /etc/functions conflicts between attempted installs of px-red-rootfs-base-1.0+git0+4cc4fd9ad7_4fa63909c4-21r7.noarch and rules-engine-1.3.1-r1.core2_64

Anything else we need to know?: Please setup environment variables which can be configured about the path to store its files. Or please define its own directory to keep its content like '/etc/ekuiper'

[ngjaying]

@anilchoudhury Thanks for opening this issue. How do you build kuiperd, please paste the command. By default, the relative path will be used, which means the kuiperd will only add files under ${kuiperPath} like ${kuiperPath}/etc/service. You can also build with absolute path as below remember to replace $(PKG_VSN) with your version number.

GO111MODULE=on CGO_ENABLED=1 go build -trimpath -ldflags="-s -w -X main.Version=$(PKG_VSN) -X main.LoadFileType=absolute" -o kuiperd cmd/kuiperd/main.go

Then all the configuration files will be placed at /etc/kuiper/* like /etc/kuiper/services

[AnilChoudhury-Eaton]

Hi @ngjaying, Thanks for your help. I took some time to find it working with edgex in yocto recipe.

I have compiled and installed in following way:

• Added the /etc directory content to my yocto recipe, so that I can customize it for myself.
• Need to copy the whole /etc contents (including /etc/sources, /etc/functions, /etc/sinks, etc/miltilingual) to /etc/kuiper path
• Then need to copy the same /etc/sources, /etc/functions, /etc/sinks to /var/lib/kuiper/plugins path. I don't know why It is expecting from both path?
• Need to add -modcacherw flag to build to allow cleaning during yocto clean. else need root privilege
• Need to add -tags edgex to build for edgex

GO_IMPORT="github.com/lf-edge/ekuiper"
FILESEXTRAPATHS_prepend := "${THISDIR}/files:"

PV = "1.3.1"
SRC_URI = "git://github.com/lf-edge/ekuiper;protocol=https;nobranch=1;destsuffix=${BPN}-${PV}/src/${GO_IMPORT};rev=4be9eb49c256b1e6e918b16f7d32f4a6ed2a4e46"

SRC_URI += "file://rules-engine.service \
	    file://rules-engine-pre.service \
	    file://rootfs/etc/* \
	    file://rootfs/etc/sources/* \
	    file://rootfs/etc/sinks/* \
	    file://rootfs/etc/functions/* \
	"
inherit red_go systemd
DEPENDS = "pkgconfig-native zeromq"

SYSTEMD_SERVICE_${PN} = "rules-engine.service rules-engine-pre.service"
SYSTEMD_AUTO_ENABLE_${PN} = "enable"

INSANE_SKIP_${PN} += "already-stripped"

export kuiperPath = "kuiper"

do_compile() {
  # Custom compile command as recommended by eKuiper
  GO111MODULE=on CGO_ENABLED=1 go build -modcacherw -trimpath -ldflags="-s -w -X main.Version=${PV} -X main.LoadFileType=absolute" **-tags edgex** -o kuiperd cmd/kuiperd/main.go
}

do_install_append() {
  # Copy the service files
  install -m 0755 -d ${D}${systemd_unitdir}/system
  install -m 664 ${WORKDIR}/rules-engine.service ${D}${systemd_unitdir}/system
  install -m 664 ${WORKDIR}/rules-engine-pre.service ${D}${systemd_unitdir}/system

  # Copy the binary file
  install -d 775 ${D}/${base_bindir_native}/ 
  install -m 550 ${S}/src/${GO_IMPORT}/kuiperd ${D}/${base_bindir_native}/

  # Install Kuiper service configuration files
  install -d 0775 ${D}/${sysconfdir}/${kuiperPath}
  cp -rf ${WORKDIR}/rootfs/${sysconfdir}/* ${D}/${sysconfdir}/${kuiperPath}/

  # Install Kuiper plugin configurations for source, sink etc. to /var/lib/kuiper in addition to /etc/kuiper as it looks at both path 
  install -d 0775 ${D}/${localstatedir}/lib/${kuiperPath}/plugins
  cp -rf ${WORKDIR}/rootfs/${sysconfdir}/sources ${D}/${localstatedir}/lib/${kuiperPath}/plugins/
  cp -rf ${WORKDIR}/rootfs/${sysconfdir}/sinks ${D}/${localstatedir}/lib/${kuiperPath}/plugins/
  cp -rf ${WORKDIR}/rootfs/${sysconfdir}/functions ${D}/${localstatedir}/lib/${kuiperPath}/plugins/
}

[ngjaying]

@anilchoudhury Thanks very much for the following up. Those absolute path are actually hard coded in https://github.com/lf-edge/ekuiper/blob/master/internal/conf/path.go in the AbsoluteMapping map

[AnilChoudhury-Eaton]

Then I would suggest to find a way to override them at build time. This will help us decide where to put them on our device.

[ngjaying]

OK, that's fair. As you see, those path are already parameterized, we just need to export them in the conf file. @Rory-Z Can you think of any problems to make these path configurable?