datadog-agent
datadog-agent copied to clipboard
Published
20 hours ago •
DataDog
DataDog
packaging: ensure a minimal set of required files are present
What does this PR do?
This invokes a new dd-pkg command to ensure a set of predefined files are present in our packages
Motivation
Catching missing files early, in case no test covers those See BARX-528
Additional Notes
Possible Drawbacks / Trade-offs
Describe how to test/QA your changes
Gitlab CI Configuration Changes
Modified Jobs
.heroku_build_base
.heroku_build_base:
artifacts:
expire_in: 2 weeks
paths:
- $OMNIBUS_PACKAGE_DIR
cache:
- key:
files:
- omnibus/Gemfile
- release.json
prefix: omnibus-deps-$CI_JOB_NAME-$OMNIBUS_RUBY_VERSION-$OMNIBUS_SOFTWARE
paths:
- omnibus/vendor/bundle
image: 486234852809.dkr.ecr.us-east-1.amazonaws.com/ci/datadog-agent-buildimages/deb_x64$DATADOG_AGENT_BUILDIMAGES_SUFFIX:$DATADOG_AGENT_BUILDIMAGES
needs:
- go_mod_tidy_check
- build_system-probe-x64
- go_deps
- generate_minimized_btfs_x64
rules:
- if: $CI_COMMIT_BRANCH =~ /^mq-working-branch-/
when: never
- when: on_success
script:
- mkdir -p $GOPATH/pkg/mod && tar xJf modcache.tar.xz -C $GOPATH/pkg/mod
- rm -f modcache.tar.xz
- pushd omnibus && bundle config set --local path 'vendor/bundle' && popd
- echo "About to build for $RELEASE_VERSION"
- rm -rf $OMNIBUS_PACKAGE_DIR/*
- set +x
- printf -- "$($CI_PROJECT_DIR/tools/ci/fetch_secret.sh $DEB_GPG_KEY)" | gpg --import
--batch
- EXIT="${PIPESTATUS[0]}"; if [ $EXIT -ne 0 ]; then echo "Unable to locate credentials
needs gitlab runner restart"; exit $EXIT; fi
- DEB_SIGNING_PASSPHRASE=$($CI_PROJECT_DIR/tools/ci/fetch_secret.sh $DEB_SIGNING_PASSPHRASE)
|| exit $?; export DEB_SIGNING_PASSPHRASE
- tar -xf $CI_PROJECT_DIR/sysprobe-build-outputs.tar.xz
- mkdir -p /tmp/system-probe
- $S3_CP_CMD $S3_PERMANENT_ARTIFACTS_URI/clang-$CLANG_LLVM_VER.${PACKAGE_ARCH} /tmp/system-probe/clang-bpf
- $S3_CP_CMD $S3_PERMANENT_ARTIFACTS_URI/llc-$CLANG_LLVM_VER.${PACKAGE_ARCH} /tmp/system-probe/llc-bpf
- cp $CI_PROJECT_DIR/minimized-btfs.tar.xz /tmp/system-probe/minimized-btfs.tar.xz
- chmod 0744 /tmp/system-probe/clang-bpf /tmp/system-probe/llc-bpf
- inv -e omnibus.build --release-version "$RELEASE_VERSION" --major-version "$AGENT_MAJOR_VERSION"
--python-runtimes "$PYTHON_RUNTIMES" --base-dir $OMNIBUS_BASE_DIR ${USE_S3_CACHING}
--skip-deps --go-mod-cache="$GOPATH/pkg/mod" --system-probe-bin=/tmp/system-probe
--flavor heroku
- ls -la $OMNIBUS_PACKAGE_DIR
- curl -sSL "https://dd-package-tools.s3.amazonaws.com/dd-pkg/${DD_PKG_VERSION}/dd-pkg_Linux_${DD_PKG_ARCH}.tar.gz"
| tar -xz -C /usr/local/bin dd-pkg
- find $OMNIBUS_PACKAGE_DIR -iregex '.*\.\(deb\|rpm\)' | xargs dd-pkg lint
+ - "if [ -n \"$PACKAGE_REQUIRED_FILES_LIST\" ]; then\n find $OMNIBUS_PACKAGE_DIR\
+ \ \\( -name '*.deb' -or -name '*.rpm' \\) -a -not -name '*-dbg[_-]*' | xargs dd-pkg\
+ \ check-files --required-files ${PACKAGE_REQUIRED_FILES_LIST}\nfi\n"
- $S3_CP_CMD $OMNIBUS_PACKAGE_DIR/datadog-heroku-agent_*_${PACKAGE_ARCH}.deb $S3_ARTIFACTS_URI/$DESTINATION_DEB
- $S3_CP_CMD $OMNIBUS_PACKAGE_DIR/datadog-heroku-agent-dbg_*_${PACKAGE_ARCH}.deb
$S3_ARTIFACTS_URI/$DESTINATION_DBG_DEB
- $S3_CP_CMD $OMNIBUS_PACKAGE_DIR/version-manifest.json $S3_SBOM_STORAGE_URI/$CI_JOB_NAME/version-manifest.json
stage: package_build
tags:
- arch:amd64
variables:
DD_PKG_ARCH: x86_64
KUBERNETES_CPU_REQUEST: 16
KUBERNETES_MEMORY_LIMIT: 32Gi
KUBERNETES_MEMORY_REQUEST: 32Gi
PACKAGE_ARCH: amd64
.lint_linux_packages
.lint_linux_packages:
- curl -sSL "https://dd-package-tools.s3.amazonaws.com/dd-pkg/${DD_PKG_VERSION}/dd-pkg_Linux_${DD_PKG_ARCH}.tar.gz"
| tar -xz -C /usr/local/bin dd-pkg
- find $OMNIBUS_PACKAGE_DIR -iregex '.*\.\(deb\|rpm\)' | xargs dd-pkg lint
+ - "if [ -n \"$PACKAGE_REQUIRED_FILES_LIST\" ]; then\n find $OMNIBUS_PACKAGE_DIR \\\
+ ( -name '*.deb' -or -name '*.rpm' \\) -a -not -name '*-dbg[_-]*' | xargs dd-pkg\
+ \ check-files --required-files ${PACKAGE_REQUIRED_FILES_LIST}\nfi\n"
.package_deb_common
.package_deb_common:
artifacts:
expire_in: 2 weeks
paths:
- $OMNIBUS_PACKAGE_DIR
cache:
- key:
files:
- omnibus/Gemfile
- release.json
prefix: omnibus-deps-$CI_JOB_NAME-$OMNIBUS_RUBY_VERSION-$OMNIBUS_SOFTWARE
paths:
- omnibus/vendor/bundle
script:
- pushd omnibus && bundle config set --local path 'vendor/bundle' && popd
- echo "About to package for $RELEASE_VERSION"
- set +x
- printf -- "$($CI_PROJECT_DIR/tools/ci/fetch_secret.sh $DEB_GPG_KEY)" | gpg --import
--batch
- EXIT="${PIPESTATUS[0]}"; if [ $EXIT -ne 0 ]; then echo "Unable to locate credentials
needs gitlab runner restart"; exit $EXIT; fi
- DEB_SIGNING_PASSPHRASE=$($CI_PROJECT_DIR/tools/ci/fetch_secret.sh $DEB_SIGNING_PASSPHRASE)
|| exit $?; export DEB_SIGNING_PASSPHRASE
- inv -e omnibus.build --release-version "$RELEASE_VERSION" --major-version "$AGENT_MAJOR_VERSION"
--base-dir $OMNIBUS_BASE_DIR --skip-deps --target-project ${DD_PROJECT} ${OMNIBUS_EXTRA_ARGS}
- curl -sSL "https://dd-package-tools.s3.amazonaws.com/dd-pkg/${DD_PKG_VERSION}/dd-pkg_Linux_${DD_PKG_ARCH}.tar.gz"
| tar -xz -C /usr/local/bin dd-pkg
- find $OMNIBUS_PACKAGE_DIR -iregex '.*\.\(deb\|rpm\)' | xargs dd-pkg lint
+ - "if [ -n \"$PACKAGE_REQUIRED_FILES_LIST\" ]; then\n find $OMNIBUS_PACKAGE_DIR\
+ \ \\( -name '*.deb' -or -name '*.rpm' \\) -a -not -name '*-dbg[_-]*' | xargs dd-pkg\
+ \ check-files --required-files ${PACKAGE_REQUIRED_FILES_LIST}\nfi\n"
- $S3_CP_CMD $OMNIBUS_PACKAGE_DIR/datadog-${DD_PROJECT}_*_${PACKAGE_ARCH}.deb $S3_ARTIFACTS_URI/$DESTINATION_DEB
stage: packaging
variables:
KUBERNETES_CPU_REQUEST: 16
KUBERNETES_MEMORY_LIMIT: 32Gi
KUBERNETES_MEMORY_REQUEST: 32Gi
OMNIBUS_PACKAGE_ARTIFACT_DIR: $OMNIBUS_PACKAGE_DIR
+ PACKAGE_REQUIRED_FILES_LIST: test/required_files/agent-deb.txt
.package_iot_deb_common
.package_iot_deb_common:
artifacts:
expire_in: 2 weeks
paths:
- $OMNIBUS_PACKAGE_DIR
cache:
- key:
files:
- omnibus/Gemfile
- release.json
prefix: omnibus-deps-$CI_JOB_NAME-$OMNIBUS_RUBY_VERSION-$OMNIBUS_SOFTWARE
paths:
- omnibus/vendor/bundle
rules:
- if: $CI_COMMIT_BRANCH =~ /^mq-working-branch-/
when: never
- when: on_success
script:
- pushd omnibus && bundle config set --local path 'vendor/bundle' && popd
- echo "About to package for $RELEASE_VERSION"
- set +x
- printf -- "$($CI_PROJECT_DIR/tools/ci/fetch_secret.sh $DEB_GPG_KEY)" | gpg --import
--batch
- EXIT="${PIPESTATUS[0]}"; if [ $EXIT -ne 0 ]; then echo "Unable to locate credentials
needs gitlab runner restart"; exit $EXIT; fi
- DEB_SIGNING_PASSPHRASE=$($CI_PROJECT_DIR/tools/ci/fetch_secret.sh $DEB_SIGNING_PASSPHRASE)
|| exit $?; export DEB_SIGNING_PASSPHRASE
- inv -e omnibus.build --release-version "$RELEASE_VERSION" --base-dir $OMNIBUS_BASE_DIR
--skip-deps --flavor iot
- curl -sSL "https://dd-package-tools.s3.amazonaws.com/dd-pkg/${DD_PKG_VERSION}/dd-pkg_Linux_${DD_PKG_ARCH}.tar.gz"
| tar -xz -C /usr/local/bin dd-pkg
- find $OMNIBUS_PACKAGE_DIR -iregex '.*\.\(deb\|rpm\)' | xargs dd-pkg lint
+ - "if [ -n \"$PACKAGE_REQUIRED_FILES_LIST\" ]; then\n find $OMNIBUS_PACKAGE_DIR\
+ \ \\( -name '*.deb' -or -name '*.rpm' \\) -a -not -name '*-dbg[_-]*' | xargs dd-pkg\
+ \ check-files --required-files ${PACKAGE_REQUIRED_FILES_LIST}\nfi\n"
- $S3_CP_CMD $OMNIBUS_PACKAGE_DIR/datadog-*_${PACKAGE_ARCH}.deb $S3_ARTIFACTS_URI/$DESTINATION_DEB
stage: packaging
variables:
AGENT_MAJOR_VERSION: 7
KUBERNETES_CPU_REQUEST: 16
KUBERNETES_MEMORY_LIMIT: 32Gi
KUBERNETES_MEMORY_REQUEST: 32Gi
OMNIBUS_PACKAGE_ARTIFACT_DIR: $OMNIBUS_PACKAGE_DIR
+ PACKAGE_REQUIRED_FILES_LIST: test/required_files/iot-agent-deb.txt
RELEASE_VERSION: $RELEASE_VERSION_7
.package_iot_rpm_common
.package_iot_rpm_common:
artifacts:
expire_in: 2 weeks
paths:
- $OMNIBUS_PACKAGE_DIR
cache:
- key:
files:
- omnibus/Gemfile
- release.json
prefix: omnibus-deps-$CI_JOB_NAME-$OMNIBUS_RUBY_VERSION-$OMNIBUS_SOFTWARE
paths:
- omnibus/vendor/bundle
rules:
- if: $CI_COMMIT_BRANCH =~ /^mq-working-branch-/
when: never
- when: on_success
script:
- echo "About to build for $RELEASE_VERSION"
- pushd omnibus && bundle config set --local path 'vendor/bundle' && popd
- printf -- "$($CI_PROJECT_DIR/tools/ci/fetch_secret.sh $RPM_GPG_KEY)" | gpg --import
--batch
- EXIT="${PIPESTATUS[0]}"; if [ $EXIT -ne 0 ]; then echo "Unable to locate credentials
needs gitlab runner restart"; exit $EXIT; fi
- RPM_SIGNING_PASSPHRASE=$($CI_PROJECT_DIR/tools/ci/fetch_secret.sh $RPM_SIGNING_PASSPHRASE)
|| exit $?; export RPM_SIGNING_PASSPHRASE
- inv -e omnibus.build --release-version "$RELEASE_VERSION" --base-dir $OMNIBUS_BASE_DIR
--skip-deps --flavor=iot ${OMNIBUS_EXTRA_ARGS}
- ls -la $OMNIBUS_PACKAGE_DIR/
- curl -sSL "https://dd-package-tools.s3.amazonaws.com/dd-pkg/${DD_PKG_VERSION}/dd-pkg_Linux_${DD_PKG_ARCH}.tar.gz"
| tar -xz -C /usr/local/bin dd-pkg
- find $OMNIBUS_PACKAGE_DIR -iregex '.*\.\(deb\|rpm\)' | xargs dd-pkg lint
+ - "if [ -n \"$PACKAGE_REQUIRED_FILES_LIST\" ]; then\n find $OMNIBUS_PACKAGE_DIR\
+ \ \\( -name '*.deb' -or -name '*.rpm' \\) -a -not -name '*-dbg[_-]*' | xargs dd-pkg\
+ \ check-files --required-files ${PACKAGE_REQUIRED_FILES_LIST}\nfi\n"
stage: packaging
variables:
KUBERNETES_CPU_REQUEST: 16
KUBERNETES_MEMORY_LIMIT: 32Gi
KUBERNETES_MEMORY_REQUEST: 32Gi
OMNIBUS_PACKAGE_ARTIFACT_DIR: $OMNIBUS_PACKAGE_DIR
+ PACKAGE_REQUIRED_FILES_LIST: test/required_files/iot-agent-rpm.txt
RELEASE_VERSION: $RELEASE_VERSION_7
.package_ot_deb_common
.package_ot_deb_common:
artifacts:
expire_in: 2 weeks
paths:
- $OMNIBUS_PACKAGE_DIR
cache:
- key:
files:
- omnibus/Gemfile
- release.json
prefix: omnibus-deps-$CI_JOB_NAME-$OMNIBUS_RUBY_VERSION-$OMNIBUS_SOFTWARE
paths:
- omnibus/vendor/bundle
script:
- pushd omnibus && bundle config set --local path 'vendor/bundle' && popd
- echo "About to package for $RELEASE_VERSION"
- set +x
- printf -- "$($CI_PROJECT_DIR/tools/ci/fetch_secret.sh $DEB_GPG_KEY)" | gpg --import
--batch
- EXIT="${PIPESTATUS[0]}"; if [ $EXIT -ne 0 ]; then echo "Unable to locate credentials
needs gitlab runner restart"; exit $EXIT; fi
- DEB_SIGNING_PASSPHRASE=$($CI_PROJECT_DIR/tools/ci/fetch_secret.sh $DEB_SIGNING_PASSPHRASE)
|| exit $?; export DEB_SIGNING_PASSPHRASE
- inv -e omnibus.build --release-version "$RELEASE_VERSION" --major-version "$AGENT_MAJOR_VERSION"
--base-dir $OMNIBUS_BASE_DIR --skip-deps --target-project ${DD_PROJECT} --flavor
ot ${OMNIBUS_EXTRA_ARGS}
- curl -sSL "https://dd-package-tools.s3.amazonaws.com/dd-pkg/${DD_PKG_VERSION}/dd-pkg_Linux_${DD_PKG_ARCH}.tar.gz"
| tar -xz -C /usr/local/bin dd-pkg
- find $OMNIBUS_PACKAGE_DIR -iregex '.*\.\(deb\|rpm\)' | xargs dd-pkg lint
+ - "if [ -n \"$PACKAGE_REQUIRED_FILES_LIST\" ]; then\n find $OMNIBUS_PACKAGE_DIR\
+ \ \\( -name '*.deb' -or -name '*.rpm' \\) -a -not -name '*-dbg[_-]*' | xargs dd-pkg\
+ \ check-files --required-files ${PACKAGE_REQUIRED_FILES_LIST}\nfi\n"
- $S3_CP_CMD $OMNIBUS_PACKAGE_DIR/datadog-ot-agent_*_${PACKAGE_ARCH}.deb $S3_ARTIFACTS_URI/$DESTINATION_DEB
stage: packaging
variables:
KUBERNETES_CPU_REQUEST: 16
KUBERNETES_MEMORY_LIMIT: 32Gi
KUBERNETES_MEMORY_REQUEST: 32Gi
OMNIBUS_PACKAGE_ARTIFACT_DIR: $OMNIBUS_PACKAGE_DIR
+ PACKAGE_REQUIRED_FILES_LIST: test/required_files/agent-deb.txt
.package_rpm_common
.package_rpm_common:
artifacts:
expire_in: 2 weeks
paths:
- $OMNIBUS_PACKAGE_DIR
before_script: null
cache:
- key:
files:
- omnibus/Gemfile
- release.json
prefix: omnibus-deps-$CI_JOB_NAME-$OMNIBUS_RUBY_VERSION-$OMNIBUS_SOFTWARE
paths:
- omnibus/vendor/bundle
rules:
- if: $CI_COMMIT_BRANCH =~ /^mq-working-branch-/
when: never
- when: on_success
script:
- echo "About to build for $RELEASE_VERSION"
- pushd omnibus && bundle config set --local path 'vendor/bundle' && popd
- printf -- "$($CI_PROJECT_DIR/tools/ci/fetch_secret.sh $RPM_GPG_KEY)" | gpg --import
--batch
- EXIT="${PIPESTATUS[0]}"; if [ $EXIT -ne 0 ]; then echo "Unable to locate credentials
needs gitlab runner restart"; exit $EXIT; fi
- RPM_SIGNING_PASSPHRASE=$($CI_PROJECT_DIR/tools/ci/fetch_secret.sh $RPM_SIGNING_PASSPHRASE)
|| exit $?; export RPM_SIGNING_PASSPHRASE
- inv -e omnibus.build --release-version "$RELEASE_VERSION" --major-version "$AGENT_MAJOR_VERSION"
--base-dir $OMNIBUS_BASE_DIR --skip-deps --target-project=${DD_PROJECT} ${OMNIBUS_EXTRA_ARGS}
- ls -la $OMNIBUS_PACKAGE_DIR/
- curl -sSL "https://dd-package-tools.s3.amazonaws.com/dd-pkg/${DD_PKG_VERSION}/dd-pkg_Linux_${DD_PKG_ARCH}.tar.gz"
| tar -xz -C /usr/local/bin dd-pkg
- find $OMNIBUS_PACKAGE_DIR -iregex '.*\.\(deb\|rpm\)' | xargs dd-pkg lint
+ - "if [ -n \"$PACKAGE_REQUIRED_FILES_LIST\" ]; then\n find $OMNIBUS_PACKAGE_DIR\
+ \ \\( -name '*.deb' -or -name '*.rpm' \\) -a -not -name '*-dbg[_-]*' | xargs dd-pkg\
+ \ check-files --required-files ${PACKAGE_REQUIRED_FILES_LIST}\nfi\n"
stage: packaging
variables:
KUBERNETES_CPU_REQUEST: 16
KUBERNETES_MEMORY_LIMIT: 32Gi
KUBERNETES_MEMORY_REQUEST: 32Gi
OMNIBUS_PACKAGE_ARTIFACT_DIR: $OMNIBUS_PACKAGE_DIR
+ PACKAGE_REQUIRED_FILES_LIST: test/required_files/agent-rpm.txt
.package_suse_rpm_common
.package_suse_rpm_common:
artifacts:
expire_in: 2 weeks
paths:
- $OMNIBUS_PACKAGE_DIR_SUSE
before_script: null
cache:
- key:
files:
- omnibus/Gemfile
- release.json
prefix: omnibus-deps-$CI_JOB_NAME-$OMNIBUS_RUBY_VERSION-$OMNIBUS_SOFTWARE
paths:
- omnibus/vendor/bundle
rules:
- if: $CI_COMMIT_BRANCH =~ /^mq-working-branch-/
when: never
- when: on_success
script:
- echo "About to build for $RELEASE_VERSION"
- pushd omnibus && bundle config set --local path 'vendor/bundle' && popd
- printf -- "$($CI_PROJECT_DIR/tools/ci/fetch_secret.sh $RPM_GPG_KEY)" | gpg --import
--batch
- EXIT="${PIPESTATUS[0]}"; if [ $EXIT -ne 0 ]; then echo "Unable to locate credentials
needs gitlab runner restart"; exit $EXIT; fi
- RPM_SIGNING_PASSPHRASE=$($CI_PROJECT_DIR/tools/ci/fetch_secret.sh $RPM_SIGNING_PASSPHRASE)
|| exit $?; export RPM_SIGNING_PASSPHRASE
- inv -e omnibus.build --release-version "$RELEASE_VERSION" --major-version "$AGENT_MAJOR_VERSION"
--base-dir $OMNIBUS_BASE_DIR --skip-deps --target-project=${DD_PROJECT} ${OMNIBUS_EXTRA_ARGS}
- ls -la $OMNIBUS_PACKAGE_DIR/
- curl -sSL "https://dd-package-tools.s3.amazonaws.com/dd-pkg/${DD_PKG_VERSION}/dd-pkg_Linux_${DD_PKG_ARCH}.tar.gz"
| tar -xz -C /usr/local/bin dd-pkg
- find $OMNIBUS_PACKAGE_DIR -iregex '.*\.\(deb\|rpm\)' | xargs dd-pkg lint
+ - "if [ -n \"$PACKAGE_REQUIRED_FILES_LIST\" ]; then\n find $OMNIBUS_PACKAGE_DIR\
+ \ \\( -name '*.deb' -or -name '*.rpm' \\) -a -not -name '*-dbg[_-]*' | xargs dd-pkg\
+ \ check-files --required-files ${PACKAGE_REQUIRED_FILES_LIST}\nfi\n"
- mkdir -p $OMNIBUS_PACKAGE_DIR_SUSE && mv $OMNIBUS_PACKAGE_DIR/*.rpm $OMNIBUS_PACKAGE_DIR_SUSE/
stage: packaging
variables:
KUBERNETES_CPU_REQUEST: 16
KUBERNETES_MEMORY_LIMIT: 32Gi
KUBERNETES_MEMORY_REQUEST: 32Gi
OMNIBUS_EXTRA_ARGS: --host-distribution=suse
OMNIBUS_PACKAGE_ARTIFACT_DIR: $OMNIBUS_PACKAGE_DIR
+ PACKAGE_REQUIRED_FILES_LIST: test/required_files/agent-rpm.txt
agent_deb-arm64-a6
agent_deb-arm64-a6:
artifacts:
expire_in: 2 weeks
paths:
- $OMNIBUS_PACKAGE_DIR
cache:
- key:
files:
- omnibus/Gemfile
- release.json
prefix: omnibus-deps-$CI_JOB_NAME-$OMNIBUS_RUBY_VERSION-$OMNIBUS_SOFTWARE
paths:
- omnibus/vendor/bundle
image: 486234852809.dkr.ecr.us-east-1.amazonaws.com/ci/datadog-agent-buildimages/deb_arm64$DATADOG_AGENT_ARMBUILDIMAGES_SUFFIX:$DATADOG_AGENT_ARMBUILDIMAGES
needs:
- datadog-agent-6-arm64
rules:
- if: $CI_COMMIT_BRANCH =~ /^mq-working-branch-/
when: never
- when: on_success
script:
- pushd omnibus && bundle config set --local path 'vendor/bundle' && popd
- echo "About to package for $RELEASE_VERSION"
- set +x
- printf -- "$($CI_PROJECT_DIR/tools/ci/fetch_secret.sh $DEB_GPG_KEY)" | gpg --import
--batch
- EXIT="${PIPESTATUS[0]}"; if [ $EXIT -ne 0 ]; then echo "Unable to locate credentials
needs gitlab runner restart"; exit $EXIT; fi
- DEB_SIGNING_PASSPHRASE=$($CI_PROJECT_DIR/tools/ci/fetch_secret.sh $DEB_SIGNING_PASSPHRASE)
|| exit $?; export DEB_SIGNING_PASSPHRASE
- inv -e omnibus.build --release-version "$RELEASE_VERSION" --major-version "$AGENT_MAJOR_VERSION"
--base-dir $OMNIBUS_BASE_DIR --skip-deps --target-project ${DD_PROJECT} ${OMNIBUS_EXTRA_ARGS}
- curl -sSL "https://dd-package-tools.s3.amazonaws.com/dd-pkg/${DD_PKG_VERSION}/dd-pkg_Linux_${DD_PKG_ARCH}.tar.gz"
| tar -xz -C /usr/local/bin dd-pkg
- find $OMNIBUS_PACKAGE_DIR -iregex '.*\.\(deb\|rpm\)' | xargs dd-pkg lint
+ - "if [ -n \"$PACKAGE_REQUIRED_FILES_LIST\" ]; then\n find $OMNIBUS_PACKAGE_DIR\
+ \ \\( -name '*.deb' -or -name '*.rpm' \\) -a -not -name '*-dbg[_-]*' | xargs dd-pkg\
+ \ check-files --required-files ${PACKAGE_REQUIRED_FILES_LIST}\nfi\n"
- $S3_CP_CMD $OMNIBUS_PACKAGE_DIR/datadog-${DD_PROJECT}_*_${PACKAGE_ARCH}.deb $S3_ARTIFACTS_URI/$DESTINATION_DEB
stage: packaging
tags:
- arch:arm64
variables:
AGENT_MAJOR_VERSION: 6
DD_PKG_ARCH: arm64
DD_PROJECT: agent
DESTINATION_DEB: datadog-agent_6_arm64.deb
KUBERNETES_CPU_REQUEST: 16
KUBERNETES_MEMORY_LIMIT: 32Gi
KUBERNETES_MEMORY_REQUEST: 32Gi
OMNIBUS_PACKAGE_ARTIFACT_DIR: $OMNIBUS_PACKAGE_DIR
PACKAGE_ARCH: arm64
+ PACKAGE_REQUIRED_FILES_LIST: test/required_files/agent-deb.txt
RELEASE_VERSION: $RELEASE_VERSION_6
agent_deb-arm64-a7
agent_deb-arm64-a7:
artifacts:
expire_in: 2 weeks
paths:
- $OMNIBUS_PACKAGE_DIR
cache:
- key:
files:
- omnibus/Gemfile
- release.json
prefix: omnibus-deps-$CI_JOB_NAME-$OMNIBUS_RUBY_VERSION-$OMNIBUS_SOFTWARE
paths:
- omnibus/vendor/bundle
image: 486234852809.dkr.ecr.us-east-1.amazonaws.com/ci/datadog-agent-buildimages/deb_arm64$DATADOG_AGENT_ARMBUILDIMAGES_SUFFIX:$DATADOG_AGENT_ARMBUILDIMAGES
needs:
- datadog-agent-7-arm64
rules:
- if: $CI_COMMIT_BRANCH =~ /^mq-working-branch-/
when: never
- when: on_success
script:
- pushd omnibus && bundle config set --local path 'vendor/bundle' && popd
- echo "About to package for $RELEASE_VERSION"
- set +x
- printf -- "$($CI_PROJECT_DIR/tools/ci/fetch_secret.sh $DEB_GPG_KEY)" | gpg --import
--batch
- EXIT="${PIPESTATUS[0]}"; if [ $EXIT -ne 0 ]; then echo "Unable to locate credentials
needs gitlab runner restart"; exit $EXIT; fi
- DEB_SIGNING_PASSPHRASE=$($CI_PROJECT_DIR/tools/ci/fetch_secret.sh $DEB_SIGNING_PASSPHRASE)
|| exit $?; export DEB_SIGNING_PASSPHRASE
- inv -e omnibus.build --release-version "$RELEASE_VERSION" --major-version "$AGENT_MAJOR_VERSION"
--base-dir $OMNIBUS_BASE_DIR --skip-deps --target-project ${DD_PROJECT} ${OMNIBUS_EXTRA_ARGS}
- curl -sSL "https://dd-package-tools.s3.amazonaws.com/dd-pkg/${DD_PKG_VERSION}/dd-pkg_Linux_${DD_PKG_ARCH}.tar.gz"
| tar -xz -C /usr/local/bin dd-pkg
- find $OMNIBUS_PACKAGE_DIR -iregex '.*\.\(deb\|rpm\)' | xargs dd-pkg lint
+ - "if [ -n \"$PACKAGE_REQUIRED_FILES_LIST\" ]; then\n find $OMNIBUS_PACKAGE_DIR\
+ \ \\( -name '*.deb' -or -name '*.rpm' \\) -a -not -name '*-dbg[_-]*' | xargs dd-pkg\
+ \ check-files --required-files ${PACKAGE_REQUIRED_FILES_LIST}\nfi\n"
- $S3_CP_CMD $OMNIBUS_PACKAGE_DIR/datadog-${DD_PROJECT}_*_${PACKAGE_ARCH}.deb $S3_ARTIFACTS_URI/$DESTINATION_DEB
stage: packaging
tags:
- arch:arm64
variables:
AGENT_MAJOR_VERSION: 7
DD_PKG_ARCH: arm64
DD_PROJECT: agent
DESTINATION_DEB: datadog-agent_7_arm64.deb
KUBERNETES_CPU_REQUEST: 16
KUBERNETES_MEMORY_LIMIT: 32Gi
KUBERNETES_MEMORY_REQUEST: 32Gi
OMNIBUS_PACKAGE_ARTIFACT_DIR: $OMNIBUS_PACKAGE_DIR
PACKAGE_ARCH: arm64
+ PACKAGE_REQUIRED_FILES_LIST: test/required_files/agent-deb.txt
RELEASE_VERSION: $RELEASE_VERSION_7
agent_deb-x64-a6
agent_deb-x64-a6:
artifacts:
expire_in: 2 weeks
paths:
- $OMNIBUS_PACKAGE_DIR
cache:
- key:
files:
- omnibus/Gemfile
- release.json
prefix: omnibus-deps-$CI_JOB_NAME-$OMNIBUS_RUBY_VERSION-$OMNIBUS_SOFTWARE
paths:
- omnibus/vendor/bundle
image: 486234852809.dkr.ecr.us-east-1.amazonaws.com/ci/datadog-agent-buildimages/deb_x64$DATADOG_AGENT_BUILDIMAGES_SUFFIX:$DATADOG_AGENT_BUILDIMAGES
needs:
- datadog-agent-6-x64
rules:
- if: $CI_COMMIT_BRANCH =~ /^mq-working-branch-/
when: never
- when: on_success
script:
- pushd omnibus && bundle config set --local path 'vendor/bundle' && popd
- echo "About to package for $RELEASE_VERSION"
- set +x
- printf -- "$($CI_PROJECT_DIR/tools/ci/fetch_secret.sh $DEB_GPG_KEY)" | gpg --import
--batch
- EXIT="${PIPESTATUS[0]}"; if [ $EXIT -ne 0 ]; then echo "Unable to locate credentials
needs gitlab runner restart"; exit $EXIT; fi
- DEB_SIGNING_PASSPHRASE=$($CI_PROJECT_DIR/tools/ci/fetch_secret.sh $DEB_SIGNING_PASSPHRASE)
|| exit $?; export DEB_SIGNING_PASSPHRASE
- inv -e omnibus.build --release-version "$RELEASE_VERSION" --major-version "$AGENT_MAJOR_VERSION"
--base-dir $OMNIBUS_BASE_DIR --skip-deps --target-project ${DD_PROJECT} ${OMNIBUS_EXTRA_ARGS}
- curl -sSL "https://dd-package-tools.s3.amazonaws.com/dd-pkg/${DD_PKG_VERSION}/dd-pkg_Linux_${DD_PKG_ARCH}.tar.gz"
| tar -xz -C /usr/local/bin dd-pkg
- find $OMNIBUS_PACKAGE_DIR -iregex '.*\.\(deb\|rpm\)' | xargs dd-pkg lint
+ - "if [ -n \"$PACKAGE_REQUIRED_FILES_LIST\" ]; then\n find $OMNIBUS_PACKAGE_DIR\
+ \ \\( -name '*.deb' -or -name '*.rpm' \\) -a -not -name '*-dbg[_-]*' | xargs dd-pkg\
+ \ check-files --required-files ${PACKAGE_REQUIRED_FILES_LIST}\nfi\n"
- $S3_CP_CMD $OMNIBUS_PACKAGE_DIR/datadog-${DD_PROJECT}_*_${PACKAGE_ARCH}.deb $S3_ARTIFACTS_URI/$DESTINATION_DEB
stage: packaging
tags:
- arch:amd64
variables:
AGENT_MAJOR_VERSION: 6
DD_PKG_ARCH: x86_64
DD_PROJECT: agent
DESTINATION_DEB: datadog-agent_6_amd64.deb
KUBERNETES_CPU_REQUEST: 16
KUBERNETES_MEMORY_LIMIT: 32Gi
KUBERNETES_MEMORY_REQUEST: 32Gi
OMNIBUS_PACKAGE_ARTIFACT_DIR: $OMNIBUS_PACKAGE_DIR
PACKAGE_ARCH: amd64
+ PACKAGE_REQUIRED_FILES_LIST: test/required_files/agent-deb.txt
RELEASE_VERSION: $RELEASE_VERSION_6
agent_deb-x64-a7
agent_deb-x64-a7:
artifacts:
expire_in: 2 weeks
paths:
- $OMNIBUS_PACKAGE_DIR
cache:
- key:
files:
- omnibus/Gemfile
- release.json
prefix: omnibus-deps-$CI_JOB_NAME-$OMNIBUS_RUBY_VERSION-$OMNIBUS_SOFTWARE
paths:
- omnibus/vendor/bundle
image: 486234852809.dkr.ecr.us-east-1.amazonaws.com/ci/datadog-agent-buildimages/deb_x64$DATADOG_AGENT_BUILDIMAGES_SUFFIX:$DATADOG_AGENT_BUILDIMAGES
needs:
- datadog-agent-7-x64
rules:
- if: $CI_COMMIT_BRANCH =~ /^mq-working-branch-/
when: never
- when: on_success
script:
- pushd omnibus && bundle config set --local path 'vendor/bundle' && popd
- echo "About to package for $RELEASE_VERSION"
- set +x
- printf -- "$($CI_PROJECT_DIR/tools/ci/fetch_secret.sh $DEB_GPG_KEY)" | gpg --import
--batch
- EXIT="${PIPESTATUS[0]}"; if [ $EXIT -ne 0 ]; then echo "Unable to locate credentials
needs gitlab runner restart"; exit $EXIT; fi
- DEB_SIGNING_PASSPHRASE=$($CI_PROJECT_DIR/tools/ci/fetch_secret.sh $DEB_SIGNING_PASSPHRASE)
|| exit $?; export DEB_SIGNING_PASSPHRASE
- inv -e omnibus.build --release-version "$RELEASE_VERSION" --major-version "$AGENT_MAJOR_VERSION"
--base-dir $OMNIBUS_BASE_DIR --skip-deps --target-project ${DD_PROJECT} ${OMNIBUS_EXTRA_ARGS}
- curl -sSL "https://dd-package-tools.s3.amazonaws.com/dd-pkg/${DD_PKG_VERSION}/dd-pkg_Linux_${DD_PKG_ARCH}.tar.gz"
| tar -xz -C /usr/local/bin dd-pkg
- find $OMNIBUS_PACKAGE_DIR -iregex '.*\.\(deb\|rpm\)' | xargs dd-pkg lint
+ - "if [ -n \"$PACKAGE_REQUIRED_FILES_LIST\" ]; then\n find $OMNIBUS_PACKAGE_DIR\
+ \ \\( -name '*.deb' -or -name '*.rpm' \\) -a -not -name '*-dbg[_-]*' | xargs dd-pkg\
+ \ check-files --required-files ${PACKAGE_REQUIRED_FILES_LIST}\nfi\n"
- $S3_CP_CMD $OMNIBUS_PACKAGE_DIR/datadog-${DD_PROJECT}_*_${PACKAGE_ARCH}.deb $S3_ARTIFACTS_URI/$DESTINATION_DEB
stage: packaging
tags:
- arch:amd64
variables:
AGENT_MAJOR_VERSION: 7
DD_PKG_ARCH: x86_64
DD_PROJECT: agent
DESTINATION_DEB: datadog-agent_7_amd64.deb
KUBERNETES_CPU_REQUEST: 16
KUBERNETES_MEMORY_LIMIT: 32Gi
KUBERNETES_MEMORY_REQUEST: 32Gi
OMNIBUS_PACKAGE_ARTIFACT_DIR: $OMNIBUS_PACKAGE_DIR
PACKAGE_ARCH: amd64
+ PACKAGE_REQUIRED_FILES_LIST: test/required_files/agent-deb.txt
RELEASE_VERSION: $RELEASE_VERSION_7
agent_heroku_deb-x64-a6
agent_heroku_deb-x64-a6:
artifacts:
expire_in: 2 weeks
paths:
- $OMNIBUS_PACKAGE_DIR
cache:
- key:
files:
- omnibus/Gemfile
- release.json
prefix: omnibus-deps-$CI_JOB_NAME-$OMNIBUS_RUBY_VERSION-$OMNIBUS_SOFTWARE
paths:
- omnibus/vendor/bundle
image: 486234852809.dkr.ecr.us-east-1.amazonaws.com/ci/datadog-agent-buildimages/deb_x64$DATADOG_AGENT_BUILDIMAGES_SUFFIX:$DATADOG_AGENT_BUILDIMAGES
needs:
- go_mod_tidy_check
- build_system-probe-x64
- go_deps
- generate_minimized_btfs_x64
rules:
- if: $CI_COMMIT_BRANCH =~ /^mq-working-branch-/
when: never
- when: on_success
script:
- mkdir -p $GOPATH/pkg/mod && tar xJf modcache.tar.xz -C $GOPATH/pkg/mod
- rm -f modcache.tar.xz
- pushd omnibus && bundle config set --local path 'vendor/bundle' && popd
- echo "About to build for $RELEASE_VERSION"
- rm -rf $OMNIBUS_PACKAGE_DIR/*
- set +x
- printf -- "$($CI_PROJECT_DIR/tools/ci/fetch_secret.sh $DEB_GPG_KEY)" | gpg --import
--batch
- EXIT="${PIPESTATUS[0]}"; if [ $EXIT -ne 0 ]; then echo "Unable to locate credentials
needs gitlab runner restart"; exit $EXIT; fi
- DEB_SIGNING_PASSPHRASE=$($CI_PROJECT_DIR/tools/ci/fetch_secret.sh $DEB_SIGNING_PASSPHRASE)
|| exit $?; export DEB_SIGNING_PASSPHRASE
- tar -xf $CI_PROJECT_DIR/sysprobe-build-outputs.tar.xz
- mkdir -p /tmp/system-probe
- $S3_CP_CMD $S3_PERMANENT_ARTIFACTS_URI/clang-$CLANG_LLVM_VER.${PACKAGE_ARCH} /tmp/system-probe/clang-bpf
- $S3_CP_CMD $S3_PERMANENT_ARTIFACTS_URI/llc-$CLANG_LLVM_VER.${PACKAGE_ARCH} /tmp/system-probe/llc-bpf
- cp $CI_PROJECT_DIR/minimized-btfs.tar.xz /tmp/system-probe/minimized-btfs.tar.xz
- chmod 0744 /tmp/system-probe/clang-bpf /tmp/system-probe/llc-bpf
- inv -e omnibus.build --release-version "$RELEASE_VERSION" --major-version "$AGENT_MAJOR_VERSION"
--python-runtimes "$PYTHON_RUNTIMES" --base-dir $OMNIBUS_BASE_DIR ${USE_S3_CACHING}
--skip-deps --go-mod-cache="$GOPATH/pkg/mod" --system-probe-bin=/tmp/system-probe
--flavor heroku
- ls -la $OMNIBUS_PACKAGE_DIR
- curl -sSL "https://dd-package-tools.s3.amazonaws.com/dd-pkg/${DD_PKG_VERSION}/dd-pkg_Linux_${DD_PKG_ARCH}.tar.gz"
| tar -xz -C /usr/local/bin dd-pkg
- find $OMNIBUS_PACKAGE_DIR -iregex '.*\.\(deb\|rpm\)' | xargs dd-pkg lint
+ - "if [ -n \"$PACKAGE_REQUIRED_FILES_LIST\" ]; then\n find $OMNIBUS_PACKAGE_DIR\
+ \ \\( -name '*.deb' -or -name '*.rpm' \\) -a -not -name '*-dbg[_-]*' | xargs dd-pkg\
+ \ check-files --required-files ${PACKAGE_REQUIRED_FILES_LIST}\nfi\n"
- $S3_CP_CMD $OMNIBUS_PACKAGE_DIR/datadog-heroku-agent_*_${PACKAGE_ARCH}.deb $S3_ARTIFACTS_URI/$DESTINATION_DEB
- $S3_CP_CMD $OMNIBUS_PACKAGE_DIR/datadog-heroku-agent-dbg_*_${PACKAGE_ARCH}.deb
$S3_ARTIFACTS_URI/$DESTINATION_DBG_DEB
- $S3_CP_CMD $OMNIBUS_PACKAGE_DIR/version-manifest.json $S3_SBOM_STORAGE_URI/$CI_JOB_NAME/version-manifest.json
stage: package_build
tags:
- arch:amd64
variables:
AGENT_MAJOR_VERSION: 6
DD_PKG_ARCH: x86_64
DESTINATION_DBG_DEB: datadog-heroku-agent-dbg_6_amd64.deb
DESTINATION_DEB: datadog-heroku-agent_6_amd64.deb
KUBERNETES_CPU_REQUEST: 16
KUBERNETES_MEMORY_LIMIT: 32Gi
KUBERNETES_MEMORY_REQUEST: 32Gi
PACKAGE_ARCH: amd64
PYTHON_RUNTIMES: '2,3'
RELEASE_VERSION: $RELEASE_VERSION_6
agent_heroku_deb-x64-a7
agent_heroku_deb-x64-a7:
artifacts:
expire_in: 2 weeks
paths:
- $OMNIBUS_PACKAGE_DIR
cache:
- key:
files:
- omnibus/Gemfile
- release.json
prefix: omnibus-deps-$CI_JOB_NAME-$OMNIBUS_RUBY_VERSION-$OMNIBUS_SOFTWARE
paths:
- omnibus/vendor/bundle
image: 486234852809.dkr.ecr.us-east-1.amazonaws.com/ci/datadog-agent-buildimages/deb_x64$DATADOG_AGENT_BUILDIMAGES_SUFFIX:$DATADOG_AGENT_BUILDIMAGES
needs:
- go_mod_tidy_check
- build_system-probe-x64
- go_deps
- generate_minimized_btfs_x64
rules:
- if: $CI_COMMIT_BRANCH =~ /^mq-working-branch-/
when: never
- when: on_success
script:
- mkdir -p $GOPATH/pkg/mod && tar xJf modcache.tar.xz -C $GOPATH/pkg/mod
- rm -f modcache.tar.xz
- pushd omnibus && bundle config set --local path 'vendor/bundle' && popd
- echo "About to build for $RELEASE_VERSION"
- rm -rf $OMNIBUS_PACKAGE_DIR/*
- set +x
- printf -- "$($CI_PROJECT_DIR/tools/ci/fetch_secret.sh $DEB_GPG_KEY)" | gpg --import
--batch
- EXIT="${PIPESTATUS[0]}"; if [ $EXIT -ne 0 ]; then echo "Unable to locate credentials
needs gitlab runner restart"; exit $EXIT; fi
- DEB_SIGNING_PASSPHRASE=$($CI_PROJECT_DIR/tools/ci/fetch_secret.sh $DEB_SIGNING_PASSPHRASE)
|| exit $?; export DEB_SIGNING_PASSPHRASE
- tar -xf $CI_PROJECT_DIR/sysprobe-build-outputs.tar.xz
- mkdir -p /tmp/system-probe
- $S3_CP_CMD $S3_PERMANENT_ARTIFACTS_URI/clang-$CLANG_LLVM_VER.${PACKAGE_ARCH} /tmp/system-probe/clang-bpf
- $S3_CP_CMD $S3_PERMANENT_ARTIFACTS_URI/llc-$CLANG_LLVM_VER.${PACKAGE_ARCH} /tmp/system-probe/llc-bpf
- cp $CI_PROJECT_DIR/minimized-btfs.tar.xz /tmp/system-probe/minimized-btfs.tar.xz
- chmod 0744 /tmp/system-probe/clang-bpf /tmp/system-probe/llc-bpf
- inv -e omnibus.build --release-version "$RELEASE_VERSION" --major-version "$AGENT_MAJOR_VERSION"
--python-runtimes "$PYTHON_RUNTIMES" --base-dir $OMNIBUS_BASE_DIR ${USE_S3_CACHING}
--skip-deps --go-mod-cache="$GOPATH/pkg/mod" --system-probe-bin=/tmp/system-probe
--flavor heroku
- ls -la $OMNIBUS_PACKAGE_DIR
- curl -sSL "https://dd-package-tools.s3.amazonaws.com/dd-pkg/${DD_PKG_VERSION}/dd-pkg_Linux_${DD_PKG_ARCH}.tar.gz"
| tar -xz -C /usr/local/bin dd-pkg
- find $OMNIBUS_PACKAGE_DIR -iregex '.*\.\(deb\|rpm\)' | xargs dd-pkg lint
+ - "if [ -n \"$PACKAGE_REQUIRED_FILES_LIST\" ]; then\n find $OMNIBUS_PACKAGE_DIR\
+ \ \\( -name '*.deb' -or -name '*.rpm' \\) -a -not -name '*-dbg[_-]*' | xargs dd-pkg\
+ \ check-files --required-files ${PACKAGE_REQUIRED_FILES_LIST}\nfi\n"
- $S3_CP_CMD $OMNIBUS_PACKAGE_DIR/datadog-heroku-agent_*_${PACKAGE_ARCH}.deb $S3_ARTIFACTS_URI/$DESTINATION_DEB
- $S3_CP_CMD $OMNIBUS_PACKAGE_DIR/datadog-heroku-agent-dbg_*_${PACKAGE_ARCH}.deb
$S3_ARTIFACTS_URI/$DESTINATION_DBG_DEB
- $S3_CP_CMD $OMNIBUS_PACKAGE_DIR/version-manifest.json $S3_SBOM_STORAGE_URI/$CI_JOB_NAME/version-manifest.json
stage: package_build
tags:
- arch:amd64
variables:
AGENT_MAJOR_VERSION: 7
DD_PKG_ARCH: x86_64
DESTINATION_DBG_DEB: datadog-heroku-agent-dbg_7_amd64.deb
DESTINATION_DEB: datadog-heroku-agent_7_amd64.deb
KUBERNETES_CPU_REQUEST: 16
KUBERNETES_MEMORY_LIMIT: 32Gi
KUBERNETES_MEMORY_REQUEST: 32Gi
PACKAGE_ARCH: amd64
PYTHON_RUNTIMES: '3'
RELEASE_VERSION: $RELEASE_VERSION_7
agent_rpm-arm64-a6
agent_rpm-arm64-a6:
artifacts:
expire_in: 2 weeks
paths:
- $OMNIBUS_PACKAGE_DIR
before_script: null
cache:
- key:
files:
- omnibus/Gemfile
- release.json
prefix: omnibus-deps-$CI_JOB_NAME-$OMNIBUS_RUBY_VERSION-$OMNIBUS_SOFTWARE
paths:
- omnibus/vendor/bundle
image: 486234852809.dkr.ecr.us-east-1.amazonaws.com/ci/datadog-agent-buildimages/rpm_arm64$DATADOG_AGENT_ARMBUILDIMAGES_SUFFIX:$DATADOG_AGENT_ARMBUILDIMAGES
needs:
- datadog-agent-6-arm64
rules:
- if: $CI_COMMIT_BRANCH =~ /^mq-working-branch-/
when: never
- when: on_success
script:
- echo "About to build for $RELEASE_VERSION"
- pushd omnibus && bundle config set --local path 'vendor/bundle' && popd
- printf -- "$($CI_PROJECT_DIR/tools/ci/fetch_secret.sh $RPM_GPG_KEY)" | gpg --import
--batch
- EXIT="${PIPESTATUS[0]}"; if [ $EXIT -ne 0 ]; then echo "Unable to locate credentials
needs gitlab runner restart"; exit $EXIT; fi
- RPM_SIGNING_PASSPHRASE=$($CI_PROJECT_DIR/tools/ci/fetch_secret.sh $RPM_SIGNING_PASSPHRASE)
|| exit $?; export RPM_SIGNING_PASSPHRASE
- inv -e omnibus.build --release-version "$RELEASE_VERSION" --major-version "$AGENT_MAJOR_VERSION"
--base-dir $OMNIBUS_BASE_DIR --skip-deps --target-project=${DD_PROJECT} ${OMNIBUS_EXTRA_ARGS}
- ls -la $OMNIBUS_PACKAGE_DIR/
- curl -sSL "https://dd-package-tools.s3.amazonaws.com/dd-pkg/${DD_PKG_VERSION}/dd-pkg_Linux_${DD_PKG_ARCH}.tar.gz"
| tar -xz -C /usr/local/bin dd-pkg
- find $OMNIBUS_PACKAGE_DIR -iregex '.*\.\(deb\|rpm\)' | xargs dd-pkg lint
+ - "if [ -n \"$PACKAGE_REQUIRED_FILES_LIST\" ]; then\n find $OMNIBUS_PACKAGE_DIR\
+ \ \\( -name '*.deb' -or -name '*.rpm' \\) -a -not -name '*-dbg[_-]*' | xargs dd-pkg\
+ \ check-files --required-files ${PACKAGE_REQUIRED_FILES_LIST}\nfi\n"
stage: packaging
tags:
- arch:arm64
variables:
AGENT_MAJOR_VERSION: 6
DD_PKG_ARCH: arm64
DD_PROJECT: agent
KUBERNETES_CPU_REQUEST: 16
KUBERNETES_MEMORY_LIMIT: 32Gi
KUBERNETES_MEMORY_REQUEST: 32Gi
OMNIBUS_PACKAGE_ARTIFACT_DIR: $OMNIBUS_PACKAGE_DIR
PACKAGE_ARCH: arm64
+ PACKAGE_REQUIRED_FILES_LIST: test/required_files/agent-rpm.txt
RELEASE_VERSION: $RELEASE_VERSION_6
agent_rpm-arm64-a7
agent_rpm-arm64-a7:
artifacts:
expire_in: 2 weeks
paths:
- $OMNIBUS_PACKAGE_DIR
before_script: null
cache:
- key:
files:
- omnibus/Gemfile
- release.json
prefix: omnibus-deps-$CI_JOB_NAME-$OMNIBUS_RUBY_VERSION-$OMNIBUS_SOFTWARE
paths:
- omnibus/vendor/bundle
image: 486234852809.dkr.ecr.us-east-1.amazonaws.com/ci/datadog-agent-buildimages/rpm_arm64$DATADOG_AGENT_ARMBUILDIMAGES_SUFFIX:$DATADOG_AGENT_ARMBUILDIMAGES
needs:
- datadog-agent-7-arm64
rules:
- if: $CI_COMMIT_BRANCH =~ /^mq-working-branch-/
when: never
- when: on_success
script:
- echo "About to build for $RELEASE_VERSION"
- pushd omnibus && bundle config set --local path 'vendor/bundle' && popd
- printf -- "$($CI_PROJECT_DIR/tools/ci/fetch_secret.sh $RPM_GPG_KEY)" | gpg --import
--batch
- EXIT="${PIPESTATUS[0]}"; if [ $EXIT -ne 0 ]; then echo "Unable to locate credentials
needs gitlab runner restart"; exit $EXIT; fi
- RPM_SIGNING_PASSPHRASE=$($CI_PROJECT_DIR/tools/ci/fetch_secret.sh $RPM_SIGNING_PASSPHRASE)
|| exit $?; export RPM_SIGNING_PASSPHRASE
- inv -e omnibus.build --release-version "$RELEASE_VERSION" --major-version "$AGENT_MAJOR_VERSION"
--base-dir $OMNIBUS_BASE_DIR --skip-deps --target-project=${DD_PROJECT} ${OMNIBUS_EXTRA_ARGS}
- ls -la $OMNIBUS_PACKAGE_DIR/
- curl -sSL "https://dd-package-tools.s3.amazonaws.com/dd-pkg/${DD_PKG_VERSION}/dd-pkg_Linux_${DD_PKG_ARCH}.tar.gz"
| tar -xz -C /usr/local/bin dd-pkg
- find $OMNIBUS_PACKAGE_DIR -iregex '.*\.\(deb\|rpm\)' | xargs dd-pkg lint
+ - "if [ -n \"$PACKAGE_REQUIRED_FILES_LIST\" ]; then\n find $OMNIBUS_PACKAGE_DIR\
+ \ \\( -name '*.deb' -or -name '*.rpm' \\) -a -not -name '*-dbg[_-]*' | xargs dd-pkg\
+ \ check-files --required-files ${PACKAGE_REQUIRED_FILES_LIST}\nfi\n"
stage: packaging
tags:
- arch:arm64
variables:
AGENT_MAJOR_VERSION: 7
DD_PKG_ARCH: arm64
DD_PROJECT: agent
KUBERNETES_CPU_REQUEST: 16
KUBERNETES_MEMORY_LIMIT: 32Gi
KUBERNETES_MEMORY_REQUEST: 32Gi
OMNIBUS_PACKAGE_ARTIFACT_DIR: $OMNIBUS_PACKAGE_DIR
PACKAGE_ARCH: arm64
+ PACKAGE_REQUIRED_FILES_LIST: test/required_files/agent-rpm.txt
RELEASE_VERSION: $RELEASE_VERSION_7
agent_rpm-x64-a6
agent_rpm-x64-a6:
artifacts:
expire_in: 2 weeks
paths:
- $OMNIBUS_PACKAGE_DIR
before_script: null
cache:
- key:
files:
- omnibus/Gemfile
- release.json
prefix: omnibus-deps-$CI_JOB_NAME-$OMNIBUS_RUBY_VERSION-$OMNIBUS_SOFTWARE
paths:
- omnibus/vendor/bundle
image: 486234852809.dkr.ecr.us-east-1.amazonaws.com/ci/datadog-agent-buildimages/rpm_x64$DATADOG_AGENT_BUILDIMAGES_SUFFIX:$DATADOG_AGENT_BUILDIMAGES
needs:
- datadog-agent-6-x64
rules:
- if: $CI_COMMIT_BRANCH =~ /^mq-working-branch-/
when: never
- when: on_success
script:
- echo "About to build for $RELEASE_VERSION"
- pushd omnibus && bundle config set --local path 'vendor/bundle' && popd
- printf -- "$($CI_PROJECT_DIR/tools/ci/fetch_secret.sh $RPM_GPG_KEY)" | gpg --import
--batch
- EXIT="${PIPESTATUS[0]}"; if [ $EXIT -ne 0 ]; then echo "Unable to locate credentials
needs gitlab runner restart"; exit $EXIT; fi
- RPM_SIGNING_PASSPHRASE=$($CI_PROJECT_DIR/tools/ci/fetch_secret.sh $RPM_SIGNING_PASSPHRASE)
|| exit $?; export RPM_SIGNING_PASSPHRASE
- inv -e omnibus.build --release-version "$RELEASE_VERSION" --major-version "$AGENT_MAJOR_VERSION"
--base-dir $OMNIBUS_BASE_DIR --skip-deps --target-project=${DD_PROJECT} ${OMNIBUS_EXTRA_ARGS}
- ls -la $OMNIBUS_PACKAGE_DIR/
- curl -sSL "https://dd-package-tools.s3.amazonaws.com/dd-pkg/${DD_PKG_VERSION}/dd-pkg_Linux_${DD_PKG_ARCH}.tar.gz"
| tar -xz -C /usr/local/bin dd-pkg
- find $OMNIBUS_PACKAGE_DIR -iregex '.*\.\(deb\|rpm\)' | xargs dd-pkg lint
+ - "if [ -n \"$PACKAGE_REQUIRED_FILES_LIST\" ]; then\n find $OMNIBUS_PACKAGE_DIR\
+ \ \\( -name '*.deb' -or -name '*.rpm' \\) -a -not -name '*-dbg[_-]*' | xargs dd-pkg\
+ \ check-files --required-files ${PACKAGE_REQUIRED_FILES_LIST}\nfi\n"
stage: packaging
tags:
- arch:amd64
variables:
AGENT_MAJOR_VERSION: 6
DD_PKG_ARCH: x86_64
DD_PROJECT: agent
KUBERNETES_CPU_REQUEST: 16
KUBERNETES_MEMORY_LIMIT: 32Gi
KUBERNETES_MEMORY_REQUEST: 32Gi
OMNIBUS_PACKAGE_ARTIFACT_DIR: $OMNIBUS_PACKAGE_DIR
PACKAGE_ARCH: amd64
+ PACKAGE_REQUIRED_FILES_LIST: test/required_files/agent-rpm.txt
RELEASE_VERSION: $RELEASE_VERSION_6
agent_rpm-x64-a7
agent_rpm-x64-a7:
artifacts:
expire_in: 2 weeks
paths:
- $OMNIBUS_PACKAGE_DIR
before_script: null
cache:
- key:
files:
- omnibus/Gemfile
- release.json
prefix: omnibus-deps-$CI_JOB_NAME-$OMNIBUS_RUBY_VERSION-$OMNIBUS_SOFTWARE
paths:
- omnibus/vendor/bundle
image: 486234852809.dkr.ecr.us-east-1.amazonaws.com/ci/datadog-agent-buildimages/rpm_x64$DATADOG_AGENT_BUILDIMAGES_SUFFIX:$DATADOG_AGENT_BUILDIMAGES
needs:
- datadog-agent-7-x64
rules:
- if: $CI_COMMIT_BRANCH =~ /^mq-working-branch-/
when: never
- when: on_success
script:
- echo "About to build for $RELEASE_VERSION"
- pushd omnibus && bundle config set --local path 'vendor/bundle' && popd
- printf -- "$($CI_PROJECT_DIR/tools/ci/fetch_secret.sh $RPM_GPG_KEY)" | gpg --import
--batch
- EXIT="${PIPESTATUS[0]}"; if [ $EXIT -ne 0 ]; then echo "Unable to locate credentials
needs gitlab runner restart"; exit $EXIT; fi
- RPM_SIGNING_PASSPHRASE=$($CI_PROJECT_DIR/tools/ci/fetch_secret.sh $RPM_SIGNING_PASSPHRASE)
|| exit $?; export RPM_SIGNING_PASSPHRASE
- inv -e omnibus.build --release-version "$RELEASE_VERSION" --major-version "$AGENT_MAJOR_VERSION"
--base-dir $OMNIBUS_BASE_DIR --skip-deps --target-project=${DD_PROJECT} ${OMNIBUS_EXTRA_ARGS}
- ls -la $OMNIBUS_PACKAGE_DIR/
- curl -sSL "https://dd-package-tools.s3.amazonaws.com/dd-pkg/${DD_PKG_VERSION}/dd-pkg_Linux_${DD_PKG_ARCH}.tar.gz"
| tar -xz -C /usr/local/bin dd-pkg
- find $OMNIBUS_PACKAGE_DIR -iregex '.*\.\(deb\|rpm\)' | xargs dd-pkg lint
+ - "if [ -n \"$PACKAGE_REQUIRED_FILES_LIST\" ]; then\n find $OMNIBUS_PACKAGE_DIR\
+ \ \\( -name '*.deb' -or -name '*.rpm' \\) -a -not -name '*-dbg[_-]*' | xargs dd-pkg\
+ \ check-files --required-files ${PACKAGE_REQUIRED_FILES_LIST}\nfi\n"
stage: packaging
tags:
- arch:amd64
variables:
AGENT_MAJOR_VERSION: 7
DD_PKG_ARCH: x86_64
DD_PROJECT: agent
KUBERNETES_CPU_REQUEST: 16
KUBERNETES_MEMORY_LIMIT: 32Gi
KUBERNETES_MEMORY_REQUEST: 32Gi
OMNIBUS_PACKAGE_ARTIFACT_DIR: $OMNIBUS_PACKAGE_DIR
PACKAGE_ARCH: amd64
+ PACKAGE_REQUIRED_FILES_LIST: test/required_files/agent-rpm.txt
RELEASE_VERSION: $RELEASE_VERSION_7
agent_suse-arm64-a7
agent_suse-arm64-a7:
artifacts:
expire_in: 2 weeks
paths:
- $OMNIBUS_PACKAGE_DIR_SUSE
before_script: null
cache:
- key:
files:
- omnibus/Gemfile
- release.json
prefix: omnibus-deps-$CI_JOB_NAME-$OMNIBUS_RUBY_VERSION-$OMNIBUS_SOFTWARE
paths:
- omnibus/vendor/bundle
image: 486234852809.dkr.ecr.us-east-1.amazonaws.com/ci/datadog-agent-buildimages/rpm_arm64$DATADOG_AGENT_ARMBUILDIMAGES_SUFFIX:$DATADOG_AGENT_ARMBUILDIMAGES
needs:
- datadog-agent-7-arm64
rules:
- if: $CI_COMMIT_BRANCH =~ /^mq-working-branch-/
when: never
- when: on_success
script:
- echo "About to build for $RELEASE_VERSION"
- pushd omnibus && bundle config set --local path 'vendor/bundle' && popd
- printf -- "$($CI_PROJECT_DIR/tools/ci/fetch_secret.sh $RPM_GPG_KEY)" | gpg --import
--batch
- EXIT="${PIPESTATUS[0]}"; if [ $EXIT -ne 0 ]; then echo "Unable to locate credentials
needs gitlab runner restart"; exit $EXIT; fi
- RPM_SIGNING_PASSPHRASE=$($CI_PROJECT_DIR/tools/ci/fetch_secret.sh $RPM_SIGNING_PASSPHRASE)
|| exit $?; export RPM_SIGNING_PASSPHRASE
- inv -e omnibus.build --release-version "$RELEASE_VERSION" --major-version "$AGENT_MAJOR_VERSION"
--base-dir $OMNIBUS_BASE_DIR --skip-deps --target-project=${DD_PROJECT} ${OMNIBUS_EXTRA_ARGS}
- ls -la $OMNIBUS_PACKAGE_DIR/
- curl -sSL "https://dd-package-tools.s3.amazonaws.com/dd-pkg/${DD_PKG_VERSION}/dd-pkg_Linux_${DD_PKG_ARCH}.tar.gz"
| tar -xz -C /usr/local/bin dd-pkg
- find $OMNIBUS_PACKAGE_DIR -iregex '.*\.\(deb\|rpm\)' | xargs dd-pkg lint
+ - "if [ -n \"$PACKAGE_REQUIRED_FILES_LIST\" ]; then\n find $OMNIBUS_PACKAGE_DIR\
+ \ \\( -name '*.deb' -or -name '*.rpm' \\) -a -not -name '*-dbg[_-]*' | xargs dd-pkg\
+ \ check-files --required-files ${PACKAGE_REQUIRED_FILES_LIST}\nfi\n"
- mkdir -p $OMNIBUS_PACKAGE_DIR_SUSE && mv $OMNIBUS_PACKAGE_DIR/*.rpm $OMNIBUS_PACKAGE_DIR_SUSE/
stage: packaging
tags:
- arch:arm64
variables:
AGENT_MAJOR_VERSION: 7
DD_PKG_ARCH: arm64
DD_PRODUCT: agent
KUBERNETES_CPU_REQUEST: 16
KUBERNETES_MEMORY_LIMIT: 32Gi
KUBERNETES_MEMORY_REQUEST: 32Gi
OMNIBUS_EXTRA_ARGS: --host-distribution=suse
OMNIBUS_PACKAGE_ARTIFACT_DIR: $OMNIBUS_PACKAGE_DIR
PACKAGE_ARCH: arm64
+ PACKAGE_REQUIRED_FILES_LIST: test/required_files/agent-rpm.txt
RELEASE_VERSION: $RELEASE_VERSION_7
agent_suse-x64-a6
agent_suse-x64-a6:
artifacts:
expire_in: 2 weeks
paths:
- $OMNIBUS_PACKAGE_DIR_SUSE
before_script: null
cache:
- key:
files:
- omnibus/Gemfile
- release.json
prefix: omnibus-deps-$CI_JOB_NAME-$OMNIBUS_RUBY_VERSION-$OMNIBUS_SOFTWARE
paths:
- omnibus/vendor/bundle
image: 486234852809.dkr.ecr.us-east-1.amazonaws.com/ci/datadog-agent-buildimages/rpm_x64$DATADOG_AGENT_BUILDIMAGES_SUFFIX:$DATADOG_AGENT_BUILDIMAGES
needs:
- datadog-agent-6-x64
rules:
- if: $CI_COMMIT_BRANCH =~ /^mq-working-branch-/
when: never
- when: on_success
script:
- echo "About to build for $RELEASE_VERSION"
- pushd omnibus && bundle config set --local path 'vendor/bundle' && popd
- printf -- "$($CI_PROJECT_DIR/tools/ci/fetch_secret.sh $RPM_GPG_KEY)" | gpg --import
--batch
- EXIT="${PIPESTATUS[0]}"; if [ $EXIT -ne 0 ]; then echo "Unable to locate credentials
needs gitlab runner restart"; exit $EXIT; fi
- RPM_SIGNING_PASSPHRASE=$($CI_PROJECT_DIR/tools/ci/fetch_secret.sh $RPM_SIGNING_PASSPHRASE)
|| exit $?; export RPM_SIGNING_PASSPHRASE
- inv -e omnibus.build --release-version "$RELEASE_VERSION" --major-version "$AGENT_MAJOR_VERSION"
--base-dir $OMNIBUS_BASE_DIR --skip-deps --target-project=${DD_PROJECT} ${OMNIBUS_EXTRA_ARGS}
- ls -la $OMNIBUS_PACKAGE_DIR/
- curl -sSL "https://dd-package-tools.s3.amazonaws.com/dd-pkg/${DD_PKG_VERSION}/dd-pkg_Linux_${DD_PKG_ARCH}.tar.gz"
| tar -xz -C /usr/local/bin dd-pkg
- find $OMNIBUS_PACKAGE_DIR -iregex '.*\.\(deb\|rpm\)' | xargs dd-pkg lint
+ - "if [ -n \"$PACKAGE_REQUIRED_FILES_LIST\" ]; then\n find $OMNIBUS_PACKAGE_DIR\
+ \ \\( -name '*.deb' -or -name '*.rpm' \\) -a -not -name '*-dbg[_-]*' | xargs dd-pkg\
+ \ check-files --required-files ${PACKAGE_REQUIRED_FILES_LIST}\nfi\n"
- mkdir -p $OMNIBUS_PACKAGE_DIR_SUSE && mv $OMNIBUS_PACKAGE_DIR/*.rpm $OMNIBUS_PACKAGE_DIR_SUSE/
stage: packaging
tags:
- arch:amd64
variables:
AGENT_MAJOR_VERSION: 6
DD_PKG_ARCH: x86_64
DD_PRODUCT: agent
KUBERNETES_CPU_REQUEST: 16
KUBERNETES_MEMORY_LIMIT: 32Gi
KUBERNETES_MEMORY_REQUEST: 32Gi
OMNIBUS_EXTRA_ARGS: --host-distribution=suse
OMNIBUS_PACKAGE_ARTIFACT_DIR: $OMNIBUS_PACKAGE_DIR
PACKAGE_ARCH: amd64
+ PACKAGE_REQUIRED_FILES_LIST: test/required_files/agent-rpm.txt
RELEASE_VERSION: $RELEASE_VERSION_6
agent_suse-x64-a7
agent_suse-x64-a7:
artifacts:
expire_in: 2 weeks
paths:
- $OMNIBUS_PACKAGE_DIR_SUSE
before_script: null
cache:
- key:
files:
- omnibus/Gemfile
- release.json
prefix: omnibus-deps-$CI_JOB_NAME-$OMNIBUS_RUBY_VERSION-$OMNIBUS_SOFTWARE
paths:
- omnibus/vendor/bundle
image: 486234852809.dkr.ecr.us-east-1.amazonaws.com/ci/datadog-agent-buildimages/rpm_x64$DATADOG_AGENT_BUILDIMAGES_SUFFIX:$DATADOG_AGENT_BUILDIMAGES
needs:
- datadog-agent-7-x64
rules:
- if: $CI_COMMIT_BRANCH =~ /^mq-working-branch-/
when: never
- when: on_success
script:
- echo "About to build for $RELEASE_VERSION"
- pushd omnibus && bundle config set --local path 'vendor/bundle' && popd
- printf -- "$($CI_PROJECT_DIR/tools/ci/fetch_secret.sh $RPM_GPG_KEY)" | gpg --import
--batch
- EXIT="${PIPESTATUS[0]}"; if [ $EXIT -ne 0 ]; then echo "Unable to locate credentials
needs gitlab runner restart"; exit $EXIT; fi
- RPM_SIGNING_PASSPHRASE=$($CI_PROJECT_DIR/tools/ci/fetch_secret.sh $RPM_SIGNING_PASSPHRASE)
|| exit $?; export RPM_SIGNING_PASSPHRASE
- inv -e omnibus.build --release-version "$RELEASE_VERSION" --major-version "$AGENT_MAJOR_VERSION"
--base-dir $OMNIBUS_BASE_DIR --skip-deps --target-project=${DD_PROJECT} ${OMNIBUS_EXTRA_ARGS}
- ls -la $OMNIBUS_PACKAGE_DIR/
- curl -sSL "https://dd-package-tools.s3.amazonaws.com/dd-pkg/${DD_PKG_VERSION}/dd-pkg_Linux_${DD_PKG_ARCH}.tar.gz"
| tar -xz -C /usr/local/bin dd-pkg
- find $OMNIBUS_PACKAGE_DIR -iregex '.*\.\(deb\|rpm\)' | xargs dd-pkg lint
+ - "if [ -n \"$PACKAGE_REQUIRED_FILES_LIST\" ]; then\n find $OMNIBUS_PACKAGE_DIR\
+ \ \\( -name '*.deb' -or -name '*.rpm' \\) -a -not -name '*-dbg[_-]*' | xargs dd-pkg\
+ \ check-files --required-files ${PACKAGE_REQUIRED_FILES_LIST}\nfi\n"
- mkdir -p $OMNIBUS_PACKAGE_DIR_SUSE && mv $OMNIBUS_PACKAGE_DIR/*.rpm $OMNIBUS_PACKAGE_DIR_SUSE/
stage: packaging
tags:
- arch:amd64
variables:
AGENT_MAJOR_VERSION: 7
DD_PKG_ARCH: x86_64
DD_PRODUCT: agent
KUBERNETES_CPU_REQUEST: 16
KUBERNETES_MEMORY_LIMIT: 32Gi
KUBERNETES_MEMORY_REQUEST: 32Gi
OMNIBUS_EXTRA_ARGS: --host-distribution=suse
OMNIBUS_PACKAGE_ARTIFACT_DIR: $OMNIBUS_PACKAGE_DIR
PACKAGE_ARCH: amd64
+ PACKAGE_REQUIRED_FILES_LIST: test/required_files/agent-rpm.txt
RELEASE_VERSION: $RELEASE_VERSION_7
dogstatsd_deb-arm64
dogstatsd_deb-arm64:
artifacts:
expire_in: 2 weeks
paths:
- $OMNIBUS_PACKAGE_DIR
cache:
- key:
files:
- omnibus/Gemfile
- release.json
prefix: omnibus-deps-$CI_JOB_NAME-$OMNIBUS_RUBY_VERSION-$OMNIBUS_SOFTWARE
paths:
- omnibus/vendor/bundle
image: 486234852809.dkr.ecr.us-east-1.amazonaws.com/ci/datadog-agent-buildimages/deb_arm64$DATADOG_AGENT_ARMBUILDIMAGES_SUFFIX:$DATADOG_AGENT_ARMBUILDIMAGES
needs:
- dogstatsd-arm64
rules:
- if: $CI_COMMIT_BRANCH =~ /^mq-working-branch-/
when: never
- when: on_success
script:
- pushd omnibus && bundle config set --local path 'vendor/bundle' && popd
- echo "About to package for $RELEASE_VERSION"
- set +x
- printf -- "$($CI_PROJECT_DIR/tools/ci/fetch_secret.sh $DEB_GPG_KEY)" | gpg --import
--batch
- EXIT="${PIPESTATUS[0]}"; if [ $EXIT -ne 0 ]; then echo "Unable to locate credentials
needs gitlab runner restart"; exit $EXIT; fi
- DEB_SIGNING_PASSPHRASE=$($CI_PROJECT_DIR/tools/ci/fetch_secret.sh $DEB_SIGNING_PASSPHRASE)
|| exit $?; export DEB_SIGNING_PASSPHRASE
- inv -e omnibus.build --release-version "$RELEASE_VERSION" --major-version "$AGENT_MAJOR_VERSION"
--base-dir $OMNIBUS_BASE_DIR --skip-deps --target-project ${DD_PROJECT} ${OMNIBUS_EXTRA_ARGS}
- curl -sSL "https://dd-package-tools.s3.amazonaws.com/dd-pkg/${DD_PKG_VERSION}/dd-pkg_Linux_${DD_PKG_ARCH}.tar.gz"
| tar -xz -C /usr/local/bin dd-pkg
- find $OMNIBUS_PACKAGE_DIR -iregex '.*\.\(deb\|rpm\)' | xargs dd-pkg lint
+ - "if [ -n \"$PACKAGE_REQUIRED_FILES_LIST\" ]; then\n find $OMNIBUS_PACKAGE_DIR\
+ \ \\( -name '*.deb' -or -name '*.rpm' \\) -a -not -name '*-dbg[_-]*' | xargs dd-pkg\
+ \ check-files --required-files ${PACKAGE_REQUIRED_FILES_LIST}\nfi\n"
- $S3_CP_CMD $OMNIBUS_PACKAGE_DIR/datadog-${DD_PROJECT}_*_${PACKAGE_ARCH}.deb $S3_ARTIFACTS_URI/$DESTINATION_DEB
stage: packaging
tags:
- arch:arm64
variables:
AGENT_MAJOR_VERSION: 7
DD_PKG_ARCH: arm64
DD_PROJECT: dogstatsd
DESTINATION_DEB: datadog-dogstatsd_arm64.deb
KUBERNETES_CPU_REQUEST: 16
KUBERNETES_MEMORY_LIMIT: 32Gi
KUBERNETES_MEMORY_REQUEST: 32Gi
OMNIBUS_PACKAGE_ARTIFACT_DIR: $OMNIBUS_PACKAGE_DIR
PACKAGE_ARCH: arm64
+ PACKAGE_REQUIRED_FILES_LIST: test/required_files/dogstatsd-deb.txt
RELEASE_VERSION: $RELEASE_VERSION_7
dogstatsd_deb-x64
dogstatsd_deb-x64:
artifacts:
expire_in: 2 weeks
paths:
- $OMNIBUS_PACKAGE_DIR
cache:
- key:
files:
- omnibus/Gemfile
- release.json
prefix: omnibus-deps-$CI_JOB_NAME-$OMNIBUS_RUBY_VERSION-$OMNIBUS_SOFTWARE
paths:
- omnibus/vendor/bundle
image: 486234852809.dkr.ecr.us-east-1.amazonaws.com/ci/datadog-agent-buildimages/deb_x64$DATADOG_AGENT_BUILDIMAGES_SUFFIX:$DATADOG_AGENT_BUILDIMAGES
needs:
- dogstatsd-x64
rules:
- if: $CI_COMMIT_BRANCH =~ /^mq-working-branch-/
when: never
- when: on_success
script:
- pushd omnibus && bundle config set --local path 'vendor/bundle' && popd
- echo "About to package for $RELEASE_VERSION"
- set +x
- printf -- "$($CI_PROJECT_DIR/tools/ci/fetch_secret.sh $DEB_GPG_KEY)" | gpg --import
--batch
- EXIT="${PIPESTATUS[0]}"; if [ $EXIT -ne 0 ]; then echo "Unable to locate credentials
needs gitlab runner restart"; exit $EXIT; fi
- DEB_SIGNING_PASSPHRASE=$($CI_PROJECT_DIR/tools/ci/fetch_secret.sh $DEB_SIGNING_PASSPHRASE)
|| exit $?; export DEB_SIGNING_PASSPHRASE
- inv -e omnibus.build --release-version "$RELEASE_VERSION" --major-version "$AGENT_MAJOR_VERSION"
--base-dir $OMNIBUS_BASE_DIR --skip-deps --target-project ${DD_PROJECT} ${OMNIBUS_EXTRA_ARGS}
- curl -sSL "https://dd-package-tools.s3.amazonaws.com/dd-pkg/${DD_PKG_VERSION}/dd-pkg_Linux_${DD_PKG_ARCH}.tar.gz"
| tar -xz -C /usr/local/bin dd-pkg
- find $OMNIBUS_PACKAGE_DIR -iregex '.*\.\(deb\|rpm\)' | xargs dd-pkg lint
+ - "if [ -n \"$PACKAGE_REQUIRED_FILES_LIST\" ]; then\n find $OMNIBUS_PACKAGE_DIR\
+ \ \\( -name '*.deb' -or -name '*.rpm' \\) -a -not -name '*-dbg[_-]*' | xargs dd-pkg\
+ \ check-files --required-files ${PACKAGE_REQUIRED_FILES_LIST}\nfi\n"
- $S3_CP_CMD $OMNIBUS_PACKAGE_DIR/datadog-${DD_PROJECT}_*_${PACKAGE_ARCH}.deb $S3_ARTIFACTS_URI/$DESTINATION_DEB
stage: packaging
tags:
- arch:amd64
variables:
AGENT_MAJOR_VERSION: 7
DD_PKG_ARCH: x86_64
DD_PROJECT: dogstatsd
DESTINATION_DEB: datadog-dogstatsd_amd64.deb
KUBERNETES_CPU_REQUEST: 16
KUBERNETES_MEMORY_LIMIT: 32Gi
KUBERNETES_MEMORY_REQUEST: 32Gi
OMNIBUS_PACKAGE_ARTIFACT_DIR: $OMNIBUS_PACKAGE_DIR
PACKAGE_ARCH: amd64
+ PACKAGE_REQUIRED_FILES_LIST: test/required_files/dogstatsd-deb.txt
RELEASE_VERSION: $RELEASE_VERSION_7
dogstatsd_rpm-x64
dogstatsd_rpm-x64:
artifacts:
expire_in: 2 weeks
paths:
- $OMNIBUS_PACKAGE_DIR
before_script: null
cache:
- key:
files:
- omnibus/Gemfile
- release.json
prefix: omnibus-deps-$CI_JOB_NAME-$OMNIBUS_RUBY_VERSION-$OMNIBUS_SOFTWARE
paths:
- omnibus/vendor/bundle
image: 486234852809.dkr.ecr.us-east-1.amazonaws.com/ci/datadog-agent-buildimages/rpm_x64$DATADOG_AGENT_BUILDIMAGES_SUFFIX:$DATADOG_AGENT_BUILDIMAGES
needs:
- dogstatsd-x64
rules:
- if: $CI_COMMIT_BRANCH =~ /^mq-working-branch-/
when: never
- when: on_success
script:
- echo "About to build for $RELEASE_VERSION"
- pushd omnibus && bundle config set --local path 'vendor/bundle' && popd
- printf -- "$($CI_PROJECT_DIR/tools/ci/fetch_secret.sh $RPM_GPG_KEY)" | gpg --import
--batch
- EXIT="${PIPESTATUS[0]}"; if [ $EXIT -ne 0 ]; then echo "Unable to locate credentials
needs gitlab runner restart"; exit $EXIT; fi
- RPM_SIGNING_PASSPHRASE=$($CI_PROJECT_DIR/tools/ci/fetch_secret.sh $RPM_SIGNING_PASSPHRASE)
|| exit $?; export RPM_SIGNING_PASSPHRASE
- inv -e omnibus.build --release-version "$RELEASE_VERSION" --major-version "$AGENT_MAJOR_VERSION"
--base-dir $OMNIBUS_BASE_DIR --skip-deps --target-project=${DD_PROJECT} ${OMNIBUS_EXTRA_ARGS}
- ls -la $OMNIBUS_PACKAGE_DIR/
- curl -sSL "https://dd-package-tools.s3.amazonaws.com/dd-pkg/${DD_PKG_VERSION}/dd-pkg_Linux_${DD_PKG_ARCH}.tar.gz"
| tar -xz -C /usr/local/bin dd-pkg
- find $OMNIBUS_PACKAGE_DIR -iregex '.*\.\(deb\|rpm\)' | xargs dd-pkg lint
+ - "if [ -n \"$PACKAGE_REQUIRED_FILES_LIST\" ]; then\n find $OMNIBUS_PACKAGE_DIR\
+ \ \\( -name '*.deb' -or -name '*.rpm' \\) -a -not -name '*-dbg[_-]*' | xargs dd-pkg\
+ \ check-files --required-files ${PACKAGE_REQUIRED_FILES_LIST}\nfi\n"
stage: packaging
tags:
- arch:amd64
variables:
AGENT_MAJOR_VERSION: 7
DD_PKG_ARCH: x86_64
DD_PROJECT: dogstatsd
KUBERNETES_CPU_REQUEST: 16
KUBERNETES_MEMORY_LIMIT: 32Gi
KUBERNETES_MEMORY_REQUEST: 32Gi
OMNIBUS_PACKAGE_ARTIFACT_DIR: $OMNIBUS_PACKAGE_DIR
PACKAGE_ARCH: amd64
+ PACKAGE_REQUIRED_FILES_LIST: test/required_files/dogstatsd-rpm.txt
RELEASE_VERSION: $RELEASE_VERSION_7
dogstatsd_suse-x64
dogstatsd_suse-x64:
artifacts:
expire_in: 2 weeks
paths:
- $OMNIBUS_PACKAGE_DIR_SUSE
before_script: null
cache:
- key:
files:
- omnibus/Gemfile
- release.json
prefix: omnibus-deps-$CI_JOB_NAME-$OMNIBUS_RUBY_VERSION-$OMNIBUS_SOFTWARE
paths:
- omnibus/vendor/bundle
image: 486234852809.dkr.ecr.us-east-1.amazonaws.com/ci/datadog-agent-buildimages/rpm_x64$DATADOG_AGENT_BUILDIMAGES_SUFFIX:$DATADOG_AGENT_BUILDIMAGES
needs:
- dogstatsd-x64
rules:
- if: $CI_COMMIT_BRANCH =~ /^mq-working-branch-/
when: never
- when: on_success
script:
- echo "About to build for $RELEASE_VERSION"
- pushd omnibus && bundle config set --local path 'vendor/bundle' && popd
- printf -- "$($CI_PROJECT_DIR/tools/ci/fetch_secret.sh $RPM_GPG_KEY)" | gpg --import
--batch
- EXIT="${PIPESTATUS[0]}"; if [ $EXIT -ne 0 ]; then echo "Unable to locate credentials
needs gitlab runner restart"; exit $EXIT; fi
- RPM_SIGNING_PASSPHRASE=$($CI_PROJECT_DIR/tools/ci/fetch_secret.sh $RPM_SIGNING_PASSPHRASE)
|| exit $?; export RPM_SIGNING_PASSPHRASE
- inv -e omnibus.build --release-version "$RELEASE_VERSION" --major-version "$AGENT_MAJOR_VERSION"
--base-dir $OMNIBUS_BASE_DIR --skip-deps --target-project=${DD_PROJECT} ${OMNIBUS_EXTRA_ARGS}
- ls -la $OMNIBUS_PACKAGE_DIR/
- curl -sSL "https://dd-package-tools.s3.amazonaws.com/dd-pkg/${DD_PKG_VERSION}/dd-pkg_Linux_${DD_PKG_ARCH}.tar.gz"
| tar -xz -C /usr/local/bin dd-pkg
- find $OMNIBUS_PACKAGE_DIR -iregex '.*\.\(deb\|rpm\)' | xargs dd-pkg lint
+ - "if [ -n \"$PACKAGE_REQUIRED_FILES_LIST\" ]; then\n find $OMNIBUS_PACKAGE_DIR\
+ \ \\( -name '*.deb' -or -name '*.rpm' \\) -a -not -name '*-dbg[_-]*' | xargs dd-pkg\
+ \ check-files --required-files ${PACKAGE_REQUIRED_FILES_LIST}\nfi\n"
- mkdir -p $OMNIBUS_PACKAGE_DIR_SUSE && mv $OMNIBUS_PACKAGE_DIR/*.rpm $OMNIBUS_PACKAGE_DIR_SUSE/
stage: packaging
tags:
- arch:amd64
variables:
AGENT_MAJOR_VERSION: 7
DD_PKG_ARCH: x86_64
DD_PROJECT: dogstatsd
KUBERNETES_CPU_REQUEST: 16
KUBERNETES_MEMORY_LIMIT: 32Gi
KUBERNETES_MEMORY_REQUEST: 32Gi
OMNIBUS_EXTRA_ARGS: --host-distribution=suse
OMNIBUS_PACKAGE_ARTIFACT_DIR: $OMNIBUS_PACKAGE_DIR
PACKAGE_ARCH: amd64
+ PACKAGE_REQUIRED_FILES_LIST: test/required_files/dogstatsd-rpm.txt
RELEASE_VERSION: $RELEASE_VERSION_7
installer_deb-amd64
installer_deb-amd64:
artifacts:
expire_in: 2 weeks
paths:
- $OMNIBUS_PACKAGE_DIR
cache:
- key:
files:
- omnibus/Gemfile
- release.json
prefix: omnibus-deps-$CI_JOB_NAME-$OMNIBUS_RUBY_VERSION-$OMNIBUS_SOFTWARE
paths:
- omnibus/vendor/bundle
image: 486234852809.dkr.ecr.us-east-1.amazonaws.com/ci/datadog-agent-buildimages/deb_x64$DATADOG_AGENT_BUILDIMAGES_SUFFIX:$DATADOG_AGENT_BUILDIMAGES
needs:
- installer-amd64
rules:
- if: $CI_COMMIT_BRANCH =~ /^mq-working-branch-/
when: never
- when: on_success
script:
- pushd omnibus && bundle config set --local path 'vendor/bundle' && popd
- echo "About to package for $RELEASE_VERSION"
- set +x
- printf -- "$($CI_PROJECT_DIR/tools/ci/fetch_secret.sh $DEB_GPG_KEY)" | gpg --import
--batch
- EXIT="${PIPESTATUS[0]}"; if [ $EXIT -ne 0 ]; then echo "Unable to locate credentials
needs gitlab runner restart"; exit $EXIT; fi
- DEB_SIGNING_PASSPHRASE=$($CI_PROJECT_DIR/tools/ci/fetch_secret.sh $DEB_SIGNING_PASSPHRASE)
|| exit $?; export DEB_SIGNING_PASSPHRASE
- inv -e omnibus.build --release-version "$RELEASE_VERSION" --major-version "$AGENT_MAJOR_VERSION"
--base-dir $OMNIBUS_BASE_DIR --skip-deps --target-project ${DD_PROJECT} ${OMNIBUS_EXTRA_ARGS}
- curl -sSL "https://dd-package-tools.s3.amazonaws.com/dd-pkg/${DD_PKG_VERSION}/dd-pkg_Linux_${DD_PKG_ARCH}.tar.gz"
| tar -xz -C /usr/local/bin dd-pkg
- find $OMNIBUS_PACKAGE_DIR -iregex '.*\.\(deb\|rpm\)' | xargs dd-pkg lint
+ - "if [ -n \"$PACKAGE_REQUIRED_FILES_LIST\" ]; then\n find $OMNIBUS_PACKAGE_DIR\
+ \ \\( -name '*.deb' -or -name '*.rpm' \\) -a -not -name '*-dbg[_-]*' | xargs dd-pkg\
+ \ check-files --required-files ${PACKAGE_REQUIRED_FILES_LIST}\nfi\n"
- $S3_CP_CMD $OMNIBUS_PACKAGE_DIR/datadog-${DD_PROJECT}_*_${PACKAGE_ARCH}.deb $S3_ARTIFACTS_URI/$DESTINATION_DEB
stage: packaging
tags:
- arch:amd64
variables:
AGENT_MAJOR_VERSION: 7
DD_PKG_ARCH: x86_64
DD_PROJECT: installer
DESTINATION_DEB: datadog-installer_7_amd64.deb
KUBERNETES_CPU_REQUEST: 16
KUBERNETES_MEMORY_LIMIT: 32Gi
KUBERNETES_MEMORY_REQUEST: 32Gi
OMNIBUS_PACKAGE_ARTIFACT_DIR: $OMNIBUS_PACKAGE_DIR
PACKAGE_ARCH: amd64
+ PACKAGE_REQUIRED_FILES_LIST: ''
RELEASE_VERSION: $RELEASE_VERSION_7
installer_deb-arm64
installer_deb-arm64:
artifacts:
expire_in: 2 weeks
paths:
- $OMNIBUS_PACKAGE_DIR
cache:
- key:
files:
- omnibus/Gemfile
- release.json
prefix: omnibus-deps-$CI_JOB_NAME-$OMNIBUS_RUBY_VERSION-$OMNIBUS_SOFTWARE
paths:
- omnibus/vendor/bundle
image: 486234852809.dkr.ecr.us-east-1.amazonaws.com/ci/datadog-agent-buildimages/deb_arm64$DATADOG_AGENT_ARMBUILDIMAGES_SUFFIX:$DATADOG_AGENT_ARMBUILDIMAGES
needs:
- installer-arm64
rules:
- if: $CI_COMMIT_BRANCH =~ /^mq-working-branch-/
when: never
- when: on_success
script:
- pushd omnibus && bundle config set --local path 'vendor/bundle' && popd
- echo "About to package for $RELEASE_VERSION"
- set +x
- printf -- "$($CI_PROJECT_DIR/tools/ci/fetch_secret.sh $DEB_GPG_KEY)" | gpg --import
--batch
- EXIT="${PIPESTATUS[0]}"; if [ $EXIT -ne 0 ]; then echo "Unable to locate credentials
needs gitlab runner restart"; exit $EXIT; fi
- DEB_SIGNING_PASSPHRASE=$($CI_PROJECT_DIR/tools/ci/fetch_secret.sh $DEB_SIGNING_PASSPHRASE)
|| exit $?; export DEB_SIGNING_PASSPHRASE
- inv -e omnibus.build --release-version "$RELEASE_VERSION" --major-version "$AGENT_MAJOR_VERSION"
--base-dir $OMNIBUS_BASE_DIR --skip-deps --target-project ${DD_PROJECT} ${OMNIBUS_EXTRA_ARGS}
- curl -sSL "https://dd-package-tools.s3.amazonaws.com/dd-pkg/${DD_PKG_VERSION}/dd-pkg_Linux_${DD_PKG_ARCH}.tar.gz"
| tar -xz -C /usr/local/bin dd-pkg
- find $OMNIBUS_PACKAGE_DIR -iregex '.*\.\(deb\|rpm\)' | xargs dd-pkg lint
+ - "if [ -n \"$PACKAGE_REQUIRED_FILES_LIST\" ]; then\n find $OMNIBUS_PACKAGE_DIR\
+ \ \\( -name '*.deb' -or -name '*.rpm' \\) -a -not -name '*-dbg[_-]*' | xargs dd-pkg\
+ \ check-files --required-files ${PACKAGE_REQUIRED_FILES_LIST}\nfi\n"
- $S3_CP_CMD $OMNIBUS_PACKAGE_DIR/datadog-${DD_PROJECT}_*_${PACKAGE_ARCH}.deb $S3_ARTIFACTS_URI/$DESTINATION_DEB
stage: packaging
tags:
- arch:arm64
variables:
AGENT_MAJOR_VERSION: 7
DD_PKG_ARCH: arm64
DD_PROJECT: installer
DESTINATION_DEB: datadog-installer_7_arm64.deb
KUBERNETES_CPU_REQUEST: 16
KUBERNETES_MEMORY_LIMIT: 32Gi
KUBERNETES_MEMORY_REQUEST: 32Gi
OMNIBUS_PACKAGE_ARTIFACT_DIR: $OMNIBUS_PACKAGE_DIR
PACKAGE_ARCH: arm64
+ PACKAGE_REQUIRED_FILES_LIST: ''
RELEASE_VERSION: $RELEASE_VERSION_7
installer_rpm-amd64
installer_rpm-amd64:
artifacts:
expire_in: 2 weeks
paths:
- $OMNIBUS_PACKAGE_DIR
before_script: null
cache:
- key:
files:
- omnibus/Gemfile
- release.json
prefix: omnibus-deps-$CI_JOB_NAME-$OMNIBUS_RUBY_VERSION-$OMNIBUS_SOFTWARE
paths:
- omnibus/vendor/bundle
image: 486234852809.dkr.ecr.us-east-1.amazonaws.com/ci/datadog-agent-buildimages/rpm_x64$DATADOG_AGENT_BUILDIMAGES_SUFFIX:$DATADOG_AGENT_BUILDIMAGES
needs:
- installer-amd64
rules:
- if: $CI_COMMIT_BRANCH =~ /^mq-working-branch-/
when: never
- when: on_success
script:
- echo "About to build for $RELEASE_VERSION"
- pushd omnibus && bundle config set --local path 'vendor/bundle' && popd
- printf -- "$($CI_PROJECT_DIR/tools/ci/fetch_secret.sh $RPM_GPG_KEY)" | gpg --import
--batch
- EXIT="${PIPESTATUS[0]}"; if [ $EXIT -ne 0 ]; then echo "Unable to locate credentials
needs gitlab runner restart"; exit $EXIT; fi
- RPM_SIGNING_PASSPHRASE=$($CI_PROJECT_DIR/tools/ci/fetch_secret.sh $RPM_SIGNING_PASSPHRASE)
|| exit $?; export RPM_SIGNING_PASSPHRASE
- inv -e omnibus.build --release-version "$RELEASE_VERSION" --major-version "$AGENT_MAJOR_VERSION"
--base-dir $OMNIBUS_BASE_DIR --skip-deps --target-project=${DD_PROJECT} ${OMNIBUS_EXTRA_ARGS}
- ls -la $OMNIBUS_PACKAGE_DIR/
- curl -sSL "https://dd-package-tools.s3.amazonaws.com/dd-pkg/${DD_PKG_VERSION}/dd-pkg_Linux_${DD_PKG_ARCH}.tar.gz"
| tar -xz -C /usr/local/bin dd-pkg
- find $OMNIBUS_PACKAGE_DIR -iregex '.*\.\(deb\|rpm\)' | xargs dd-pkg lint
+ - "if [ -n \"$PACKAGE_REQUIRED_FILES_LIST\" ]; then\n find $OMNIBUS_PACKAGE_DIR\
+ \ \\( -name '*.deb' -or -name '*.rpm' \\) -a -not -name '*-dbg[_-]*' | xargs dd-pkg\
+ \ check-files --required-files ${PACKAGE_REQUIRED_FILES_LIST}\nfi\n"
stage: packaging
tags:
- arch:amd64
variables:
AGENT_MAJOR_VERSION: 7
DD_PKG_ARCH: x86_64
DD_PROJECT: installer
KUBERNETES_CPU_REQUEST: 16
KUBERNETES_MEMORY_LIMIT: 32Gi
KUBERNETES_MEMORY_REQUEST: 32Gi
OMNIBUS_PACKAGE_ARTIFACT_DIR: $OMNIBUS_PACKAGE_DIR
PACKAGE_ARCH: amd64
+ PACKAGE_REQUIRED_FILES_LIST: ''
RELEASE_VERSION: $RELEASE_VERSION_7
installer_rpm-arm64
installer_rpm-arm64:
artifacts:
expire_in: 2 weeks
paths:
- $OMNIBUS_PACKAGE_DIR
before_script: null
cache:
- key:
files:
- omnibus/Gemfile
- release.json
prefix: omnibus-deps-$CI_JOB_NAME-$OMNIBUS_RUBY_VERSION-$OMNIBUS_SOFTWARE
paths:
- omnibus/vendor/bundle
image: 486234852809.dkr.ecr.us-east-1.amazonaws.com/ci/datadog-agent-buildimages/rpm_arm64$DATADOG_AGENT_ARMBUILDIMAGES_SUFFIX:$DATADOG_AGENT_ARMBUILDIMAGES
needs:
- installer-arm64
rules:
- if: $CI_COMMIT_BRANCH =~ /^mq-working-branch-/
when: never
- when: on_success
script:
- echo "About to build for $RELEASE_VERSION"
- pushd omnibus && bundle config set --local path 'vendor/bundle' && popd
- printf -- "$($CI_PROJECT_DIR/tools/ci/fetch_secret.sh $RPM_GPG_KEY)" | gpg --import
--batch
- EXIT="${PIPESTATUS[0]}"; if [ $EXIT -ne 0 ]; then echo "Unable to locate credentials
needs gitlab runner restart"; exit $EXIT; fi
- RPM_SIGNING_PASSPHRASE=$($CI_PROJECT_DIR/tools/ci/fetch_secret.sh $RPM_SIGNING_PASSPHRASE)
|| exit $?; export RPM_SIGNING_PASSPHRASE
- inv -e omnibus.build --release-version "$RELEASE_VERSION" --major-version "$AGENT_MAJOR_VERSION"
--base-dir $OMNIBUS_BASE_DIR --skip-deps --target-project=${DD_PROJECT} ${OMNIBUS_EXTRA_ARGS}
- ls -la $OMNIBUS_PACKAGE_DIR/
- curl -sSL "https://dd-package-tools.s3.amazonaws.com/dd-pkg/${DD_PKG_VERSION}/dd-pkg_Linux_${DD_PKG_ARCH}.tar.gz"
| tar -xz -C /usr/local/bin dd-pkg
- find $OMNIBUS_PACKAGE_DIR -iregex '.*\.\(deb\|rpm\)' | xargs dd-pkg lint
+ - "if [ -n \"$PACKAGE_REQUIRED_FILES_LIST\" ]; then\n find $OMNIBUS_PACKAGE_DIR\
+ \ \\( -name '*.deb' -or -name '*.rpm' \\) -a -not -name '*-dbg[_-]*' | xargs dd-pkg\
+ \ check-files --required-files ${PACKAGE_REQUIRED_FILES_LIST}\nfi\n"
stage: packaging
tags:
- arch:arm64
variables:
AGENT_MAJOR_VERSION: 7
DD_PKG_ARCH: arm64
DD_PROJECT: installer
KUBERNETES_CPU_REQUEST: 16
KUBERNETES_MEMORY_LIMIT: 32Gi
KUBERNETES_MEMORY_REQUEST: 32Gi
OMNIBUS_PACKAGE_ARTIFACT_DIR: $OMNIBUS_PACKAGE_DIR
PACKAGE_ARCH: arm64
+ PACKAGE_REQUIRED_FILES_LIST: ''
RELEASE_VERSION: $RELEASE_VERSION_7
installer_suse_rpm-amd64
installer_suse_rpm-amd64:
artifacts:
expire_in: 2 weeks
paths:
- $OMNIBUS_PACKAGE_DIR_SUSE
before_script: null
cache:
- key:
files:
- omnibus/Gemfile
- release.json
prefix: omnibus-deps-$CI_JOB_NAME-$OMNIBUS_RUBY_VERSION-$OMNIBUS_SOFTWARE
paths:
- omnibus/vendor/bundle
image: 486234852809.dkr.ecr.us-east-1.amazonaws.com/ci/datadog-agent-buildimages/rpm_x64$DATADOG_AGENT_BUILDIMAGES_SUFFIX:$DATADOG_AGENT_BUILDIMAGES
needs:
- installer-amd64
rules:
- if: $CI_COMMIT_BRANCH =~ /^mq-working-branch-/
when: never
- when: on_success
script:
- echo "About to build for $RELEASE_VERSION"
- pushd omnibus && bundle config set --local path 'vendor/bundle' && popd
- printf -- "$($CI_PROJECT_DIR/tools/ci/fetch_secret.sh $RPM_GPG_KEY)" | gpg --import
--batch
- EXIT="${PIPESTATUS[0]}"; if [ $EXIT -ne 0 ]; then echo "Unable to locate credentials
needs gitlab runner restart"; exit $EXIT; fi
- RPM_SIGNING_PASSPHRASE=$($CI_PROJECT_DIR/tools/ci/fetch_secret.sh $RPM_SIGNING_PASSPHRASE)
|| exit $?; export RPM_SIGNING_PASSPHRASE
- inv -e omnibus.build --release-version "$RELEASE_VERSION" --major-version "$AGENT_MAJOR_VERSION"
--base-dir $OMNIBUS_BASE_DIR --skip-deps --target-project=${DD_PROJECT} ${OMNIBUS_EXTRA_ARGS}
- ls -la $OMNIBUS_PACKAGE_DIR/
- curl -sSL "https://dd-package-tools.s3.amazonaws.com/dd-pkg/${DD_PKG_VERSION}/dd-pkg_Linux_${DD_PKG_ARCH}.tar.gz"
| tar -xz -C /usr/local/bin dd-pkg
- find $OMNIBUS_PACKAGE_DIR -iregex '.*\.\(deb\|rpm\)' | xargs dd-pkg lint
+ - "if [ -n \"$PACKAGE_REQUIRED_FILES_LIST\" ]; then\n find $OMNIBUS_PACKAGE_DIR\
+ \ \\( -name '*.deb' -or -name '*.rpm' \\) -a -not -name '*-dbg[_-]*' | xargs dd-pkg\
+ \ check-files --required-files ${PACKAGE_REQUIRED_FILES_LIST}\nfi\n"
- mkdir -p $OMNIBUS_PACKAGE_DIR_SUSE && mv $OMNIBUS_PACKAGE_DIR/*.rpm $OMNIBUS_PACKAGE_DIR_SUSE/
stage: packaging
tags:
- arch:amd64
variables:
AGENT_MAJOR_VERSION: 7
DD_PKG_ARCH: x86_64
DD_PROJECT: installer
KUBERNETES_CPU_REQUEST: 16
KUBERNETES_MEMORY_LIMIT: 32Gi
KUBERNETES_MEMORY_REQUEST: 32Gi
OMNIBUS_EXTRA_ARGS: --host-distribution=suse
OMNIBUS_PACKAGE_ARTIFACT_DIR: $OMNIBUS_PACKAGE_DIR
PACKAGE_ARCH: amd64
+ PACKAGE_REQUIRED_FILES_LIST: ''
RELEASE_VERSION: $RELEASE_VERSION_7
installer_suse_rpm-arm64
installer_suse_rpm-arm64:
artifacts:
expire_in: 2 weeks
paths:
- $OMNIBUS_PACKAGE_DIR_SUSE
before_script: null
cache:
- key:
files:
- omnibus/Gemfile
- release.json
prefix: omnibus-deps-$CI_JOB_NAME-$OMNIBUS_RUBY_VERSION-$OMNIBUS_SOFTWARE
paths:
- omnibus/vendor/bundle
image: 486234852809.dkr.ecr.us-east-1.amazonaws.com/ci/datadog-agent-buildimages/rpm_arm64$DATADOG_AGENT_ARMBUILDIMAGES_SUFFIX:$DATADOG_AGENT_ARMBUILDIMAGES
needs:
- installer-arm64
rules:
- if: $CI_COMMIT_BRANCH =~ /^mq-working-branch-/
when: never
- when: on_success
script:
- echo "About to build for $RELEASE_VERSION"
- pushd omnibus && bundle config set --local path 'vendor/bundle' && popd
- printf -- "$($CI_PROJECT_DIR/tools/ci/fetch_secret.sh $RPM_GPG_KEY)" | gpg --import
--batch
- EXIT="${PIPESTATUS[0]}"; if [ $EXIT -ne 0 ]; then echo "Unable to locate credentials
needs gitlab runner restart"; exit $EXIT; fi
- RPM_SIGNING_PASSPHRASE=$($CI_PROJECT_DIR/tools/ci/fetch_secret.sh $RPM_SIGNING_PASSPHRASE)
|| exit $?; export RPM_SIGNING_PASSPHRASE
- inv -e omnibus.build --release-version "$RELEASE_VERSION" --major-version "$AGENT_MAJOR_VERSION"
--base-dir $OMNIBUS_BASE_DIR --skip-deps --target-project=${DD_PROJECT} ${OMNIBUS_EXTRA_ARGS}
- ls -la $OMNIBUS_PACKAGE_DIR/
- curl -sSL "https://dd-package-tools.s3.amazonaws.com/dd-pkg/${DD_PKG_VERSION}/dd-pkg_Linux_${DD_PKG_ARCH}.tar.gz"
| tar -xz -C /usr/local/bin dd-pkg
- find $OMNIBUS_PACKAGE_DIR -iregex '.*\.\(deb\|rpm\)' | xargs dd-pkg lint
+ - "if [ -n \"$PACKAGE_REQUIRED_FILES_LIST\" ]; then\n find $OMNIBUS_PACKAGE_DIR\
+ \ \\( -name '*.deb' -or -name '*.rpm' \\) -a -not -name '*-dbg[_-]*' | xargs dd-pkg\
+ \ check-files --required-files ${PACKAGE_REQUIRED_FILES_LIST}\nfi\n"
- mkdir -p $OMNIBUS_PACKAGE_DIR_SUSE && mv $OMNIBUS_PACKAGE_DIR/*.rpm $OMNIBUS_PACKAGE_DIR_SUSE/
stage: packaging
tags:
- arch:arm64
variables:
AGENT_MAJOR_VERSION: 7
DD_PKG_ARCH: arm64
DD_PROJECT: installer
KUBERNETES_CPU_REQUEST: 16
KUBERNETES_MEMORY_LIMIT: 32Gi
KUBERNETES_MEMORY_REQUEST: 32Gi
OMNIBUS_EXTRA_ARGS: --host-distribution=suse
OMNIBUS_PACKAGE_ARTIFACT_DIR: $OMNIBUS_PACKAGE_DIR
PACKAGE_ARCH: arm64
+ PACKAGE_REQUIRED_FILES_LIST: ''
RELEASE_VERSION: $RELEASE_VERSION_7
iot_agent_deb-arm64
iot_agent_deb-arm64:
artifacts:
expire_in: 2 weeks
paths:
- $OMNIBUS_PACKAGE_DIR
cache:
- key:
files:
- omnibus/Gemfile
- release.json
prefix: omnibus-deps-$CI_JOB_NAME-$OMNIBUS_RUBY_VERSION-$OMNIBUS_SOFTWARE
paths:
- omnibus/vendor/bundle
image: 486234852809.dkr.ecr.us-east-1.amazonaws.com/ci/datadog-agent-buildimages/deb_arm64$DATADOG_AGENT_ARMBUILDIMAGES_SUFFIX:$DATADOG_AGENT_ARMBUILDIMAGES
needs:
- iot-agent-arm64
rules:
- if: $CI_COMMIT_BRANCH =~ /^mq-working-branch-/
when: never
- when: on_success
script:
- pushd omnibus && bundle config set --local path 'vendor/bundle' && popd
- echo "About to package for $RELEASE_VERSION"
- set +x
- printf -- "$($CI_PROJECT_DIR/tools/ci/fetch_secret.sh $DEB_GPG_KEY)" | gpg --import
--batch
- EXIT="${PIPESTATUS[0]}"; if [ $EXIT -ne 0 ]; then echo "Unable to locate credentials
needs gitlab runner restart"; exit $EXIT; fi
- DEB_SIGNING_PASSPHRASE=$($CI_PROJECT_DIR/tools/ci/fetch_secret.sh $DEB_SIGNING_PASSPHRASE)
|| exit $?; export DEB_SIGNING_PASSPHRASE
- inv -e omnibus.build --release-version "$RELEASE_VERSION" --base-dir $OMNIBUS_BASE_DIR
--skip-deps --flavor iot
- curl -sSL "https://dd-package-tools.s3.amazonaws.com/dd-pkg/${DD_PKG_VERSION}/dd-pkg_Linux_${DD_PKG_ARCH}.tar.gz"
| tar -xz -C /usr/local/bin dd-pkg
- find $OMNIBUS_PACKAGE_DIR -iregex '.*\.\(deb\|rpm\)' | xargs dd-pkg lint
+ - "if [ -n \"$PACKAGE_REQUIRED_FILES_LIST\" ]; then\n find $OMNIBUS_PACKAGE_DIR\
+ \ \\( -name '*.deb' -or -name '*.rpm' \\) -a -not -name '*-dbg[_-]*' | xargs dd-pkg\
+ \ check-files --required-files ${PACKAGE_REQUIRED_FILES_LIST}\nfi\n"
- $S3_CP_CMD $OMNIBUS_PACKAGE_DIR/datadog-*_${PACKAGE_ARCH}.deb $S3_ARTIFACTS_URI/$DESTINATION_DEB
stage: packaging
tags:
- arch:arm64
variables:
AGENT_MAJOR_VERSION: 7
DD_PKG_ARCH: arm64
DESTINATION_DEB: datadog-iot-agent_7_arm64.deb
KUBERNETES_CPU_REQUEST: 16
KUBERNETES_MEMORY_LIMIT: 32Gi
KUBERNETES_MEMORY_REQUEST: 32Gi
OMNIBUS_PACKAGE_ARTIFACT_DIR: $OMNIBUS_PACKAGE_DIR
PACKAGE_ARCH: arm64
+ PACKAGE_REQUIRED_FILES_LIST: test/required_files/iot-agent-deb.txt
RELEASE_VERSION: $RELEASE_VERSION_7
iot_agent_deb-armhf
iot_agent_deb-armhf:
artifacts:
expire_in: 2 weeks
paths:
- $OMNIBUS_PACKAGE_DIR
cache:
- key:
files:
- omnibus/Gemfile
- release.json
prefix: omnibus-deps-$CI_JOB_NAME-$OMNIBUS_RUBY_VERSION-$OMNIBUS_SOFTWARE
paths:
- omnibus/vendor/bundle
image: 486234852809.dkr.ecr.us-east-1.amazonaws.com/ci/datadog-agent-buildimages/deb_armhf$DATADOG_AGENT_ARMBUILDIMAGES_SUFFIX:$DATADOG_AGENT_ARMBUILDIMAGES
needs:
- iot-agent-armhf
rules:
- if: $CI_COMMIT_BRANCH =~ /^mq-working-branch-/
when: never
- when: on_success
script:
- pushd omnibus && bundle config set --local path 'vendor/bundle' && popd
- echo "About to package for $RELEASE_VERSION"
- set +x
- printf -- "$($CI_PROJECT_DIR/tools/ci/fetch_secret.sh $DEB_GPG_KEY)" | gpg --import
--batch
- EXIT="${PIPESTATUS[0]}"; if [ $EXIT -ne 0 ]; then echo "Unable to locate credentials
needs gitlab runner restart"; exit $EXIT; fi
- DEB_SIGNING_PASSPHRASE=$($CI_PROJECT_DIR/tools/ci/fetch_secret.sh $DEB_SIGNING_PASSPHRASE)
|| exit $?; export DEB_SIGNING_PASSPHRASE
- inv -e omnibus.build --release-version "$RELEASE_VERSION" --base-dir $OMNIBUS_BASE_DIR
--skip-deps --flavor iot
- curl -sSL "https://dd-package-tools.s3.amazonaws.com/dd-pkg/${DD_PKG_VERSION}/dd-pkg_Linux_${DD_PKG_ARCH}.tar.gz"
| tar -xz -C /usr/local/bin dd-pkg
- find $OMNIBUS_PACKAGE_DIR -iregex '.*\.\(deb\|rpm\)' | xargs dd-pkg lint
+ - "if [ -n \"$PACKAGE_REQUIRED_FILES_LIST\" ]; then\n find $OMNIBUS_PACKAGE_DIR\
+ \ \\( -name '*.deb' -or -name '*.rpm' \\) -a -not -name '*-dbg[_-]*' | xargs dd-pkg\
+ \ check-files --required-files ${PACKAGE_REQUIRED_FILES_LIST}\nfi\n"
- $S3_CP_CMD $OMNIBUS_PACKAGE_DIR/datadog-*_${PACKAGE_ARCH}.deb $S3_ARTIFACTS_URI/$DESTINATION_DEB
stage: packaging
tags:
- arch:arm64
variables:
AGENT_MAJOR_VERSION: 7
DD_PKG_ARCH: arm64
DESTINATION_DEB: datadog-iot-agent_7_armhf.deb
FORCED_PACKAGE_COMPRESSION_LEVEL: 5
KUBERNETES_CPU_REQUEST: 16
KUBERNETES_MEMORY_LIMIT: 32Gi
KUBERNETES_MEMORY_REQUEST: 32Gi
OMNIBUS_PACKAGE_ARTIFACT_DIR: $OMNIBUS_PACKAGE_DIR
PACKAGE_ARCH: armhf
+ PACKAGE_REQUIRED_FILES_LIST: test/required_files/iot-agent-deb.txt
RELEASE_VERSION: $RELEASE_VERSION_7
iot_agent_deb-x64
iot_agent_deb-x64:
artifacts:
expire_in: 2 weeks
paths:
- $OMNIBUS_PACKAGE_DIR
cache:
- key:
files:
- omnibus/Gemfile
- release.json
prefix: omnibus-deps-$CI_JOB_NAME-$OMNIBUS_RUBY_VERSION-$OMNIBUS_SOFTWARE
paths:
- omnibus/vendor/bundle
image: 486234852809.dkr.ecr.us-east-1.amazonaws.com/ci/datadog-agent-buildimages/deb_x64$DATADOG_AGENT_BUILDIMAGES_SUFFIX:$DATADOG_AGENT_BUILDIMAGES
needs:
- iot-agent-x64
rules:
- if: $CI_COMMIT_BRANCH =~ /^mq-working-branch-/
when: never
- when: on_success
script:
- pushd omnibus && bundle config set --local path 'vendor/bundle' && popd
- echo "About to package for $RELEASE_VERSION"
- set +x
- printf -- "$($CI_PROJECT_DIR/tools/ci/fetch_secret.sh $DEB_GPG_KEY)" | gpg --import
--batch
- EXIT="${PIPESTATUS[0]}"; if [ $EXIT -ne 0 ]; then echo "Unable to locate credentials
needs gitlab runner restart"; exit $EXIT; fi
- DEB_SIGNING_PASSPHRASE=$($CI_PROJECT_DIR/tools/ci/fetch_secret.sh $DEB_SIGNING_PASSPHRASE)
|| exit $?; export DEB_SIGNING_PASSPHRASE
- inv -e omnibus.build --release-version "$RELEASE_VERSION" --base-dir $OMNIBUS_BASE_DIR
--skip-deps --flavor iot
- curl -sSL "https://dd-package-tools.s3.amazonaws.com/dd-pkg/${DD_PKG_VERSION}/dd-pkg_Linux_${DD_PKG_ARCH}.tar.gz"
| tar -xz -C /usr/local/bin dd-pkg
- find $OMNIBUS_PACKAGE_DIR -iregex '.*\.\(deb\|rpm\)' | xargs dd-pkg lint
+ - "if [ -n \"$PACKAGE_REQUIRED_FILES_LIST\" ]; then\n find $OMNIBUS_PACKAGE_DIR\
+ \ \\( -name '*.deb' -or -name '*.rpm' \\) -a -not -name '*-dbg[_-]*' | xargs dd-pkg\
+ \ check-files --required-files ${PACKAGE_REQUIRED_FILES_LIST}\nfi\n"
- $S3_CP_CMD $OMNIBUS_PACKAGE_DIR/datadog-*_${PACKAGE_ARCH}.deb $S3_ARTIFACTS_URI/$DESTINATION_DEB
stage: packaging
tags:
- arch:amd64
variables:
AGENT_MAJOR_VERSION: 7
DD_PKG_ARCH: x86_64
DESTINATION_DEB: datadog-iot-agent_7_amd64.deb
KUBERNETES_CPU_REQUEST: 16
KUBERNETES_MEMORY_LIMIT: 32Gi
KUBERNETES_MEMORY_REQUEST: 32Gi
OMNIBUS_PACKAGE_ARTIFACT_DIR: $OMNIBUS_PACKAGE_DIR
PACKAGE_ARCH: amd64
+ PACKAGE_REQUIRED_FILES_LIST: test/required_files/iot-agent-deb.txt
RELEASE_VERSION: $RELEASE_VERSION_7
iot_agent_rpm-arm64
iot_agent_rpm-arm64:
artifacts:
expire_in: 2 weeks
paths:
- $OMNIBUS_PACKAGE_DIR
cache:
- key:
files:
- omnibus/Gemfile
- release.json
prefix: omnibus-deps-$CI_JOB_NAME-$OMNIBUS_RUBY_VERSION-$OMNIBUS_SOFTWARE
paths:
- omnibus/vendor/bundle
image: 486234852809.dkr.ecr.us-east-1.amazonaws.com/ci/datadog-agent-buildimages/rpm_arm64$DATADOG_AGENT_ARMBUILDIMAGES_SUFFIX:$DATADOG_AGENT_ARMBUILDIMAGES
needs:
- iot-agent-arm64
rules:
- if: $CI_COMMIT_BRANCH =~ /^mq-working-branch-/
when: never
- when: on_success
script:
- echo "About to build for $RELEASE_VERSION"
- pushd omnibus && bundle config set --local path 'vendor/bundle' && popd
- printf -- "$($CI_PROJECT_DIR/tools/ci/fetch_secret.sh $RPM_GPG_KEY)" | gpg --import
--batch
- EXIT="${PIPESTATUS[0]}"; if [ $EXIT -ne 0 ]; then echo "Unable to locate credentials
needs gitlab runner restart"; exit $EXIT; fi
- RPM_SIGNING_PASSPHRASE=$($CI_PROJECT_DIR/tools/ci/fetch_secret.sh $RPM_SIGNING_PASSPHRASE)
|| exit $?; export RPM_SIGNING_PASSPHRASE
- inv -e omnibus.build --release-version "$RELEASE_VERSION" --base-dir $OMNIBUS_BASE_DIR
--skip-deps --flavor=iot ${OMNIBUS_EXTRA_ARGS}
- ls -la $OMNIBUS_PACKAGE_DIR/
- curl -sSL "https://dd-package-tools.s3.amazonaws.com/dd-pkg/${DD_PKG_VERSION}/dd-pkg_Linux_${DD_PKG_ARCH}.tar.gz"
| tar -xz -C /usr/local/bin dd-pkg
- find $OMNIBUS_PACKAGE_DIR -iregex '.*\.\(deb\|rpm\)' | xargs dd-pkg lint
+ - "if [ -n \"$PACKAGE_REQUIRED_FILES_LIST\" ]; then\n find $OMNIBUS_PACKAGE_DIR\
+ \ \\( -name '*.deb' -or -name '*.rpm' \\) -a -not -name '*-dbg[_-]*' | xargs dd-pkg\
+ \ check-files --required-files ${PACKAGE_REQUIRED_FILES_LIST}\nfi\n"
stage: packaging
tags:
- arch:arm64
variables:
DD_PKG_ARCH: arm64
KUBERNETES_CPU_REQUEST: 16
KUBERNETES_MEMORY_LIMIT: 32Gi
KUBERNETES_MEMORY_REQUEST: 32Gi
OMNIBUS_PACKAGE_ARTIFACT_DIR: $OMNIBUS_PACKAGE_DIR
PACKAGE_ARCH: arm64
+ PACKAGE_REQUIRED_FILES_LIST: test/required_files/iot-agent-rpm.txt
RELEASE_VERSION: $RELEASE_VERSION_7
iot_agent_rpm-armhf
iot_agent_rpm-armhf:
artifacts:
expire_in: 2 weeks
paths:
- $OMNIBUS_PACKAGE_DIR
before_script:
- export LD_PRELOAD="/usr/local/lib/libfakearmv7l.so"
cache:
- key:
files:
- omnibus/Gemfile
- release.json
prefix: omnibus-deps-$CI_JOB_NAME-$OMNIBUS_RUBY_VERSION-$OMNIBUS_SOFTWARE
paths:
- omnibus/vendor/bundle
image: 486234852809.dkr.ecr.us-east-1.amazonaws.com/ci/datadog-agent-buildimages/rpm_armhf$DATADOG_AGENT_ARMBUILDIMAGES_SUFFIX:$DATADOG_AGENT_ARMBUILDIMAGES
needs:
- iot-agent-armhf
rules:
- if: $CI_COMMIT_BRANCH =~ /^mq-working-branch-/
when: never
- when: on_success
script:
- echo "About to build for $RELEASE_VERSION"
- pushd omnibus && bundle config set --local path 'vendor/bundle' && popd
- printf -- "$($CI_PROJECT_DIR/tools/ci/fetch_secret.sh $RPM_GPG_KEY)" | gpg --import
--batch
- EXIT="${PIPESTATUS[0]}"; if [ $EXIT -ne 0 ]; then echo "Unable to locate credentials
needs gitlab runner restart"; exit $EXIT; fi
- RPM_SIGNING_PASSPHRASE=$($CI_PROJECT_DIR/tools/ci/fetch_secret.sh $RPM_SIGNING_PASSPHRASE)
|| exit $?; export RPM_SIGNING_PASSPHRASE
- inv -e omnibus.build --release-version "$RELEASE_VERSION" --base-dir $OMNIBUS_BASE_DIR
--skip-deps --flavor=iot ${OMNIBUS_EXTRA_ARGS}
- ls -la $OMNIBUS_PACKAGE_DIR/
- curl -sSL "https://dd-package-tools.s3.amazonaws.com/dd-pkg/${DD_PKG_VERSION}/dd-pkg_Linux_${DD_PKG_ARCH}.tar.gz"
| tar -xz -C /usr/local/bin dd-pkg
- find $OMNIBUS_PACKAGE_DIR -iregex '.*\.\(deb\|rpm\)' | xargs dd-pkg lint
+ - "if [ -n \"$PACKAGE_REQUIRED_FILES_LIST\" ]; then\n find $OMNIBUS_PACKAGE_DIR\
+ \ \\( -name '*.deb' -or -name '*.rpm' \\) -a -not -name '*-dbg[_-]*' | xargs dd-pkg\
+ \ check-files --required-files ${PACKAGE_REQUIRED_FILES_LIST}\nfi\n"
stage: packaging
tags:
- arch:arm64
variables:
DD_PKG_ARCH: arm64
KUBERNETES_CPU_REQUEST: 16
KUBERNETES_MEMORY_LIMIT: 32Gi
KUBERNETES_MEMORY_REQUEST: 32Gi
OMNIBUS_PACKAGE_ARTIFACT_DIR: $OMNIBUS_PACKAGE_DIR
PACKAGE_ARCH: armhf
+ PACKAGE_REQUIRED_FILES_LIST: test/required_files/iot-agent-rpm.txt
RELEASE_VERSION: $RELEASE_VERSION_7
iot_agent_rpm-x64
iot_agent_rpm-x64:
artifacts:
expire_in: 2 weeks
paths:
- $OMNIBUS_PACKAGE_DIR
cache:
- key:
files:
- omnibus/Gemfile
- release.json
prefix: omnibus-deps-$CI_JOB_NAME-$OMNIBUS_RUBY_VERSION-$OMNIBUS_SOFTWARE
paths:
- omnibus/vendor/bundle
image: 486234852809.dkr.ecr.us-east-1.amazonaws.com/ci/datadog-agent-buildimages/rpm_x64$DATADOG_AGENT_BUILDIMAGES_SUFFIX:$DATADOG_AGENT_BUILDIMAGES
needs:
- iot-agent-x64
rules:
- if: $CI_COMMIT_BRANCH =~ /^mq-working-branch-/
when: never
- when: on_success
script:
- echo "About to build for $RELEASE_VERSION"
- pushd omnibus && bundle config set --local path 'vendor/bundle' && popd
- printf -- "$($CI_PROJECT_DIR/tools/ci/fetch_secret.sh $RPM_GPG_KEY)" | gpg --import
--batch
- EXIT="${PIPESTATUS[0]}"; if [ $EXIT -ne 0 ]; then echo "Unable to locate credentials
needs gitlab runner restart"; exit $EXIT; fi
- RPM_SIGNING_PASSPHRASE=$($CI_PROJECT_DIR/tools/ci/fetch_secret.sh $RPM_SIGNING_PASSPHRASE)
|| exit $?; export RPM_SIGNING_PASSPHRASE
- inv -e omnibus.build --release-version "$RELEASE_VERSION" --base-dir $OMNIBUS_BASE_DIR
--skip-deps --flavor=iot ${OMNIBUS_EXTRA_ARGS}
- ls -la $OMNIBUS_PACKAGE_DIR/
- curl -sSL "https://dd-package-tools.s3.amazonaws.com/dd-pkg/${DD_PKG_VERSION}/dd-pkg_Linux_${DD_PKG_ARCH}.tar.gz"
| tar -xz -C /usr/local/bin dd-pkg
- find $OMNIBUS_PACKAGE_DIR -iregex '.*\.\(deb\|rpm\)' | xargs dd-pkg lint
+ - "if [ -n \"$PACKAGE_REQUIRED_FILES_LIST\" ]; then\n find $OMNIBUS_PACKAGE_DIR\
+ \ \\( -name '*.deb' -or -name '*.rpm' \\) -a -not -name '*-dbg[_-]*' | xargs dd-pkg\
+ \ check-files --required-files ${PACKAGE_REQUIRED_FILES_LIST}\nfi\n"
stage: packaging
tags:
- arch:amd64
variables:
DD_PKG_ARCH: x86_64
KUBERNETES_CPU_REQUEST: 16
KUBERNETES_MEMORY_LIMIT: 32Gi
KUBERNETES_MEMORY_REQUEST: 32Gi
OMNIBUS_PACKAGE_ARTIFACT_DIR: $OMNIBUS_PACKAGE_DIR
PACKAGE_ARCH: amd64
+ PACKAGE_REQUIRED_FILES_LIST: test/required_files/iot-agent-rpm.txt
RELEASE_VERSION: $RELEASE_VERSION_7
iot_agent_suse-x64
iot_agent_suse-x64:
artifacts:
expire_in: 2 weeks
paths:
- $OMNIBUS_PACKAGE_DIR_SUSE
cache:
- key:
files:
- omnibus/Gemfile
- release.json
prefix: omnibus-deps-$CI_JOB_NAME-$OMNIBUS_RUBY_VERSION-$OMNIBUS_SOFTWARE
paths:
- omnibus/vendor/bundle
image: 486234852809.dkr.ecr.us-east-1.amazonaws.com/ci/datadog-agent-buildimages/rpm_x64$DATADOG_AGENT_BUILDIMAGES_SUFFIX:$DATADOG_AGENT_BUILDIMAGES
needs:
- iot-agent-x64
rules:
- if: $CI_COMMIT_BRANCH =~ /^mq-working-branch-/
when: never
- when: on_success
script:
- echo "About to build for $RELEASE_VERSION"
- pushd omnibus && bundle config set --local path 'vendor/bundle' && popd
- printf -- "$($CI_PROJECT_DIR/tools/ci/fetch_secret.sh $RPM_GPG_KEY)" | gpg --import
--batch
- EXIT="${PIPESTATUS[0]}"; if [ $EXIT -ne 0 ]; then echo "Unable to locate credentials
needs gitlab runner restart"; exit $EXIT; fi
- RPM_SIGNING_PASSPHRASE=$($CI_PROJECT_DIR/tools/ci/fetch_secret.sh $RPM_SIGNING_PASSPHRASE)
|| exit $?; export RPM_SIGNING_PASSPHRASE
- inv -e omnibus.build --release-version "$RELEASE_VERSION" --base-dir $OMNIBUS_BASE_DIR
--skip-deps --flavor=iot ${OMNIBUS_EXTRA_ARGS}
- ls -la $OMNIBUS_PACKAGE_DIR/
- curl -sSL "https://dd-package-tools.s3.amazonaws.com/dd-pkg/${DD_PKG_VERSION}/dd-pkg_Linux_${DD_PKG_ARCH}.tar.gz"
| tar -xz -C /usr/local/bin dd-pkg
- find $OMNIBUS_PACKAGE_DIR -iregex '.*\.\(deb\|rpm\)' | xargs dd-pkg lint
+ - "if [ -n \"$PACKAGE_REQUIRED_FILES_LIST\" ]; then\n find $OMNIBUS_PACKAGE_DIR\
+ \ \\( -name '*.deb' -or -name '*.rpm' \\) -a -not -name '*-dbg[_-]*' | xargs dd-pkg\
+ \ check-files --required-files ${PACKAGE_REQUIRED_FILES_LIST}\nfi\n"
- mkdir -p $OMNIBUS_PACKAGE_DIR_SUSE && mv $OMNIBUS_PACKAGE_DIR/* $OMNIBUS_PACKAGE_DIR_SUSE/
stage: packaging
tags:
- arch:amd64
variables:
DD_PKG_ARCH: x86_64
KUBERNETES_CPU_REQUEST: 16
KUBERNETES_MEMORY_LIMIT: 32Gi
KUBERNETES_MEMORY_REQUEST: 32Gi
OMNIBUS_EXTRA_ARGS: --host-distribution=suse
OMNIBUS_PACKAGE_ARTIFACT_DIR: $OMNIBUS_PACKAGE_DIR
PACKAGE_ARCH: amd64
+ PACKAGE_REQUIRED_FILES_LIST: test/required_files/iot-agent-rpm.txt
RELEASE_VERSION: $RELEASE_VERSION_7
ot_agent_deb-arm64-a7
ot_agent_deb-arm64-a7:
artifacts:
expire_in: 2 weeks
paths:
- $OMNIBUS_PACKAGE_DIR
cache:
- key:
files:
- omnibus/Gemfile
- release.json
prefix: omnibus-deps-$CI_JOB_NAME-$OMNIBUS_RUBY_VERSION-$OMNIBUS_SOFTWARE
paths:
- omnibus/vendor/bundle
image: 486234852809.dkr.ecr.us-east-1.amazonaws.com/ci/datadog-agent-buildimages/deb_arm64$DATADOG_AGENT_ARMBUILDIMAGES_SUFFIX:$DATADOG_AGENT_ARMBUILDIMAGES
needs:
- datadog-ot-agent-7-arm64
rules:
- if: $CI_COMMIT_BRANCH =~ /^mq-working-branch-/
when: never
- when: on_success
script:
- pushd omnibus && bundle config set --local path 'vendor/bundle' && popd
- echo "About to package for $RELEASE_VERSION"
- set +x
- printf -- "$($CI_PROJECT_DIR/tools/ci/fetch_secret.sh $DEB_GPG_KEY)" | gpg --import
--batch
- EXIT="${PIPESTATUS[0]}"; if [ $EXIT -ne 0 ]; then echo "Unable to locate credentials
needs gitlab runner restart"; exit $EXIT; fi
- DEB_SIGNING_PASSPHRASE=$($CI_PROJECT_DIR/tools/ci/fetch_secret.sh $DEB_SIGNING_PASSPHRASE)
|| exit $?; export DEB_SIGNING_PASSPHRASE
- inv -e omnibus.build --release-version "$RELEASE_VERSION" --major-version "$AGENT_MAJOR_VERSION"
--base-dir $OMNIBUS_BASE_DIR --skip-deps --target-project ${DD_PROJECT} --flavor
ot ${OMNIBUS_EXTRA_ARGS}
- curl -sSL "https://dd-package-tools.s3.amazonaws.com/dd-pkg/${DD_PKG_VERSION}/dd-pkg_Linux_${DD_PKG_ARCH}.tar.gz"
| tar -xz -C /usr/local/bin dd-pkg
- find $OMNIBUS_PACKAGE_DIR -iregex '.*\.\(deb\|rpm\)' | xargs dd-pkg lint
+ - "if [ -n \"$PACKAGE_REQUIRED_FILES_LIST\" ]; then\n find $OMNIBUS_PACKAGE_DIR\
+ \ \\( -name '*.deb' -or -name '*.rpm' \\) -a -not -name '*-dbg[_-]*' | xargs dd-pkg\
+ \ check-files --required-files ${PACKAGE_REQUIRED_FILES_LIST}\nfi\n"
- $S3_CP_CMD $OMNIBUS_PACKAGE_DIR/datadog-ot-agent_*_${PACKAGE_ARCH}.deb $S3_ARTIFACTS_URI/$DESTINATION_DEB
stage: packaging
tags:
- arch:arm64
variables:
AGENT_MAJOR_VERSION: 7
DD_PKG_ARCH: arm64
DD_PROJECT: agent
DESTINATION_DEB: datadog-ot-agent_7_arm64.deb
KUBERNETES_CPU_REQUEST: 16
KUBERNETES_MEMORY_LIMIT: 32Gi
KUBERNETES_MEMORY_REQUEST: 32Gi
OMNIBUS_PACKAGE_ARTIFACT_DIR: $OMNIBUS_PACKAGE_DIR
PACKAGE_ARCH: arm64
+ PACKAGE_REQUIRED_FILES_LIST: test/required_files/agent-deb.txt
RELEASE_VERSION: $RELEASE_VERSION_7
ot_agent_deb-x64-a7
ot_agent_deb-x64-a7:
artifacts:
expire_in: 2 weeks
paths:
- $OMNIBUS_PACKAGE_DIR
cache:
- key:
files:
- omnibus/Gemfile
- release.json
prefix: omnibus-deps-$CI_JOB_NAME-$OMNIBUS_RUBY_VERSION-$OMNIBUS_SOFTWARE
paths:
- omnibus/vendor/bundle
image: 486234852809.dkr.ecr.us-east-1.amazonaws.com/ci/datadog-agent-buildimages/deb_x64$DATADOG_AGENT_BUILDIMAGES_SUFFIX:$DATADOG_AGENT_BUILDIMAGES
needs:
- datadog-ot-agent-7-x64
rules:
- if: $CI_COMMIT_BRANCH =~ /^mq-working-branch-/
when: never
- when: on_success
script:
- pushd omnibus && bundle config set --local path 'vendor/bundle' && popd
- echo "About to package for $RELEASE_VERSION"
- set +x
- printf -- "$($CI_PROJECT_DIR/tools/ci/fetch_secret.sh $DEB_GPG_KEY)" | gpg --import
--batch
- EXIT="${PIPESTATUS[0]}"; if [ $EXIT -ne 0 ]; then echo "Unable to locate credentials
needs gitlab runner restart"; exit $EXIT; fi
- DEB_SIGNING_PASSPHRASE=$($CI_PROJECT_DIR/tools/ci/fetch_secret.sh $DEB_SIGNING_PASSPHRASE)
|| exit $?; export DEB_SIGNING_PASSPHRASE
- inv -e omnibus.build --release-version "$RELEASE_VERSION" --major-version "$AGENT_MAJOR_VERSION"
--base-dir $OMNIBUS_BASE_DIR --skip-deps --target-project ${DD_PROJECT} --flavor
ot ${OMNIBUS_EXTRA_ARGS}
- curl -sSL "https://dd-package-tools.s3.amazonaws.com/dd-pkg/${DD_PKG_VERSION}/dd-pkg_Linux_${DD_PKG_ARCH}.tar.gz"
| tar -xz -C /usr/local/bin dd-pkg
- find $OMNIBUS_PACKAGE_DIR -iregex '.*\.\(deb\|rpm\)' | xargs dd-pkg lint
+ - "if [ -n \"$PACKAGE_REQUIRED_FILES_LIST\" ]; then\n find $OMNIBUS_PACKAGE_DIR\
+ \ \\( -name '*.deb' -or -name '*.rpm' \\) -a -not -name '*-dbg[_-]*' | xargs dd-pkg\
+ \ check-files --required-files ${PACKAGE_REQUIRED_FILES_LIST}\nfi\n"
- $S3_CP_CMD $OMNIBUS_PACKAGE_DIR/datadog-ot-agent_*_${PACKAGE_ARCH}.deb $S3_ARTIFACTS_URI/$DESTINATION_DEB
stage: packaging
tags:
- arch:amd64
variables:
AGENT_MAJOR_VERSION: 7
DD_PKG_ARCH: x86_64
DD_PROJECT: agent
DESTINATION_DEB: datadog-ot-agent_7_amd64.deb
KUBERNETES_CPU_REQUEST: 16
KUBERNETES_MEMORY_LIMIT: 32Gi
KUBERNETES_MEMORY_REQUEST: 32Gi
OMNIBUS_PACKAGE_ARTIFACT_DIR: $OMNIBUS_PACKAGE_DIR
PACKAGE_ARCH: amd64
+ PACKAGE_REQUIRED_FILES_LIST: test/required_files/agent-deb.txt
RELEASE_VERSION: $RELEASE_VERSION_7
Changes Summary
| Removed | Modified | Added | Renamed |
|---|---|---|---|
| 0 | 40 | 0 | 0 |
![pr-commenter[bot] avatar](https://avatars.githubusercontent.com/u/365230?v=4 "Published by a pub.dev verified publisher"){kind=small}
[Fast Unit Tests Report]
On pipeline 44444411 (CI Visibility). The following jobs did not run any unit tests:
Jobs:
- tests_deb-arm64-py3
- tests_deb-x64-py3
- tests_flavor_dogstatsd_deb-x64
- tests_flavor_heroku_deb-x64
- tests_flavor_iot_deb-x64
- tests_rpm-arm64-py3
- tests_rpm-x64-py3
- tests_windows-x64
If you modified Go files and expected unit tests to run in these jobs, please double check the job logs. If you think tests should have been executed reach out to #agent-devx-help
Regression Detector
Regression Detector Results
Run ID: b8288073-b3b2-4855-afe0-52383df74d0c Metrics dashboard Target profiles
Baseline: 7c22280f70f2d0c26f8ba7c25ff58879ddd5c6bc Comparison: ea6938b9d055b0d756906785181d282fea45e3fb
Performance changes are noted in the perf column of each table:
- ✅ = significantly better comparison variant performance
- ❌ = significantly worse comparison variant performance
- ➖ = no significant change in performance
No significant changes in experiment optimization goals
Confidence level: 90.00% Effect size tolerance: |Δ mean %| ≥ 5.00%
There were no significant changes in experiment optimization goals at this confidence level and effect size tolerance.
Fine details of change detection per experiment
| perf | experiment | goal | Δ mean % | Δ mean % CI | trials | links |
|---|---|---|---|---|---|---|
| ➖ | uds_dogstatsd_to_api_cpu | % cpu utilization | +1.97 | [+1.20, +2.73] | 1 | Logs |
| ➖ | idle | memory utilization | +0.66 | [+0.61, +0.72] | 1 | Logs |
| ➖ | basic_py_check | % cpu utilization | +0.37 | [-2.43, +3.18] | 1 | Logs |
| ➖ | tcp_dd_logs_filter_exclude | ingress throughput | +0.00 | [-0.01, +0.01] | 1 | Logs |
| ➖ | uds_dogstatsd_to_api | ingress throughput | -0.00 | [-0.00, +0.00] | 1 | Logs |
| ➖ | file_tree | memory utilization | -0.24 | [-0.35, -0.13] | 1 | Logs |
| ➖ | pycheck_lots_of_tags | % cpu utilization | -0.31 | [-2.93, +2.31] | 1 | Logs |
| ➖ | otel_to_otel_logs | ingress throughput | -0.78 | [-1.60, +0.05] | 1 | Logs |
| ➖ | tcp_syslog_to_blackhole | ingress throughput | -0.88 | [-0.93, -0.83] | 1 | Logs |
Bounds Checks
| perf | experiment | bounds_check_name | replicates_passed |
|---|---|---|---|
| ❌ | idle | memory_usage | 9/10 |
Explanation
A regression test is an A/B test of target performance in a repeatable rig, where "performance" is measured as "comparison variant minus baseline variant" for an optimization goal (e.g., ingress throughput). Due to intrinsic variability in measuring that goal, we can only estimate its mean value for each experiment; we report uncertainty in that value as a 90.00% confidence interval denoted "Δ mean % CI".
For each experiment, we decide whether a change in performance is a "regression" -- a change worth investigating further -- if all of the following criteria are true:
-
Its estimated |Δ mean %| ≥ 5.00%, indicating the change is big enough to merit a closer look.
-
Its 90.00% confidence interval "Δ mean % CI" does not contain zero, indicating that if our statistical model is accurate, there is at least a 90.00% chance there is a difference in performance between baseline and comparison variants.
-
Its configuration does not mark it "erratic".
:steam_locomotive: MergeQueue: pull request added to the queue
The median merge time in main is 23m.
Use /merge -c to cancel this operation!