[Backport 2023.1] feature(artifacts-tests): add support for ubuntu 24.04

[mergify[bot]]

as part of an effort to start supporting using ubuntu 24.04 as the basis for scylla images, we are introducing a new artifact tests to cover it

Closes: scylladb/scylla-cluster-tests#7410

Testing

• [x] run local from dev machine
• [x] x86: https://jenkins.scylladb.com/job/scylla-staging/job/fruch/job/artifacts-ubuntu2404-test/2/
• [x] arm: https://jenkins.scylladb.com/job/scylla-staging/job/fruch/job/artifacts-ubuntu2404-arm-test/5/
• [x] offline installer: https://jenkins.scylladb.com/job/scylla-staging/job/fruch/job/artifacts-ubuntu2404-test/6/
• [x] offline installer non-root: https://jenkins.scylladb.com/job/scylla-staging/job/fruch/job/artifacts-ubuntu2404-test/5/

PR pre-checks (self review)

• [x] I added the relevant backport labels
• [x] I didn't leave commented-out/debugging code

Reminders

• Add New configuration option and document them (in sdcm/sct_config.py)
• Add unit tests to cover my changes (under unit-test/ folder)
• Update the Readme/doc folder relevant to this change (if needed)

(cherry picked from commit 9284e91b6f4a0799510fd9d017917cbf40665c1d )

(cherry picked from commit 1d818d2e1ecf24c022f310bc235feb8cb945760f )

(cherry picked from commit df13abc5261d69f3c2eae5448eebd2ce8f5c7f53 )

Parent PR: #7461

[mergify[bot]]

Cherry-pick of 9284e91b6f4a0799510fd9d017917cbf40665c1d has failed:

On branch mergify/bp/branch-2023.1/pr-7461
Your branch is up to date with 'origin/branch-2023.1'.

You are currently cherry-picking commit 9284e91b.
  (fix conflicts and run "git cherry-pick --continue")
  (use "git cherry-pick --skip" to skip this patch)
  (use "git cherry-pick --abort" to cancel the cherry-pick operation)

Changes to be committed:
	modified:   unit_tests/test_config.py

Unmerged paths:
  (use "git add <file>..." to mark resolution)
	both modified:   sdcm/utils/common.py

Cherry-pick of 1d818d2e1ecf24c022f310bc235feb8cb945760f has failed:

On branch mergify/bp/branch-2023.1/pr-7461
Your branch is ahead of 'origin/branch-2023.1' by 1 commit.
  (use "git push" to publish your local commits)

You are currently cherry-picking commit 1d818d2e.
  (fix conflicts and run "git cherry-pick --continue")
  (use "git cherry-pick --skip" to skip this patch)
  (use "git cherry-pick --abort" to cancel the cherry-pick operation)

Changes to be committed:
	modified:   artifacts_test.py
	new file:   configurations/arm/ubuntu2404.yaml
	new file:   jenkins-pipelines/oss/artifacts-offline-install/artifacts-ubuntu2404-arm.jenkinsfile
	new file:   jenkins-pipelines/oss/artifacts-offline-install/artifacts-ubuntu2404.jenkinsfile
	new file:   jenkins-pipelines/oss/artifacts/artifacts-ubuntu2404-arm.jenkinsfile
	new file:   jenkins-pipelines/oss/artifacts/artifacts-ubuntu2404.jenkinsfile
	modified:   sdcm/mgmt/common.py
	modified:   sdcm/utils/distro.py
	new file:   test-cases/artifacts/ubuntu2404.yaml

Unmerged paths:
  (use "git add <file>..." to mark resolution)
	both modified:   defaults/manager_versions.yaml
	added by them:   jenkins-pipelines/artifacts-ubuntu2404-nonroot.jenkinsfile

To fix up this pull request, you can check it out locally. See documentation: https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/reviewing-changes-in-pull-requests/checking-out-pull-requests-locally

[Annamikhlin]

running tests on releng-testing based on your branch and https://github.com/scylladb/scylla-enterprise-pkg/pull/282

• unified-deb - https://jenkins.scylladb.com/job/enterprise-2023.1/job/releng-testing/job/unified-deb/11/
• offline-installer - https://jenkins.scylladb.com/job/enterprise-2023.1/job/releng-testing/job/offline-installer/3/

[Annamikhlin]

@fruch - could you please take a look on the failure https://jenkins.scylladb.com/job/enterprise-2023.1/job/releng-testing/job/artifacts-offline-install/job/artifacts-ubuntu2204-test/5/?

[Annamikhlin]

Regarding https://jenkins.scylladb.com/job/enterprise-2023.1/job/releng-testing/job/artifacts/job/artifacts-ubuntu2404-test/3/ and https://jenkins.scylladb.com/job/enterprise-2023.1/job/releng-testing/job/artifacts/job/artifacts-ubuntu2404-arm-test/3/

13:01:46  Command: 'sudo DEBIAN_FRONTEND=noninteractive apt-get -o Dpkg::Options::="--force-confold" -o Dpkg::Options::="--force-confdef" install -y scylla-enterprise'
13:01:46  
13:01:46  Exit code: 100
13:01:46  
13:01:46  Stdout:
13:01:46  .....
13:01:46  The following packages have unmet dependencies:
13:01:46   scylla-enterprise-tools-core : Depends: python (>= 2.7) but it is not installable or
13:01:46                                           python2 but it is not installable
13:01:46  
13:01:46  Stderr:
13:01:46  
13:01:46  E: Unable to correct problems, you have held broken packages.

is it related to https://github.com/scylladb/scylla-cluster-tests/pull/8528?

[fruch]

Regarding https://jenkins.scylladb.com/job/enterprise-2023.1/job/releng-testing/job/artifacts/job/artifacts-ubuntu2404-test/3/ and https://jenkins.scylladb.com/job/enterprise-2023.1/job/releng-testing/job/artifacts/job/artifacts-ubuntu2404-arm-test/3/

13:01:46  Command: 'sudo DEBIAN_FRONTEND=noninteractive apt-get -o Dpkg::Options::="--force-confold" -o Dpkg::Options::="--force-confdef" install -y scylla-enterprise'
13:01:46  
13:01:46  Exit code: 100
13:01:46  
13:01:46  Stdout:
13:01:46  .....
13:01:46  The following packages have unmet dependencies:
13:01:46   scylla-enterprise-tools-core : Depends: python (>= 2.7) but it is not installable or
13:01:46                                           python2 but it is not installable
13:01:46  
13:01:46  Stderr:
13:01:46  
13:01:46  E: Unable to correct problems, you have held broken packages.

is it related to #8528?

no, it's a inherit problem, 2023.1 doesn't have the new cqlsh, hence it can't be installed on places that doesn't have python2 We are not gonna backport cqlsh repackage work into 2023.1 (at least I'm not going to do that...)

[Annamikhlin]

Regarding https://jenkins.scylladb.com/job/enterprise-2023.1/job/releng-testing/job/artifacts/job/artifacts-ubuntu2404-test/3/ and https://jenkins.scylladb.com/job/enterprise-2023.1/job/releng-testing/job/artifacts/job/artifacts-ubuntu2404-arm-test/3/

13:01:46  Command: 'sudo DEBIAN_FRONTEND=noninteractive apt-get -o Dpkg::Options::="--force-confold" -o Dpkg::Options::="--force-confdef" install -y scylla-enterprise'
13:01:46  
13:01:46  Exit code: 100
13:01:46  
13:01:46  Stdout:
13:01:46  .....
13:01:46  The following packages have unmet dependencies:
13:01:46   scylla-enterprise-tools-core : Depends: python (>= 2.7) but it is not installable or
13:01:46                                           python2 but it is not installable
13:01:46  
13:01:46  Stderr:
13:01:46  
13:01:46  E: Unable to correct problems, you have held broken packages.

is it related to #8528?

no, it's a inherit problem, 2023.1 doesn't have the new cqlsh, hence it can't be installed on places that doesn't have python2 We are not gonna backport cqlsh repackage work into 2023.1 (at least I'm not going to do that...)

ohh.. /cc @mykaul @yaronkaikov

[mykaul]

@fruch - just so I understand - cqlsh does work against 2023.1, just not where there's no Python 2, so on Ubuntu 24.04 we'll need (for example) use a dockerized cqlsh?

[fruch]

@fruch - just so I understand - cqlsh does work against 2023.1, just not where there's no Python 2, so on Ubuntu 24.04 we'll need (for example) use a dockerized cqlsh?

it's a bit more complicated, since cqlsh is part of scylla installation, and it can't be installed (i.e. scylla can't) without python2. so it need to be remove from the installation, before one can think of using a docker version of cqlsh. ontop of it, tests in dtest/sct assume cqlsh exists as part of scylla installation (and users as well), so that as well is a not that small change as well.

I don't think removing cqlsh it from 2023.1 is really an option. (also we never planned to remove it from scylla installation)

[fruch]

we found a different solution for the reason why we needed this support on 2023.1, no need for this one anymore