[TEST] Bump Docker container base images to Ubuntu22.04

[jleaniz]

[jleaniz]

GIFT PPA Ubuntu Jammy track currently missing docker-explorer packages.

[jleaniz]

bulk-extractor and docker-explorer were added to GIFT PPA stable/staging tracks for Ubuntu Jammy. Now showing some other library conflicts:

 ---> 4cb3e75152b8
Step 12/36 : RUN apt-get update && apt-get -y install     bulk-extractor     dfimagetools-tools     docker-explorer-tools     libbde-tools     libewf     libewf-python3     libewf-tools     plaso-tools     python3-dfimagetools     python3-dfvfs     python3-plaso     sleuthkit     --option Acquire::ForceIPv4=true --option Acquire::Retries=100 --option Acquire::http::Timeout=60
 ---> Running in 6a6bd9488cbe
Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease
Hit:2 http://archive.ubuntu.com/ubuntu jammy-updates InRelease
Hit:3 http://archive.ubuntu.com/ubuntu jammy-backports InRelease
Get:4 http://security.ubuntu.com/ubuntu jammy-security InRelease [110 kB]
Hit:5 https://ppa.launchpadcontent.net/gift/staging/ubuntu jammy InRelease
Fetched 110 kB in 1s (186 kB/s)
Reading package lists...
Reading package lists...
Building dependency tree...
Reading state information...
Some packages could not be installed. This may mean that you have
requested an impossible situation or if you are using the unstable
distribution that some required packages have not yet been created
or been moved out of Incoming.
The following information may help to resolve the situation:

The following packages have unmet dependencies:
 libewf : Conflicts: libewf2 but 20140807-2ubuntu2 is to be installed
 libvhdi : Conflicts: libvhdi1 but 20210425-1build1 is to be installed
 libvmdk : Conflicts: libvmdk1 but 20200926-2build1 is to be installed
E: Unable to correct problems, you have held broken packages.
The command '/bin/sh -c apt-get update && apt-get -y install     bulk-extractor     dfimagetools-tools     docker-explorer-tools     libbde-tools     libewf     libewf-python3     libewf-tools     plaso-tools     python3-dfimagetools     python3-dfvfs     python3-plaso     sleuthkit     --option Acquire::ForceIPv4=true --option Acquire::Retries=100 --option Acquire::http::Timeout=60' returned a non-zero code: 100

[aarontp]

What else is needed for this again? Is there a bug/issue tracking this? IIRC we still need some sleuthkit deps in the GIFT repo?

[jleaniz]

What else is needed for this again? Is there a bug/issue tracking this? IIRC we still need some sleuthkit deps in the GIFT repo?

Yeah i see the sleuthkit package was pushed 2 days ago, I'll rerun this and see what we get.

[aarontp]

Sounds good, thanks!

@berggren FYI

[jleaniz]

update-alternatives: error: alternative path /usr/lib/go-1.16/bin/go doesn't exist
The command '/bin/sh -c cd /opt/fraken/yara && curl -s https://api.github.com/repos/VirusTotal/Yara/releases/latest | sed -n 's/.*"tarball_url": "\(.*\)",.*/\1/p' | xargs -n1 wget -O - -q | tar -xz --strip-components=1     && ./bootstrap.sh     && ./configure     && make && sudo make install && sudo ldconfig     && sudo update-alternatives --install /usr/bin/go go /usr/lib/go-1.16/bin/go 10' returned a non-zero code: 2

Looks like Fraken tries to use go-1.16 but that is not available in 22.04. I see go 1.18 is available though. @Fryyyyy would that work with fraken?

[Fryyyyy]

Yeah the newer the better. I was forced to pin it to 1.16 for the old image since that was the newest version available there

[jleaniz]

Ubuntu 22.04 defaults to Python 3.10. We need to check the codebase for any code that needs to be upgraded. I found at least one issue with calls to collections.Callable which in python 3.10 has been moved to collections.abc.Callable. This looks related to the nose package used in run_tests.py, it seems like this nosetests library hasn't been maintained in years and will not support python 3.10

Traceback (most recent call last):
  File "/usr/local/bin/nosetests", line [8](https://github.com/google/turbinia/actions/runs/3425309140/jobs/5705970996#step:8:9), in <module>
    sys.exit(run_exit())
  File "/usr/local/lib/python3.10/dist-packages/nose/core.py", line 118, in __init__
    unittest.TestProgram.__init__(
  File "/usr/lib/python3.10/unittest/main.py", line 100, in __init__
    self.parseArgs(argv)
  File "/usr/local/lib/python3.10/dist-packages/nose/core.py", line 17[9](https://github.com/google/turbinia/actions/runs/3425309140/jobs/5705970996#step:8:10), in parseArgs
    self.createTests()
  File "/usr/local/lib/python3.[10](https://github.com/google/turbinia/actions/runs/3425309140/jobs/5705970996#step:8:11)/dist-packages/nose/core.py", line 193, in createTests
    self.test = self.testLoader.loadTestsFromNames(self.testNames)
  File "/usr/local/lib/python3.10/dist-packages/nose/loader.py", line 481, in loadTestsFromNames
    return unittest.TestLoader.loadTestsFromNames(self, names, module)
  File "/usr/lib/python3.10/unittest/loader.py", line 220, in loadTestsFromNames
    suites = [self.loadTestsFromName(name, module) for name in names]
  File "/usr/lib/python3.10/unittest/loader.py", line 220, in <listcomp>
    suites = [self.loadTestsFromName(name, module) for name in names]
  File "/usr/local/lib/python3.10/dist-packages/nose/loader.py", line 454, in loadTestsFromName
    return LazySuite(
  File "/usr/local/lib/python3.10/dist-packages/nose/suite.py", line 53, in __init__
    super(LazySuite, self).__init__()
  File "/usr/lib/python3.10/unittest/suite.py", line 22, in __init__
    self._tests = []
  File "/usr/local/lib/python3.10/dist-packages/nose/suite.py", line 106, in _set_tests
    if isinstance(tests, collections.Callable) and not is_suite:
AttributeError: module 'collections' has no attribute 'Callable'
Traceback (most recent call last):
  File "/tmp/./run_tests.py", line 21, in <module>
    subprocess.check_call([
  File "/usr/lib/python3.10/subprocess.py", line [36](https://github.com/google/turbinia/actions/runs/3425309140/jobs/5705970996#step:8:37)9, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['nosetests', '-vv', '--with-coverage', '--cover-package=turbinia', '--exe']' returned non-zero exit status 1.
Error: Process completed with exit code 1.

[jleaniz]

Note I added httpx to requirements.txt as this package is necessary for the fastapi test client.