Modularize zdtm

[rst0git]

The zdtm.py file has grown over time to more than 2500 lines. It contains a number of classes, functions and variables all combined into a single monolithic python module.

This patch moves test classes, helper functions, flavors and exceptions into separate files that are imported in zdtm.py. This makes the code easier to extend and maintain in the future.

This pull request is rebased on top of https://github.com/checkpoint-restore/criu/pull/1581.

[codecov-commenter]

Codecov Report

Merging #1733 (4c73060) into criu-dev (4173ef1) will increase coverage by 0.01%. The diff coverage is n/a.

@@             Coverage Diff              @@
##           criu-dev    #1733      +/-   ##
============================================
+ Coverage     70.69%   70.70%   +0.01%     
============================================
  Files           127      127              
  Lines         31165    31166       +1     
============================================
+ Hits          22031    22035       +4     
+ Misses         9134     9131       -3     

Impacted Files	Coverage Δ
criu/sk-unix.c	73.96% <0.00%> (+0.02%)	:arrow_up:
criu/uffd.c	79.84% <0.00%> (+0.47%)	:arrow_up:

---

Continue to review full report at Codecov.

Legend - Click here to learn more Δ = absolute <relative> (impact), ø = not affected, ? = missing data Powered by Codecov. Last update 4173ef1...4c73060. Read the comment docs.

[rst0git]

Rebased on criu-dev branch and updated PR.

[avagin]

@rst0git CI jobs failed.

[avagin]

===================== Run zdtm/transition/thread-bomb in h =====================
Start test
Traceback (most recent call last):
  File "zdtm.py", line 1939, in <module>
    fork_zdtm()
  File "zdtm.py", line 1930, in fork_zdtm
    do_run_test(tinfo[0], tinfo[1], tinfo[2], tinfo[3])
  File "zdtm.py", line 1077, in do_run_test
    t.start()
  File "/tmp/criu/test/zdtm/zdtm.py", line 118, in start
    self.__make_action('pid', env, self.__flavor.root)
  File "/tmp/criu/test/zdtm/zdtm.py", line 60, in __make_action
    preexec_fn=self.__freezer and self.__freezer.attach or None)
  File "/usr/lib64/python2.7/subprocess.py", line 711, in __init__
    errread, errwrite)
  File "/usr/lib64/python2.7/subprocess.py", line 1327, in _execute_child
    raise child_exception
TypeError: must be unicode, not str

##################################### FAIL #####################################

[avagin]

Tests on CentOS7 stil fail with "TypeError: must be unicode, not str".

[github-actions[bot]]

A friendly reminder that this PR had no activity for 30 days.

[rst0git]

Tests on CentOS7 stil fail with "TypeError: must be unicode, not str".

@avagin Thanks for pointing it out. I fixed this error by adding the following import to all python modules.

from __future__ import unicode_literals

[Snorch]

-        # This Mayakovsky-style code gets list of libraries a binary
-        # needs minus vdso and gate .so-s
-        libs = map(
-            lambda x: x[1] == '=>' and x[2] or x[0],
-            map(
-                lambda x: str(x).split(),
-                filter(
-                    lambda x: not xl.match(x),
-                    map(
-                        lambda x: str(x).strip(),
-                        filter(lambda x: str(x).startswith('\t'),
-                               stdout.decode(
-                                   'ascii').splitlines())))))
-
-        for lib in libs:
-            if not os.access(lib, os.F_OK):
-                raise test_fail_exc("Can't find lib %s required by %s" %
-                                    (lib, binary))
-            self.__copy_one(lib)
+        # This Mayakovsky-style code gets list of libraries a binary
+        # needs minus vdso and gate .so-s
+        libs = map(
+            lambda x: x[1] == '=>' and x[2] or x[0],
+            map(
+                lambda x: builtins.str(x).split(),
+                filter(
+                    lambda x: not xl.match(x),
+                    map(
+                        lambda x: builtins.str(x).strip(),
+                        filter(lambda x: builtins.str(x).startswith('\t'),
+                               ldd.stdout.read().decode(
+                                   'ascii').splitlines())))))
+
+        ldd.wait()
+
+        for lib in libs:
+            if not os.access(lib, os.F_OK):
+                raise TestFailException("Can't find lib %s required by %s" %
+                                        (lib, binary))
+            self.__copy_one(lib)

Can we move all additional changes to separate patches with explanations? Why do you add ldd.wait() here? it was not present in the original code. Also I see some changes like range -> builtins.range

[rst0git]

@Snorch Thank you for the comment. I removed ldd.wait() and changed import builtins to from builtins import ...

[Snorch]

Same with "stdout->decode(" -> "ldd.stdout.read().decode(" in the same codeblock.

Also

-                self.__subs = list(map(lambda x: x.strip(), fd.readlines()))
+                self.__subs = map(lambda x: x.strip(), fd.readlines())

Please check

vimdiff <(git show HEAD | grep -v "^+++" | grep "^+" | sed 's/^+//' | sort) <(git show HEAD | grep -v "^---" | grep "^-" | sed 's/^-//' | sort)

The diff should be minimal as possible when we just move the code around.

Also suspicious:

[snorch@turmoil criu]$ git show | grep "s.communicate()"
-        out, _ = s.communicate()

[rst0git]

Thank you for catching this @Snorch! It looks like most of these were changes from other patches (e.g., https://github.com/checkpoint-restore/criu/pull/1642) that I missed when rebasing this patch. I have updated the pull request to include these changes.