Cannot roundtrip Python 3.9 stdlib xml/dom/minidom.py

[bnavigator]

[    2s] + PYTHONPATH=/home/abuild/rpmbuild/BUILDROOT/python-astor-0.8.1-0.x86_64/usr/lib/python3.9/site-packages
[    2s] + PYTHONDONTWRITEBYTECODE=1
[    2s] + pytest-3.9 --ignore=_build.python39 --ignore=_build.python38 -v tests
[    2s] ============================= test session starts ==============================
[    2s] platform linux -- Python 3.9.2, pytest-6.2.2, py-1.10.0, pluggy-0.13.1 -- /usr/bin/python3.9
[    2s] cachedir: .pytest_cache
[    2s] rootdir: /home/abuild/rpmbuild/BUILD/astor-0.8.1
[    2s] collecting ... collected 55 items
...
[    9s] =================================== FAILURES ===================================
[    9s] ______________________ RtripTestCase.test_convert_stdlib _______________________
[    9s] 
[    9s] self = <tests.test_rtrip.RtripTestCase testMethod=test_convert_stdlib>
[    9s] 
[    9s]     def test_convert_stdlib(self):
[    9s]         srcdir = os.path.dirname(os.__file__)
[    9s]         result = astor.rtrip.convert(srcdir)
[    9s] >       self.assertEqual(result, [])
[    9s] E       AssertionError: Lists differ: ['/usr/lib64/python3.9/xml/dom/minidom.py'] != []
[    9s] E       
[    9s] E       First list contains 1 additional elements.
[    9s] E       First extra element 0:
[    9s] E       '/usr/lib64/python3.9/xml/dom/minidom.py'
[    9s] E       
[    9s] E       - ['/usr/lib64/python3.9/xml/dom/minidom.py']
[    9s] E       + []
[    9s] 
[    9s] tests/test_rtrip.py:24: AssertionError
[    9s] ------------------------------ Captured log call -------------------------------
[    9s] WARNING  root:rtrip.py:112     calculating dump -- bad
[    9s] WARNING  root:rtrip.py:143 
[    9s] Files failed to round-trip to AST:
[    9s] WARNING  root:rtrip.py:145     /usr/lib64/python3.9/xml/dom/minidom.py
[    9s] =========================== short test summary info ============================
[    9s] FAILED tests/test_rtrip.py::RtripTestCase::test_convert_stdlib - AssertionErr...
[    9s] =================== 1 failed, 52 passed, 2 skipped in 7.26s ====================

Maybe related to #181, because

abuild@skylab:~> diff -u /usr/lib64/python3.[89]/xml/dom/minidom.py
...
@@ -1787,12 +1811,17 @@
             raise xml.dom.NotSupportedErr("cannot import document type nodes")
         return _clone_node(node, deep, self)
 
-    def writexml(self, writer, indent="", addindent="", newl="", encoding=None):
-        if encoding is None:
-            writer.write('<?xml version="1.0" ?>'+newl)
-        else:
-            writer.write('<?xml version="1.0" encoding="%s"?>%s' % (
-                encoding, newl))
+    def writexml(self, writer, indent="", addindent="", newl="", encoding=None,
+                 standalone=None):
+        declarations = []
+
+        if encoding:
+            declarations.append(f'encoding="{encoding}"')
+        if standalone is not None:
+            declarations.append(f'standalone="{"yes" if standalone else "no"}"')
+
+        writer.write(f'<?xml version="1.0" {" ".join(declarations)}?>{newl}')
+
         for node in self.childNodes:
             node.writexml(writer, indent, addindent, newl) 

[akaihola]

I applied this PR:

• #230

and ran the test on Python 3.12. The test failure shows these details:

self = <tests.test_rtrip.RtripTestCase testMethod=test_convert_stdlib>

    def test_convert_stdlib(self):
        srcdir = os.path.dirname(os.__file__)
        failed_files = astor.rtrip.convert(srcdir)
    
        for file_path in failed_files:
            with open(file_path, 'r', encoding='utf-8') as f:
                source = f.read()
    
            # Parse original source to AST
            original_ast = ast.parse(source)
    
            # Convert AST back to source and re-parse
            converted_source = to_source(original_ast)
>           converted_ast = ast.parse(converted_source)

tests/test_rtrip.py:37: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

source = '"""Simple implementation of the Level 1 DOM.\n\nNamespaces and other minor Level 2 features are also supported.\n\npa...   if not Document.implementation.hasFeature(f, v):\n                return None\n    return Document.implementation\n'
filename = '<unknown>', mode = 'exec'

    def parse(source, filename='<unknown>', mode='exec', *,
              type_comments=False, feature_version=None):
        """
        Parse the source into an AST node.
        Equivalent to compile(source, filename, mode, PyCF_ONLY_AST).
        Pass type_comments=True to get back type comments where the syntax allows.
        """
        flags = PyCF_ONLY_AST
        if type_comments:
            flags |= PyCF_TYPE_COMMENTS
        if feature_version is None:
            feature_version = -1
        elif isinstance(feature_version, tuple):
            major, minor = feature_version  # Should be a 2-tuple.
            if major != 3:
                raise ValueError(f"Unsupported major version: {major}")
            feature_version = minor
        # Else it should be an int giving the minor version for 3.x.
>       return compile(source, filename, mode, flags,
                       _feature_version=feature_version)
E         File "<unknown>", line 1693
E           f'standalone="{\'yes\' if standalone else \'no\'}"')
E                           ^
E       SyntaxError: unexpected character after line continuation character

lib/python3.12/ast.py:52: SyntaxError
------------------------------------------------------------------------------- Captured log call -------------------------------------------------------------------------------
WARNING  root:rtrip.py:112     calculating dump -- bad
WARNING  root:rtrip.py:143 
Files failed to round-trip to AST:
WARNING  root:rtrip.py:145     lib/python3.12/xml/dom/minidom.py