insert_pdf gives SystemError

[wz93672]

Description of the bug

Happens only:

• if pythonw.exe is used to run the script
• for specific pdf files
• if multiple affected files are inserted, exception is thrown only for first one

SystemError: null argument to internal routine

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "C:\Users\witek\Desktop\test\test.py", line 19, in <module>
    newdoc.insert_pdf(doc)
  File "C:\Program Files\Python311\Lib\site-packages\pymupdf\__init__.py", line 4483, in insert_pdf
    extra_FzDocument_insert_pdf(
  File "C:\Program Files\Python311\Lib\site-packages\pymupdf\extra.py", line 72, in FzDocument_insert_pdf
    return _extra.FzDocument_insert_pdf(doc, src, from_page, to_page, start_at, rotate, links, annots, show_progress, final, graft_map)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
SystemError: <built-in function FzDocument_insert_pdf> returned a result with an exception set

How to reproduce the bug

GP_v2_17_2019.pdf

run with pythonw.exe

#test.py

import fitz
import datetime
import traceback
from pathlib import Path


saveargs = {'garbage':4, 'clean':True, 'deflate':True, 'deflate_images':True, 'deflate_fonts':True,}
pdfdir = Path('./pdf')


with fitz.open() as newdoc:
    
    for f in pdfdir.iterdir():
        if f.is_file() and f.suffix.lower() == '.pdf' and not f.name.startswith('_'):
            
            with fitz.open(f) as doc:
                try:
                    newdoc.insert_pdf(doc)
                    
                except Exception as e:
                    with open(pdfdir.joinpath('_errorlog.txt'), 'ta', encoding='utf-8') as logf:
                        logf.write(str(datetime.datetime.now())+'\n')
                        logf.write(str(f)+'\n')
                        logf.write('\n')
                        logf.write(traceback.format_exc())
                        logf.write('\n\n')


    newdoc.save(str(pdfdir.joinpath('_out.pdf')), **saveargs)

PyMuPDF version

1.24.3

Operating system

Windows

Python version

3.11

[julian-smith-artifex-com]

I haven't been able to reproduce this, but according to https://stackoverflow.com/questions/9705982/pythonw-exe-or-python-exe#30313091, pythonw.exe does not provide stdout or stderr, and attempting to write to them may cause an exception.

Certainly the PDF file you are using causes PyMuPDF to generate a warning diagnostic on stdout or stderr, so i think this may be causing the exception when run by pythonw.exe.

Is there any particular reason why you are using pythonw.exe here?

[wz93672]

I have a GUI application packed with pyinstaller, for sorting, counting, merging documents from different sources. It was my guess too, that a warning and lack of stdout/stderr could cause that error, I can escape and pass it, but I have done sanity check and tried the example in a different environment. Didn't get the SystemError with python 3.10.11 and pymupdf 1.23.0. And I made mistake yesterday, I'm using python 3.11.9 currently, not 3.12.

[julian-smith-artifex-com]

I'm still struggling to reproduce the original problem, even using python-3.11 on windows. But i have found that print() copes even if sys.stdout is None.

In the meantime there's another open issue about use with pyinstaller, which has lead to some changes to make all diagnostics go through python print(), instead of variously being generated with sys.stderr.write(), PySys_WriteStdout() etc.

It's possible that these changes may help with this issue. But as i can't reproduce it, i think we'll have to wait for you to test a future release.

[wz93672]

ok.

I've checked different pymupdf versions in fresh venvs. The error appears with new pymupdf implementation in 1.23.9, python version doesn't matter.

[wz93672]

Fixed in 1.24.4. Thanks.

[julian-smith-artifex-com]

That's great, thanks for reporting.