IPP backend does not return a retry exit code on document-format-error errors

[michaelrsweet]

trafficstars

Related cups-filter issue: https://github.com/OpenPrinting/cups-filters/issues/463

Currently the IPP backend does not return the "retry current" exit code (7, CUPS_BACKEND_RETRY_CURRENT) when it sees a 'document-format-error' keyword in the "job-state-reasons" attribute for the submitted job. The job is basically treated as completed successfully and you get no output on the printer.

Ultimately this points to a bug in pdftopdf, but the retry-as-raster logic is being skipped which would have at least allowed the user to get dots on paper...

[jberkenbilt]

Is it possible to provide the exact pdftopdf command, the input PDF, and the output PDF so I can make sure I can reproduce this? Assuming you have a way to send raw bits to the printer (presumably that's what ipptool does), it would be useful to see if there's anything qpdf does inherently that breaks this, for example, by sending in.pdf directly to the printer (presumably works), the running qpdf in.pdf out.pdf and sending out.pdf directly to the printer. This would bypass pdftopdf entirely and would see whether there's anything qpdf is doing that's obvious. A second test would be something like qpdf --empty --pages in.pdf -- out.pdf which would strip document-level metadata which could be happening with pdftopdf depending on how it uses the qpdf library. These experiments would give us a little more to go on. The page copying code in qpdf is tricky, though I intend to improve it considerably soon. It's close to the top of my to-do list if significant features at this point.

[mario-tux]

Is it possible to provide the exact pdftopdf command, the input PDF, and the output PDF so I can make sure I can reproduce this?

I carried on the tests reported in other bug report (https://github.com/OpenPrinting/cups-filters/issues/463) using evince, Master PDF Editor and some direct prints using ipptool. The issue raises, as long I can see, with any PDF. For sake of concreteness I did the previous tests using a public available PDF example file: https://www.orimi.com/pdf-test.pdf

As preliminar note I want to stress that this is a regression: the printer worked well on my Arch Linux and now I can't print almost any PDF file. The printer is correctly working on another computer using Debian and CUPS 2.3.3 using the driver with description "Brother MFC-L6900DW series, driverless, cups-filters 1.28.7 (grayscale, 2-sided printing)" and evince.

Assuming you have a way to send raw bits to the printer (presumably that's what ipptool does), it would be useful to see if there's anything qpdf does inherently that breaks this, for example, by sending in.pdf directly to the printer (presumably works), the running qpdf in.pdf out.pdf and sending out.pdf directly to the printer. This would bypass pdftopdf entirely and would see whether there's anything qpdf is doing that's obvious.

test-1: printing pdf-test.pdf using evince: nothing on the tray; test-2: ipptool -tf pdf-test.pdf ipp://boromir.lan print-job.test: it worked; test-3: qpdf pdf-test.pdf pdf-test-qpdf.pdf ; ipptool -tf pdf-test-qpdf.pdf ipp://boromir.lan print-job.test: again nothing of the tray! This is the resulting failing pdf file generated by qpdf 10.6.3: pdf-test-qpdf.pdf;

A second test would be something like qpdf --empty --pages in.pdf -- out.pdf which would strip document-level metadata which could be happening with pdftopdf depending on how it uses the qpdf library. These experiments would give us a little more to go on. The page copying code in qpdf is tricky, though I intend to improve it considerably soon. It's close to the top of my to-do list if significant features at this point.

test-4: qpdf --empty --pages pdf-test.pdf -- pdf-test-qpdf-empty.pdf ; ipptool -tf pdf-test-qpdf-empty.pdf ipp://boromir.lan print-job.test: it worked... The resulting file: pdf-test-qpdf-empty.pdf

I hope it help. Thanks for you efforts.

[jberkenbilt]

I notice that pdf-test.pdf is encrypted. That shouldn't matter (since there is no password), but maybe it does. I was surprised that the --pages one printed and the other one didn't. That's the opposite of what I was expecting. Try qpdf --decrypt pdf-test.pdf out.pdf (or whatever you want to call it) and see if that prints with ipptool.

Do you know what version of qpdf you had the last time this worked? I checked going back to 8.0.2, and there are no differences between the PDF files that qpdf generates. This file is encrypted using one of the less secure encryption methods (128-bit, RC4). Lately some people have gotten paranoid and removed support for those even though it's kind of goofy to do that for PDF readers. I don't know if Arch Linux may have done something or if there is some kind of update to the printer firmware or something, but that would be very odd.

Here are a few things to test.

• Explicitly run some test files through qpdf --decrypt
• Grab a live CD of Ubuntu 22.04 (or Fedora or whatever -- something else but recent) and try printing from there

You say this happens with any PDF, but is it possible that the ones you are trying are encrypted? The best way to get the details on the encryption is to run qpdf --json --json-key yourfile.pdf

I'm not sure under what circumstances files pass through pdftopdf. Is it always when printing through cups, or is it only of n-up or other operations are required? I'm not sure what pdftopdf does regarding encryption, but unless it is explicitly calling QPDFWriter::copyEncryptionParameters, it would transparently send a non-encrypted output file regardless of whether the input was encrypted.

[mario-tux]

Do you know what version of qpdf you had the last time this worked? I checked going back to 8.0.2, and there are no differences between the PDF files that qpdf generates.

Sorry, I can't figure out the exact version. But I did some regression tests using the qpdf packages I found in my pacman cache: I can confirm that test-1 and test-3 continue to fail even reverting on qpdf 10.5.0, 10.4.0 and 10.3.0. So it looks not a regression in qpdf.

Here are a few things to test.

• Explicitly run some test files through qpdf --decrypt

test-4: qpdf --decrypt pdf-test.pdf pdf-test-qpdf-decrypt.pdf ; ipptool -tf pdf-test-qpdf-decrypt.pdf ipp://boromir.lan print-job.test: it didn't work.

You say this happens with any PDF, but is it possible that the ones you are trying are encrypted? The best way to get the details on the encryption is to run qpdf --json --json-key yourfile.pdf

I repeated the previous tests using a naive PDF file generated by LibreOffice; here it is: my-pdf-test.pdf; this is not encrypted:

$ qpdf --json --json-key=encrypt my-pdf-test.pdf 
{
  "encrypt": {
    "capabilities": {
      "accessibility": true,
      "extract": true,
      "moddifyannotations": true,
      "modify": true,
      "modifyassembly": true,
      "modifyforms": true,
      "modifyother": true,
      "printhigh": true,
      "printlow": true
    },
    "encrypted": false,
    "ownerpasswordmatched": false,
    "parameters": {
      "P": 0,
      "R": 0,
      "V": 0,
      "bits": 0,
      "filemethod": "none",
      "key": null,
      "method": "none",
      "streammethod": "none",
      "stringmethod": "none"
    },
    "userpasswordmatched": false
  },
  "parameters": {
    "decodelevel": "generalized"
  },
  "version": 1
}

test-1b: printing my-pdf-test.pdf using evince: nothing... test-2b: ipptool -tf my-pdf-test.pdf ipp://boromir.lan print-job.test: it worked again; test-3b: qpdf my-pdf-test.pdf my-pdf-test-qpdf.pdf ; ipptool -tf my-pdf-test-qpdf.pdf ipp://boromir.lan print-job.test: wow, it worked... :-
(I just repeated test-3 with pdf-test.pdf file and I can confirm that it doesn't work)

• Grab a live CD of Ubuntu 22.04 (or Fedora or whatever -- something else but recent) and try printing from there

I launched a live of Ubuntu 22.04 on a virtual machine; I was not able to use/install qpdf from the live; if this is important I can retry; the other tests (test-[1,2,1b,2b]) worked using the auto-discovered printer (I guess using IPP everywhere) or the direct print; this is a screenshot of the former part of the generated PPD file:

I'm not sure under what circumstances files pass through pdftopdf. Is it always when printing through cups, or is it only of n-up or other operations are required? I'm not sure what pdftopdf does regarding encryption, but unless it is explicitly calling QPDFWriter::copyEncryptionParameters, it would transparently send a non-encrypted output file regardless of whether the input was encrypted.

I can't help in this.

[jberkenbilt]

You would be able to install qpdf on the Ubuntu live image with

sudo apt update
sudo apt install qpdf

but if you were able to print from the live image just by using the autodiscovered printer, then I don't think it is necessary to do that test.

So it seems like you can print non-encrypted files fine with ipptool but perhaps not encrypted files? And from the Ubuntu live image you can print them? But from evince you can't print anything?

If I've followed all this correctly, it's starting to look to me like something messed up on your specific machine configuration. I have never use Arch Linux. Does it have a live CD? Is it possible for you in some way to print from a fresh install of whatever version of Arch Linux you have? For example, you could install virtualbox and boot up a fresh Arch Linux from there and try printing from there, or you could use a live CD.

As one more experiment, you could take your libreoffice test file and encrypt it and see what kinds of results you get. qpdf --encrypt "" owner-password 128 -- --allow-weak-crypto in.pdf out.pdf

If this fails from ipptool then it seems like there is some issue printing encrypted files at least some of the time. But if you have files that print fine from ipptool and not from evince, I'm not sure what that means. Maybe someone from the linux printing side can help rule in/out pdftopdf. I'm just not sure what the path is from evince to the printer in terms of the local cups, filters, etc. Sometimes I have tested by literally running nc printer-ip 9100 < somefile.pdf, giving it a few seconds, and then if still running hitting ctrl-c. This definitely bypasses everything that's not the printer. I used to have a brother printer and found it to be somewhat flaky. I currently have a canon printer which is also somewhat flaky but less so than the brother printer.

If this were a debian-based system, I'd suggest doing an apt purge of all the printing related tools and then reinstalling, but I don't have enough familiarity with Arch to know how to do this and I wouldn't want to send you down a path of breaking your system.

I'm about out of ideas at this point. I think it's very unlikely to be a qpdf issue, and I'm not just saying that because it gets me off the hook. :-)

[tillkamppeter]

On Ubuntu everything printed because the printer did not receive PDF. The PPD has cupsFilter2: ... lines for Apple Raster and PWG Raster with much lower cost values making CUPS throwing in gstoraster or pdftoraster after pdftopdf and so QPDF's PDF output gets rasterized on the computer and easy-to-digest Raster passed on to the printer. I remember to have re-adjusted the cost values in the PPD file generator to favor Raster because some PDF printers had problems. Ubuntu 22.04 came for sure after this re-adjustment. If you update your Arch to a newer version or at least the cups-filters you are using, you will probably be able to print this way.

[michaelrsweet]

Looking at the current IPP backend code, it appears that a document-unprintable-error reason didn't trigger the retry-current exit code due to a typo in the backend code... :/

[master 6fa64459a] Fix document-unprintable-error support (Issue #391)

[2.4.x 939368f09] Fix document-unprintable-error support (Issue #391)