paperless-ngx-postprocessor icon indicating copy to clipboard operation
paperless-ngx-postprocessor copied to clipboard

Update Paperless Release 2.16.1

Open dirk99 opened this issue 7 months ago • 5 comments

the script dosn't work until Update Paperless to release 2.16.1. is there any way to debug? Paperless show in the protocol

[2025-05-22 09:02:48,911] [INFO] [paperless.consumer] Executing post-consume script /usr/src/paperless-ngx-postprocessor/post_consume_script.sh [2025-05-22 09:02:52,108] [INFO] [paperless.consumer] /usr/src/paperless-ngx-postprocessor/post_consume_script.sh exited 0

The rules are working before Update Paperless. After Update Paperless i have run the setup_venv.sh. I have always delete the folder venv and run the setup script, but without any positive result. In my opinion there is a bug in the api from paperless.

dirk99 avatar May 22 '25 12:05 dirk99

I'm not sure I understand: which versions of Paperless-NGX does the postprocessor work with, and which versions does it not work with?

Either way, to debug further you can add the following line to the docker-compose.env file for Paperless-NGX:

PNGX_POSTPROCESSOR_VERBOSE=DEBUG

After making that change, reload Paperless-NGX as normal (e.g. docker-compose up -d) and try to process some new documents. The postprocessor will inherit that environment variable from Paperless-NGX, and print out a lot more detailed information in the logs.

jgillula avatar May 24 '25 20:05 jgillula

Hello jgillula,

with version 2.15.2 of the Paperless Webserver, the Paperless postprocessor still worked perfectly. after the update to version 2.16.1, the postprocessor no longer worked, although the same ruleset and the same documents were processed as in version 2.15.2. Since yesterday there is version 2.16.2 of the webserver, but even with this version nothing has changed. As I read in the Paperless changelog, they were working on the API and also on the timing of when the post process is triggered. I have set the environment variable “PAPERLESS_DEBUG: true” in my stack and only get the following log entries [2025-05-25 09:38:56,852] [INFO] [paperless.consumer] Executing post-consume script /usr/src/paperless-ngx-postprocessor/post_consume_script.sh

[2025-05-25 09:39:00,140] [INFO] [paperless.consumer] /usr/src/paperless-ngx-postprocessor/post_consume_script.sh exited 0

[2025-05-25 09:39:00,141] [INFO] [paperless.consumer] Document 2025-03-20 Volksbank 2025-04-26 Visa Volksbank Visa statement 02-03-2025 consumption finished

[2025-05-25 09:39:00,152] [INFO] [paperless.tasks] ConsumeTaskPlugin completed with: Success. New document id 3269 created

when I change the rule that the regex does not work I get another message [2025-05-25 09:29:58,316] [WARNING] [paperless.consumer] [2025-05-25 09:29:57,706] [WARNING] [postprocessor] Regex '(?:Sales statement.*?from (?P\d{2}.\d{2}.\d{4} to \d{2}.\d{2}.\d{4}))' for 'Visa billing name' didn't match for document_id=3268

So I can assume that the postprocessor really runs through the ruleset. But the result of the imported document is not what should be done in the ruleset when the rule is matched. Also the tag that the postprocessor was successful is not set, as well as the ASN is not set, as should happen in my last rule, which always takes effect. But even this rule is not enforced by the ASN. As described with Paperless Webserver 2.15.2 the rules runs fine.


#################################################################################### #ASN is set for each document #################################################################################### Set ASN Name: match: True metadata_postprocessing: asn: '{{ document_id }}'

dirk99 avatar May 25 '25 07:05 dirk99

Setting PAPERLESS_DEBUG: true won't cause the postprocessor to output any debug info. The postprocessor reads from its own different environment variable. As I mentioned earlier, please set PNGX_POSTPROCESSOR_VERBOSE=DEBUG and let me know if that results in any more debug info in the logs.

jgillula avatar May 25 '25 17:05 jgillula

now the dubug mode shows many messages,

[2025-05-25 20:13:00,320] [INFO] [paperless.consumer] Executing post-consume script /usr/src/paperless-ngx-postprocessor/post_consume_script.sh

[2025-05-25 20:13:03,681] [INFO] [paperless.consumer] /usr/src/paperless-ngx-postprocessor/post_consume_script.sh exited 0

[2025-05-25 20:13:03,682] [INFO] [paperless.consumer] /usr/src/paperless-ngx-postprocessor/post_consume_script.sh stderr:

[2025-05-25 20:13:03,682] [WARNING] [paperless.consumer] [2025-05-25 20:13:02,638] [DEBUG] [paperlessngx_postprocessor] Running /usr/src/paperless-ngx-postprocessor/paperlessngx_postprocessor.py with config {'auth_token': '3891b5201c4e098b30b74999cf459f886e6972b6', 'dry_run': False, 'skip_validation': False, 'backup': None, 'postprocessing_tag': 'Postprocess succesfull', 'invalid_tag': None, 'verbose': 'DEBUG', 'rulesets_dir': '/usr/src/paperless-ngx-postprocessor/rulesets.d', 'paperless_api_url': 'http://192.168.168.232:9000/api', 'paperless_src_dir': '/usr/src/paperless-ngx-postprocessor', 'mode': 'process', 'filename': None} and {'document_id': '3271', 'correspondent': None, 'document_type': None, 'tag': None, 'storage_path': None, 'created_year': None, 'created_month': None, 'created_day': None, 'created_range': None, 'added_month': None, 'added_day': None, 'added_range': None, 'asn': None, 'title': None, 'all': False}

[2025-05-25 20:13:03,683] [WARNING] [paperless.consumer] [2025-05-25 20:13:02,864] [DEBUG] [postprocessor] Loaded 7 rules

[2025-05-25 20:13:03,683] [WARNING] [paperless.consumer] [2025-05-25 20:13:02,924] [INFO] [paperlessngx_postprocessor] Processing 1 documents.

[2025-05-25 20:13:03,683] [WARNING] [paperless.consumer] [2025-05-25 20:13:03,109] [DEBUG] [postprocessor] metadata_in_filename_format={'document_id': 3271, 'correspondent': 'Volksbank', 'document_type': 'Finanzen - VisaCard-Voba', 'storage_path': 'Standart wie in der Grundkonfig', 'asn': None, 'tag_list': ['Finanzen - VisaCard-Voba', 'neues Dokument'], 'title': '2025-05-01 Visa Volksbank Visa Abrechnung 03-04-2025', 'created': '2025-04-17', 'created_year': '2025', 'created_month': '04', 'created_day': '17', 'created_date': '2025-04-17', 'created_date_object': datetime.datetime(2025, 4, 17, 0, 0), 'added': '2025-05-25T20:12:58.571914+02:00', 'added_year': '2025', 'added_month': '05', 'added_day': '25', 'added_date': '2025-05-25', 'added_date_object': datetime.datetime(2025, 5, 25, 20, 12, 58, 571914, tzinfo=tzoffset(None, 7200))}

[2025-05-25 20:13:03,683] [WARNING] [paperless.consumer] [2025-05-25 20:13:03,113] [DEBUG] [postprocessor] Rule test Name does not match

[2025-05-25 20:13:03,684] [WARNING] [paperless.consumer] [2025-05-25 20:13:03,114] [DEBUG] [postprocessor] Rule Visa Abrechnung Name matches

[2025-05-25 20:13:03,684] [WARNING] [paperless.consumer] [2025-05-25 20:13:03,114] [WARNING] [postprocessor] Regex '(?:Umsatzaufstellung.*j\n?vom (?P\d{2}.\d{2}.\d{4} bis \d{2}.\d{2}.\d{4}))' for 'Visa Abrechnung Name' didn't match for document_id=3271

[2025-05-25 20:13:03,684] [WARNING] [paperless.consumer] [2025-05-25 20:13:03,115] [DEBUG] [postprocessor] Updating 'zeitraum' using template {{ zeitraum | title }} and metadata {'created_date_object': datetime.datetime(2025, 4, 17, 0, 0), 'created_day': '17', 'created_date': '2025-04-17', 'added_date': '2025-05-25', 'title': '2025-05-01 Visa Volksbank Visa Abrechnung 03-04-2025', 'added_date_object': datetime.datetime(2025, 5, 25, 20, 12, 58, 571914, tzinfo=tzoffset(None, 7200)), 'asn': None, 'created': '2025-04-17', 'created_month': '04', 'created_year': '2025', 'correspondent': 'Volksbank', 'document_type': 'Finanzen - VisaCard-Voba', 'storage_path': 'Standart wie in der Grundkonfig', 'tag_list': ['Finanzen - VisaCard-Voba', 'neues Dokument'], 'added': '2025-05-25T20:12:58.571914+02:00', 'added_year': '2025', 'added_month': '05', 'added_day': '25', 'document_id': 3271}

[2025-05-25 20:13:03,684] [WARNING] [paperless.consumer] : 'None'->''

[2025-05-25 20:13:03,685] [WARNING] [paperless.consumer] [2025-05-25 20:13:03,116] [DEBUG] [postprocessor] Updating 'title' using template #{{ document_id }}#-{{ correspondent }}-VISA-Abrechnungszeitraum:{{zeitraum }} and metadata {'created_date_object': datetime.date(2025, 4, 17), 'created_day': '17', 'created_date': '2025-04-17', 'added_date': '2025-05-25', 'title': '2025-05-01 Visa Volksbank Visa Abrechnung 03-04-2025', 'added_date_object': datetime.datetime(2025, 5, 25, 20, 12, 58, 571914, tzinfo=tzoffset(None, 7200)), 'asn': None, 'created': '2025-04-17T00:00:00', 'created_month': '04', 'created_year': '2025', 'zeitraum': '', 'correspondent': 'Volksbank', 'document_type': 'Finanzen - VisaCard-Voba', 'storage_path': 'Standart wie in der Grundkonfig', 'tag_list': ['Finanzen - VisaCard-Voba', 'neues Dokument'], 'added': '2025-05-25T20:12:58.571914+02:00', 'added_year': '2025', 'added_month': '05', 'added_day': '25', 'document_id': 3271}

[2025-05-25 20:13:03,685] [WARNING] [paperless.consumer] : '2025-05-01 Visa Volksbank Visa Abrechnung 03-04-2025'->'#3271#-Volksbank-VISA-Abrechnungszeitraum:'

[2025-05-25 20:13:03,685] [WARNING] [paperless.consumer] [2025-05-25 20:13:03,117] [DEBUG] [postprocessor] Rule xxx does not match

[2025-05-25 20:13:03,685] [WARNING] [paperless.consumer] [2025-05-25 20:13:03,118] [DEBUG] [postprocessor] Rule yyy does not match

[2025-05-25 20:13:03,685] [WARNING] [paperless.consumer] [2025-05-25 20:13:03,119] [DEBUG] [postprocessor] Rule zzz does not match

[2025-05-25 20:13:03,686] [WARNING] [paperless.consumer] [2025-05-25 20:13:03,119] [DEBUG] [postprocessor] Rule ASN setzen Name matches

[2025-05-25 20:13:03,686] [WARNING] [paperless.consumer] [2025-05-25 20:13:03,119] [DEBUG] [postprocessor] Updating 'asn' using template {{ document_id }} and metadata {'created_date_object': datetime.date(2025, 4, 17), 'created_day': '17', 'created_date': '2025-04-17', 'created_month': '04', 'added_date': '2025-05-25', 'title': '#3271#-Volksbank-VISA-Abrechnungszeitraum:', 'added_date_object': datetime.datetime(2025, 5, 25, 20, 12, 58, 571914, tzinfo=tzoffset(None, 7200)), 'asn': None, 'created': '2025-04-17T00:00:00', 'zeitraum': '', 'created_year': '2025', 'correspondent': 'Volksbank', 'document_type': 'Finanzen - VisaCard-Voba', 'storage_path': 'Standart wie in der Grundkonfig', 'tag_list': ['Finanzen - VisaCard-Voba', 'neues Dokument'], 'added': '2025-05-25T20:12:58.571914+02:00', 'added_year': '2025', 'added_month': '05', 'added_day': '25', 'document_id': 3271}

[2025-05-25 20:13:03,686] [WARNING] [paperless.consumer] : 'None'->'3271'

[2025-05-25 20:13:03,686] [WARNING] [paperless.consumer] [2025-05-25 20:13:03,120] [DEBUG] [postprocessor] Rule ASN setzen Name matches

[2025-05-25 20:13:03,686] [WARNING] [paperless.consumer] [2025-05-25 20:13:03,120] [DEBUG] [postprocessor] Updating 'asn' using template {{ document_id }} and metadata {'created_date_object': datetime.date(2025, 4, 17), 'created_day': '17', 'created_date': '2025-04-17', 'created_month': '04', 'added_date': '2025-05-25', 'title': '#3271#-Volksbank-VISA-Abrechnungszeitraum:', 'added_date_object': datetime.datetime(2025, 5, 25, 20, 12, 58, 571914, tzinfo=tzoffset(None, 7200)), 'asn': '3271', 'created': '2025-04-17T00:00:00', 'zeitraum': '', 'created_year': '2025', 'correspondent': 'Volksbank', 'document_type': 'Finanzen - VisaCard-Voba', 'storage_path': 'Standart wie in der Grundkonfig', 'tag_list': ['Finanzen - VisaCard-Voba', 'neues Dokument'], 'added': '2025-05-25T20:12:58.571914+02:00', 'added_year': '2025', 'added_month': '05', 'added_day': '25', 'document_id': 3271}

[2025-05-25 20:13:03,687] [WARNING] [paperless.consumer] : '3271'->'3271'

[2025-05-25 20:13:03,687] [WARNING] [paperless.consumer] [2025-05-25 20:13:03,120] [DEBUG] [postprocessor] new_metadata_in_filename_format={'created_date_object': datetime.date(2025, 4, 17), 'created_day': '17', 'created_date': '2025-04-17', 'created_month': '04', 'added_date': '2025-05-25', 'title': '#3271#-Volksbank-VISA-Abrechnungszeitraum:', 'added_date_object': datetime.datetime(2025, 5, 25, 20, 12, 58, 571914, tzinfo=tzoffset(None, 7200)), 'asn': '3271', 'created': '2025-04-17T00:00:00', 'zeitraum': '', 'created_year': '2025', 'correspondent': 'Volksbank', 'document_type': 'Finanzen - VisaCard-Voba', 'storage_path': 'Standart wie in der Grundkonfig', 'tag_list': ['Finanzen - VisaCard-Voba', 'neues Dokument'], 'added': '2025-05-25T20:12:58.571914+02:00', 'added_year': '2025', 'added_month': '05', 'added_day': '25', 'document_id': 3271}

[2025-05-25 20:13:03,687] [WARNING] [paperless.consumer] [2025-05-25 20:13:03,483] [DEBUG] [paperless_api] Returning tags list from cache

[2025-05-25 20:13:03,687] [WARNING] [paperless.consumer] [2025-05-25 20:13:03,483] [DEBUG] [paperless_api] Returning tags list from cache

[2025-05-25 20:13:03,688] [WARNING] [paperless.consumer] [2025-05-25 20:13:03,484] [INFO] [postprocessor] Changes for document_id=3271:

[2025-05-25 20:13:03,688] [WARNING] [paperless.consumer] [2025-05-25 20:13:03,484] [INFO] [postprocessor] archive_serial_number: 'None' --> '3271'

[2025-05-25 20:13:03,688] [WARNING] [paperless.consumer] [2025-05-25 20:13:03,484] [INFO] [postprocessor] tags: '[294, 88]' --> '[294, 88, 326]'

[2025-05-25 20:13:03,688] [WARNING] [paperless.consumer] [2025-05-25 20:13:03,484] [INFO] [postprocessor] title: '2025-05-01 Visa Volksbank Visa Abrechnung 03-04-2025' --> '#3271#-Volksbank-VISA-Abrechnungszeitraum:'

[2025-05-25 20:13:03,689] [WARNING] [paperless.consumer] [2025-05-25 20:13:03,484] [INFO] [postprocessor] created: '2025-04-17' --> '2025-04-17T00:00:00'

[2025-05-25 20:13:03,689] [WARNING] [paperless.consumer] [2025-05-25 20:13:03,595] [INFO] [postprocessor] Validation was skipped since invalid_tag_id=None and skip_validation=False

[2025-05-25 20:13:03,689] [INFO] [paperless.consumer] Document 2025-04-17 Volksbank 2025-05-01 Visa Volksbank Visa Abrechnung 03-04-2025 consumption finished

[2025-05-25 20:13:03,695] [INFO] [paperless.tasks] ConsumeTaskPlugin completed with: Success. New document id 3271 created

it is not clear for me, wy the Titel is not changed and asn not set. The error at 2025-05-25 20:13:03,689 is not clear for me, because agg tags are o.k. (294, 88, 326). When i ask the API /api/tags in the json are all tags shown.

dirk99 avatar May 25 '25 18:05 dirk99

Docker, Paperless-ngx 2.16.1

I'm not sure if this is the same problem. But I have a similar problem. I use the plugin the first time, but as far as I can see, the ruleset is correct, but the title is not changed for the processed documents.

The goal of the rule is to find the invoicenumber and change the title to this invoice number.

This ist my ruleset: tierarztinvoicerule: match: "{{ correspondent == 'Tierärztliche Praxis am Harpener Hellweg' }}" metadata_regex: '(?P<title_new>[0-9]{5}/[0-9]{2}-[0-9]{6}BP)' metadata_postprocessing: title: '{{ title_new }}'

This is what the log says: `[2025-05-26 13:12:37,468] [DEBUG] [paperless.tasks] Skipping plugin CollatePlugin

[2025-05-26 13:12:37,474] [DEBUG] [paperless.tasks] Skipping plugin BarcodePlugin

[2025-05-26 13:12:37,474] [DEBUG] [paperless.tasks] Executing plugin WorkflowTriggerPlugin

[2025-05-26 13:12:37,482] [INFO] [paperless.tasks] WorkflowTriggerPlugin completed with:

[2025-05-26 13:12:37,482] [DEBUG] [paperless.tasks] Executing plugin ConsumeTaskPlugin

[2025-05-26 13:12:37,497] [INFO] [paperless.consumer] Consuming 27044 25-003408BP Rechnung Quittung Gutschrift (Normalpapier) Firma 1.pdf

[2025-05-26 13:12:37,500] [DEBUG] [paperless.consumer] Detected mime type: application/pdf

[2025-05-26 13:12:37,511] [DEBUG] [paperless.consumer] Parser: RasterisedDocumentParser

[2025-05-26 13:12:37,516] [DEBUG] [paperless.consumer] Parsing 27044 25-003408BP Rechnung Quittung Gutschrift (Normalpapier) Firma 1.pdf...

[2025-05-26 13:12:37,536] [INFO] [paperless.parsing.tesseract] pdftotext exited 0

[2025-05-26 13:12:37,961] [DEBUG] [paperless.parsing.tesseract] Calling OCRmyPDF with args: {'input_file': PosixPath('/tmp/paperless/paperless-ngxrmkaqhh4/27044 25-003408BP Rechnung Quittung Gutschrift (Normalpapier) Firma 1.pdf'), 'output_file': PosixPath('/tmp/paperless/paperless-c9wwa77q/archive.pdf'), 'use_threads': True, 'jobs': 4, 'language': 'deu+eng', 'output_type': 'pdfa', 'progress_bar': False, 'color_conversion_strategy': 'RGB', 'skip_text': True, 'clean': True, 'deskew': True, 'rotate_pages': True, 'rotate_pages_threshold': 12.0, 'sidecar': PosixPath('/tmp/paperless/paperless-c9wwa77q/sidecar.txt')}

[2025-05-26 13:12:38,278] [INFO] [ocrmypdf._pipeline] skipping all processing on this page

[2025-05-26 13:12:38,283] [INFO] [ocrmypdf._pipelines.ocr] Postprocessing...

[2025-05-26 13:12:38,645] [INFO] [ocrmypdf._pipeline] Image optimization ratio: 1.00 savings: 0.0%

[2025-05-26 13:12:38,646] [INFO] [ocrmypdf._pipeline] Total file size ratio: 1.06 savings: 5.8%

[2025-05-26 13:12:38,650] [INFO] [ocrmypdf._pipelines._common] Output file is a PDF/A-2B (as expected)

[2025-05-26 13:12:38,660] [DEBUG] [paperless.parsing.tesseract] Incomplete sidecar file: discarding.

[2025-05-26 13:12:38,704] [INFO] [paperless.parsing.tesseract] pdftotext exited 0

[2025-05-26 13:12:38,706] [DEBUG] [paperless.consumer] Generating thumbnail for 27044 25-003408BP Rechnung Quittung Gutschrift (Normalpapier) Firma 1.pdf...

[2025-05-26 13:12:38,712] [DEBUG] [paperless.parsing] Execute: convert -density 300 -scale 500x5000> -alpha remove -strip -auto-orient -define pdf:use-cropbox=true /tmp/paperless/paperless-c9wwa77q/archive.pdf[0] /tmp/paperless/paperless-c9wwa77q/convert.webp

[2025-05-26 13:12:39,970] [INFO] [paperless.parsing] convert exited 0

[2025-05-26 13:12:47,580] [DEBUG] [paperless.consumer] Saving record to database

[2025-05-26 13:12:47,581] [DEBUG] [paperless.consumer] Creation date from parse_date: 2025-04-11 00:00:00+00:00

[2025-05-26 13:12:49,437] [DEBUG] [paperless.consumer] Deleting file /tmp/paperless/paperless-ngxrmkaqhh4/27044 25-003408BP Rechnung Quittung Gutschrift (Normalpapier) Firma 1.pdf

[2025-05-26 13:12:49,440] [DEBUG] [paperless.parsing.tesseract] Deleting directory /tmp/paperless/paperless-c9wwa77q

[2025-05-26 13:12:49,441] [INFO] [paperless.consumer] Executing post-consume script /usr/src/paperless-ngx-postprocessor/post_consume_script.sh

[2025-05-26 13:12:55,544] [INFO] [paperless.consumer] /usr/src/paperless-ngx-postprocessor/post_consume_script.sh exited 0

[2025-05-26 13:12:55,545] [INFO] [paperless.consumer] /usr/src/paperless-ngx-postprocessor/post_consume_script.sh stderr:

[2025-05-26 13:12:55,546] [WARNING] [paperless.consumer] [2025-05-26 13:12:52,481] [DEBUG] [paperlessngx_postprocessor] Running /usr/src/paperless-ngx-postprocessor/paperlessngx_postprocessor.py with config {'auth_token': None, 'dry_run': False, 'skip_validation': False, 'backup': None, 'postprocessing_tag': None, 'invalid_tag': None, 'verbose': 'DEBUG', 'rulesets_dir': '/usr/src/paperless-ngx-postprocessor/rulesets.d', 'paperless_api_url': 'http://localhost:8000/api', 'paperless_src_dir': '/usr/src/paperless/src', 'mode': 'process', 'filename': None} and {'document_id': '23', 'correspondent': None, 'document_type': None, 'tag': None, 'storage_path': None, 'created_year': None, 'created_month': None, 'created_day': None, 'created_range': None, 'added_month': None, 'added_day': None, 'added_range': None, 'asn': None, 'title': None, 'all': False}

[2025-05-26 13:12:55,546] [WARNING] [paperless.consumer] /usr/local/lib/python3.12/site-packages/django/db/backends/utils.py:98: RuntimeWarning: Accessing the database during app initialization is discouraged. To fix this warning, avoid executing queries in AppConfig.ready() or when your app modules are imported.

[2025-05-26 13:12:55,546] [WARNING] [paperless.consumer] warnings.warn(self.APPS_NOT_READY_WARNING_MSG, category=RuntimeWarning)

[2025-05-26 13:12:55,546] [WARNING] [paperless.consumer] [2025-05-26 13:12:54,604] [DEBUG] [postprocessor] Loaded 1 rules

[2025-05-26 13:12:55,547] [WARNING] [paperless.consumer] [2025-05-26 13:12:54,730] [INFO] [paperlessngx_postprocessor] Processing 1 documents.

[2025-05-26 13:12:55,547] [WARNING] [paperless.consumer] [2025-05-26 13:12:54,889] [DEBUG] [postprocessor] metadata_in_filename_format={'document_id': 23, 'correspondent': 'Tierärztliche Praxis am Harpener Hellweg', 'document_type': 'Beleg / Quittung / Rechnung', 'storage_path': None, 'asn': None, 'tag_list': ['Hund'], 'title': '27044 25-003408BP Rechnung Quittung Gutschrift (Normalpapier) Firma 1', 'created': '2025-04-11', 'created_year': '2025', 'created_month': '04', 'created_day': '11', 'created_date': '2025-04-11', 'created_date_object': datetime.datetime(2025, 4, 11, 0, 0), 'added': '2025-05-26T13:12:47.583046Z', 'added_year': '2025', 'added_month': '05', 'added_day': '26', 'added_date': '2025-05-26', 'added_date_object': datetime.datetime(2025, 5, 26, 13, 12, 47, 583046, tzinfo=tzutc())}

[2025-05-26 13:12:55,547] [WARNING] [paperless.consumer] [2025-05-26 13:12:54,895] [DEBUG] [postprocessor] Rule tierarztinvoicerule matches

[2025-05-26 13:12:55,548] [WARNING] [paperless.consumer] [2025-05-26 13:12:54,897] [DEBUG] [postprocessor] Regex results are {'created': '2025-04-11T00:00:00', 'created_month': '04', 'created_year': '2025', 'title': '27044 25-003408BP Rechnung Quittung Gutschrift (Normalpapier) Firma 1', 'created_date_object': datetime.date(2025, 4, 11), 'added_date': '2025-05-26', 'asn': None, 'added_date_object': datetime.datetime(2025, 5, 26, 13, 12, 47, 583046, tzinfo=tzutc()), 'created_date': '2025-04-11', 'created_day': '11', 'title_new': '27044/25-003408BP'}

[2025-05-26 13:12:55,548] [WARNING] [paperless.consumer] [2025-05-26 13:12:54,898] [DEBUG] [postprocessor] Updating 'title' using template {{ title_new }} and metadata {'created': '2025-04-11T00:00:00', 'created_month': '04', 'created_year': '2025', 'title': '27044 25-003408BP Rechnung Quittung Gutschrift (Normalpapier) Firma 1', 'created_date_object': datetime.date(2025, 4, 11), 'added_date': '2025-05-26', 'asn': None, 'added_date_object': datetime.datetime(2025, 5, 26, 13, 12, 47, 583046, tzinfo=tzutc()), 'created_date': '2025-04-11', 'created_day': '11', 'title_new': '27044/25-003408BP', 'correspondent': 'Tierärztliche Praxis am Harpener Hellweg', 'document_type': 'Beleg / Quittung / Rechnung', 'storage_path': None, 'tag_list': ['Hund'], 'added': '2025-05-26T13:12:47.583046Z', 'added_year': '2025', 'added_month': '05', 'added_day': '26', 'document_id': 23}

[2025-05-26 13:12:55,548] [WARNING] [paperless.consumer] : '27044 25-003408BP Rechnung Quittung Gutschrift (Normalpapier) Firma 1'->'27044/25-003408BP'

[2025-05-26 13:12:55,548] [WARNING] [paperless.consumer] [2025-05-26 13:12:54,898] [DEBUG] [postprocessor] new_metadata_in_filename_format={'created': '2025-04-11T00:00:00', 'created_month': '04', 'created_year': '2025', 'title': '27044/25-003408BP', 'created_date_object': datetime.date(2025, 4, 11), 'added_date': '2025-05-26', 'asn': None, 'added_date_object': datetime.datetime(2025, 5, 26, 13, 12, 47, 583046, tzinfo=tzutc()), 'created_date': '2025-04-11', 'created_day': '11', 'title_new': '27044/25-003408BP', 'correspondent': 'Tierärztliche Praxis am Harpener Hellweg', 'document_type': 'Beleg / Quittung / Rechnung', 'storage_path': None, 'tag_list': ['Hund'], 'added': '2025-05-26T13:12:47.583046Z', 'added_year': '2025', 'added_month': '05', 'added_day': '26', 'document_id': 23}

[2025-05-26 13:12:55,549] [WARNING] [paperless.consumer] [2025-05-26 13:12:55,138] [INFO] [postprocessor] Changes for document_id=23:

[2025-05-26 13:12:55,549] [WARNING] [paperless.consumer] [2025-05-26 13:12:55,138] [INFO] [postprocessor] title: '27044 25-003408BP Rechnung Quittung Gutschrift (Normalpapier) Firma 1' --> '27044/25-003408BP'

[2025-05-26 13:12:55,549] [WARNING] [paperless.consumer] [2025-05-26 13:12:55,138] [INFO] [postprocessor] created: '2025-04-11' --> '2025-04-11T00:00:00'

[2025-05-26 13:12:55,550] [WARNING] [paperless.consumer] [2025-05-26 13:12:55,226] [INFO] [postprocessor] Validation was skipped since invalid_tag_id=None and skip_validation=False

[2025-05-26 13:12:55,550] [INFO] [paperless.consumer] Document 2025-04-11 Tierärztliche Praxis am Harpener Hellweg 27044 25-003408BP Rechnung Quittung Gutschrift (Normalpapier) Firma 1 consumption finished

[2025-05-26 13:12:55,561] [INFO] [paperless.tasks] ConsumeTaskPlugin completed with: Success. New document id 23 created`

SurfaceScratcher avatar May 26 '25 13:05 SurfaceScratcher

I have the same problem. From what I can see in the logs, the rule matches and is applied, but somehow the values are not changed in paperless:

[2025-05-31 15:59:28,142] [WARNING] [paperless.consumer] [2025-05-31 15:59:27,952] [INFO] [postprocessor] Changes for document_id=6912:

[2025-05-31 15:59:28,142] [WARNING] [paperless.consumer] [2025-05-31 15:59:27,952] [INFO] [postprocessor]  title: 'myname_1_TEST01_052025' --> 'Verdienstabrechnung Mai 2025'

[2025-05-31 15:59:28,142] [WARNING] [paperless.consumer] [2025-05-31 15:59:27,952] [INFO] [postprocessor]  created: '2025-05-01' --> '2025-05-21T00:00:00'

[2025-05-31 15:59:28,143] [WARNING] [paperless.consumer] [2025-05-31 15:59:27,952] [INFO] [postprocessor]  created_date: '2025-05-01' --> '2025-05-21'

[2025-05-31 15:59:28,143] [WARNING] [paperless.consumer] [2025-05-31 15:59:28,008] [INFO] [postprocessor] Validation was skipped since invalid_tag_id=None and skip_validation=False

[2025-05-31 15:59:28,143] [INFO] [paperless.consumer] Document 2025-05-01  myname_1_TEST01_052025 consumption finished

The changes mentioned are what is expected. Somehow these changes are not applied. The document name uses the old values here and in paperless the old values are used as well. DEBUG messages are only before this and show no errors or problems.

DonkeeeyKong avatar May 31 '25 14:05 DonkeeeyKong

Hi there, I got the same on 2.16.2. It was working great on 2.15.x. I can find the following lines in the debug log:

[WARNING] [paperless.consumer] /usr/local/lib/python3.12/site-packages/django/db/backends/utils.py:98: RuntimeWarning: Accessing the database during app initialization is discouraged. To fix this warning, avoid executing queries in AppConfig.ready() or when your app modules are imported.

[2025-06-10 21:36:50,060] [WARNING] [paperless.consumer]   warnings.warn(self.APPS_NOT_READY_WARNING_MSG, category=RuntimeWarning)
...
[INFO] [paperlessngx_postprocessor] Processing 1 documents.
...
[WARNING] [paperless.consumer] : '2025-06-10_709004_162150_R-0254146'->'
[WARNING] [paperless.consumer]   Rechnung 2025-06
[WARNING] [paperless.consumer] '
...
[WARNING] [paperless.consumer] [2025-06-10 21:36:49,671] [INFO] [postprocessor]  created: '2025-06-10' --> '2025-06-10T00:00:00'
...
[WARNING] [paperless.consumer] [2025-06-10 21:36:49,837] [INFO] [postprocessor] document_id=2019 is valid
...
[INFO] [paperless.tasks] ConsumeTaskPlugin completed with: Success. New document id 2019 created

But no changes are applied to the document.

Not sure if the inital warning Accessing the database during app initialization is discouraged. is related, but might say that the script cannot save the changes to database and thus no change is applied to the processed doc.

Limerick-gh avatar Jun 10 '25 20:06 Limerick-gh

Just a heads up that this seems due to a breaking change in paperless-ngx 2.16.0, which changed the format of the created date.

Even rulesets that don't try to change the created date would be affected, since the postprocessor would read the date from the paperless-ngx REST API in the new format, but then try to patch it using the old format, but the patch would then be rejected.

A fix is incoming within a few days--possibly mid-next week. Thanks to everyone for your patience. I had no idea more than a handful of people used the postprocessor!

jgillula avatar Jun 12 '25 23:06 jgillula

A fix is incoming--thanks to everyone for your patience. I had no idea more than a handful of people used the postprocessor!

It's a great tool. Thank you!

DonkeeeyKong avatar Jun 13 '25 00:06 DonkeeeyKong

This has been fixed in v2.0.1. For now, we just ask paperless-ngx to use an older version of the REST API; eventually I'll need to change things so they only work for paperless-ngx v2.16+ (and use the latest version of the REST API), but that would be a breaking change and I'm not ready to introduce that yet.

If you find any new problems with the new version, please open a new bug. Thanks!

jgillula avatar Jun 15 '25 20:06 jgillula

I can confirm that it's working again. 👍 Thank you so much! Great add-on to pngx.

Limerick-gh avatar Jun 16 '25 09:06 Limerick-gh