[BUG] my system.log fills up with the same error message from the module Impossible to process the erasure: One or more input exceptions have occurred

[ioweb-gr]

Description

I'm getting constantly the folllowing error

[2024-03-22T09:51:15.802108+00:00] main.ERROR: Impossible to process the erasure: One or more input exceptions have occurred. [{"file":"/var/www/vhosts/domain.gr/httpdocs/vendor/opengento/module-gdpr/Cron/EraseEntity.php","line":71,"function":"process","class":"Opengento\\Gdpr\\Model\\EraseEntityManagement","type":"->"},{"function":"execute","class":"Opengento\\Gdpr\\Cron\\EraseEntity","type":"->"},{"file":"/var/www/vhosts/domain.gr/httpdocs/vendor/magento/module-cron/Observer/ProcessCronQueueObserver.php","line":368,"function":"call_user_func_array"},{"file":"/var/www/vhosts/domain.gr/httpdocs/vendor/magento/module-cron/Observer/ProcessCronQueueObserver.php","line":879,"function":"_runJob","class":"Magento\\Cron\\Observer\\ProcessCronQueueObserver","type":"->"},{"file":"/var/www/vhosts/domain.gr/httpdocs/vendor/magento/module-cron/Observer/ProcessCronQueueObserver.php","line":840,"function":"tryRunJob","class":"Magento\\Cron\\Observer\\ProcessCronQueueObserver","type":"->"},{"file":"/var/www/vhosts/domain.gr/httpdocs/vendor/magento/module-cron/Observer/ProcessCronQueueObserver.php","line":280,"function":"processPendingJobs","class":"Magento\\Cron\\Observer\\ProcessCronQueueObserver","type":"->"},{"file":"/var/www/vhosts/domain.gr/httpdocs/vendor/magento/framework/Event/Invoker/InvokerDefault.php","line":88,"function":"execute","class":"Magento\\Cron\\Observer\\ProcessCronQueueObserver","type":"->"},{"file":"/var/www/vhosts/domain.gr/httpdocs/vendor/magento/framework/Event/Invoker/InvokerDefault.php","line":74,"function":"_callObserverMethod","class":"Magento\\Framework\\Event\\Invoker\\InvokerDefault","type":"->"},{"file":"/var/www/vhosts/domain.gr/httpdocs/vendor/magento/framework/Event/Manager.php","line":65,"function":"dispatch","class":"Magento\\Framework\\Event\\Invoker\\InvokerDefault","type":"->"},{"file":"/var/www/vhosts/domain.gr/httpdocs/generated/code/Magento/Framework/Event/Manager/Proxy.php","line":95,"function":"dispatch","class":"Magento\\Framework\\Event\\Manager","type":"->"},{"file":"/var/www/vhosts/domain.gr/httpdocs/vendor/magento/framework/App/Cron.php","line":86,"function":"dispatch","class":"Magento\\Framework\\Event\\Manager\\Proxy","type":"->"},{"file":"/var/www/vhosts/domain.gr/httpdocs/vendor/magento/module-cron/Console/Command/CronCommand.php","line":126,"function":"launch","class":"Magento\\Framework\\App\\Cron","type":"->"},{"file":"/var/www/vhosts/domain.gr/httpdocs/vendor/symfony/console/Command/Command.php","line":298,"function":"execute","class":"Magento\\Cron\\Console\\Command\\CronCommand","type":"->"},{"file":"/var/www/vhosts/domain.gr/httpdocs/vendor/magento/framework/Interception/Interceptor.php","line":58,"function":"run","class":"Symfony\\Component\\Console\\Command\\Command","type":"->"},{"file":"/var/www/vhosts/domain.gr/httpdocs/vendor/magento/framework/Interception/Interceptor.php","line":138,"function":"___callParent","class":"Magento\\Cron\\Console\\Command\\CronCommand\\Interceptor","type":"->"},{"file":"/var/www/vhosts/domain.gr/httpdocs/vendor/magento/framework/Interception/Interceptor.php","line":153,"function":"Magento\\Framework\\Interception\\{closure}","class":"Magento\\Cron\\Console\\Command\\CronCommand\\Interceptor","type":"->"},{"file":"/var/www/vhosts/domain.gr/httpdocs/generated/code/Magento/Cron/Console/Command/CronCommand/Interceptor.php","line":23,"function":"___callPlugins","class":"Magento\\Cron\\Console\\Command\\CronCommand\\Interceptor","type":"->"},{"file":"/var/www/vhosts/domain.gr/httpdocs/vendor/symfony/console/Application.php","line":1040,"function":"run","class":"Magento\\Cron\\Console\\Command\\CronCommand\\Interceptor","type":"->"},{"file":"/var/www/vhosts/domain.gr/httpdocs/vendor/symfony/console/Application.php","line":301,"function":"doRunCommand","class":"Symfony\\Component\\Console\\Application","type":"->"},{"file":"/var/www/vhosts/domain.gr/httpdocs/vendor/magento/framework/Console/Cli.php","line":116,"function":"doRun","class":"Symfony\\Component\\Console\\Application","type":"->"},{"file":"/var/www/vhosts/domain.gr/httpdocs/vendor/symfony/console/Application.php","line":171,"function":"doRun","class":"Magento\\Framework\\Console\\Cli","type":"->"},{"file":"/var/www/vhosts/domain.gr/httpdocs/bin/magento","line":23,"function":"run","class":"Symfony\\Component\\Console\\Application","type":"->"}] []

Prerequisites

PHP Version:

• 8.1

Magento Version:

• 2.4.6-p4

Module Version:

• 4.4.2

Desktop (if applicable):

• OS: Ubuntu 22.04
• Browser: N/A
• Version: N/A

Smartphone (if applicable):

• Device: [e.g. iPhone6]
• OS: [e.g. iOS8.1]
• Browser [e.g. stock browser, safari]
• Version [e.g. 22]

Issue Details

Steps to reproduce the behavior I constantly see this error spamming without any special action item occurring. It just spams there

Expected behavior
The error is gone

Screenshots
If applicable, add screenshots to help explain your problem.

Additional context
It turns out my log will fill up into hundreds of gigabytes in days.

[ioweb-gr]

I've been trying to debug this and it seems the cron job that's running to process erasures, had to go through a list of 981 customers. There are a couple of things I notice

1. There's no date to indicate when the deletion was requested. These requests should be logged.
2. It seems in some cases it's trying to anonymize the user, but it cannot process the address properly as it's not validating extra address attributes which are required.
3. It will repeat the process constantly without notifying the admin about the error and will never be able to complete.
4. It's highly unlikely that half our customers requested deletion. We cannot process deletion requests without logging the actual submission of the request.
5. There's an abnormally high count of rows in the table for 1641 unique customers and 1400 unique orders

The actual error in most of the cases for me is this one

• document_type is a required field.
• "regionId" is required. Insert and try again

Any ideas how to proceed with this?

I'm going to clear up the table just in case and start fresh, but I really need to see how to log every request for deletion in a manageable way.

[ioweb-gr]

Hi,

any news on what might be causing it and how I can mitigate this issue?

[thomas-kl1]

Hello, yes it seems it's related to customer validation that is not disabled in this case. $customer->set data('ignore_validation_flag') seems to be missing. Btw the module is going to be reworked in order to remove some abstraction and simplicity some cases.

[ioweb-gr]

I see. Also any ideas why half of our customers appeared in the deletion list? I doubt it half of them requested deletion.

Is it possible the module is being exploited somehow?

[thomas-kl1]

There's a config to remove automatically users after being idle for a certain amount of time. You can check their last updated date and see if it match.

[ioweb-gr]

Oh I see. Is the config enabled by default?

[thomas-kl1]

If you have enabled erasure, yes.

See https://github.com/opengento/magento2-gdpr/blob/5618a8c86efb22ac99b4c8fd66a7b54b36f0e075/etc/config.xml#L126

Value is in days GDPR requires data to be erased when user is idle.

[ioweb-gr]

I see it's impossible to disable the feature or set a virtually unlimited entity_max_age. We basically want our customers to be able to request deletion, but not automatically delete them just because they're old customers.

[thomas-kl1]

Is fixed in 5.0.0-beta1