gdpr-dump
gdpr-dump copied to clipboard
Smile-SA
cache_key feature does not working as expected
The cache_key feature does not working as expected. It only works for my last command and the last customer.
The emails for the other customers does not match with their respective order. Using phar file provided here : https://github.com/Smile-SA/gdpr-dump/releases/download/4.2.1/gdpr-dump.phar
GdprDump Version: 4.2.1 PHP Version: 8.2
Database Version: MySQL 8.0.33
Here is the config
tables:
customer_entity:
converters:
email:
cache_key: 'customer_email'
unique: true
sales_order:
limit: 30
order_by: 'entity_id desc'
skip_conversion_if: '{{customer_id}} === null'
converters:
customer_email:
cache_key: 'customer_email'
unique: true
Expected result
All order customer email should match with their anonymize customer email in customer_entity table.
Actual result
It works only for the last order. My last order customer email match well with the customer email in the customer_entity table
we had the same oberservation on our end. it did not work, but I did not have time to dig into it.
Needs more information, I can't reproduce.
With this config:
---
extends: 'magento2'
version: '2.4.6-p4'
database:
# ...
tables_whitelist:
- customer_entity
- quote
tables:
customer_entity:
converters:
email:
cache_key: 'customer_email'
unique: true
quote:
converters:
customer_email:
cache_key: 'customer_email'
unique: true
The emails do match:
~$ make dump c=build/magento.yaml | grep example.
[+] Creating 1/0
✔ Container gdpr-dump-db-1 Running0.0s
INSERT INTO `customer_entity` VALUES (1,1,'[email protected]',1,NULL,1,'2018-06-18 09:11:09','2018-06-18 09:11:09',1,0,'Default Store View',NULL,'kyrzclah',NULL,'ajelkhve',NULL,'1973-09-02','nyc10wxyfdsvxp8yvgmev90rijvviq2xb0ulayfk1cji9zpicrcwtiw9bjlwgq8lazy51jk2n8v0r9mmxyk91kttx5dtnvxl5x9','bfd44a85e7ab71923b6a91ad7f30baea','2018-06-18 09:11:09',1,1,NULL,NULL,2,0,NULL,NULL),(4,1,'[email protected]',1,NULL,1,'2018-08-23 08:04:30','2018-08-23 08:04:30',1,0,'Default Store View',NULL,'k8gc',NULL,'7vf',NULL,NULL,'42u2bk2to4wob9p4e7ihj3n3rjgzyabb228bd51xc4prlxs0lazg53nsf2d7a5flz4egqkmamlxc1948yeok40oo88z81vf5lox','14cbfe4abedb6cdc8b9139029c939792','2018-08-23 08:04:30',NULL,NULL,NULL,NULL,NULL,0,NULL,NULL),(6,1,'[email protected]',1,NULL,1,'2018-08-23 08:08:29','2018-08-23 09:16:18',1,0,'Default Store View',NULL,'4uxh',NULL,'khf',NULL,NULL,'91n4ki6lo003hdl8n93n4sb6o237wujp6ux0bf8bvx9iycuwfeolnobez6fsul5whqm84y4tptdqunzvxsepn34yv7vu6sj5iv4',NULL,NULL,NULL,NULL,NULL,NULL,1,0,NULL,NULL),(7,1,'[email protected]',1,NULL,1,'2018-08-23 08:33:14','2018-08-23 08:37:30',1,0,'Default Store View',NULL,'7l3x',NULL,'wfa',NULL,NULL,'mkjh0vi6viupqqic4ige53da3y9787840rn5teazwogmsoziovkza8642p6t96w5nbc5m5nk3ri1sxxt38zdw6i0imry8xx6f0d',NULL,NULL,NULL,NULL,NULL,NULL,NULL,0,NULL,NULL),(8,1,'[email protected]',1,NULL,1,'2018-08-23 08:34:40','2018-08-23 08:54:44',1,0,'Default Store View',NULL,'raria2m',NULL,'d0ye0s7aohv',NULL,NULL,'hgq8grpqchg12r932z49k7pnruniyeorvw2ehkbcppdzqjx1kvw5zkc0ma0mvd23ioqu8t6c8e3rlb1myyd2fcv7302zio53wde',NULL,NULL,NULL,NULL,NULL,NULL,NULL,0,NULL,NULL),(9,1,'[email protected]',1,NULL,1,'2018-08-23 12:27:31','2018-08-23 12:27:39',1,0,'Default Store View',NULL,'vznc',NULL,'fqq',NULL,NULL,NULL,'900bd7ce50d5e0789411fcf6392522d0','2018-08-23 12:27:31',NULL,NULL,NULL,NULL,1,0,NULL,NULL);
INSERT INTO `quote` VALUES (1,1,'2018-06-18 09:14:01','2018-06-18 09:14:03',NULL,0,0,0,1,1.0000,0,0.0000,0.0000,'EUR','EUR','EUR',34.0000,34.0000,NULL,1,3,1,'[email protected]',NULL,'qj7ig7pq',NULL,'m0gooque',NULL,'1973-11-08 00:00:00',NULL,1,0,NULL,NULL,'000000001',NULL,NULL,'EUR',1.0000,1.0000,NULL,'2',29.0000,29.0000,29.0000,29.0000,1,0,NULL,0,0.0000,0.0000,NULL,NULL,0.0000,0.0000,0.0000,0.0000,NULL,NULL,NULL,NULL,0.0000,0.0000,0.0000,0.0000,0.0000,0.0000,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,0,0.0000,0.0000),(2,1,'2018-06-18 09:14:03','2018-06-18 09:14:03',NULL,0,0,0,1,1.0000,0,0.0000,0.0000,'EUR','EUR','EUR',37.0000,37.0000,NULL,1,3,1,'[email protected]',NULL,'7h2v1v7y',NULL,'dgf1xoeg',NULL,'1973-04-04 00:00:00',NULL,1,0,NULL,NULL,'000000002',NULL,NULL,'EUR',1.0000,1.0000,NULL,'2',32.0000,32.0000,32.0000,32.0000,1,0,NULL,0,0.0000,0.0000,NULL,NULL,0.0000,0.0000,0.0000,0.0000,NULL,NULL,NULL,NULL,0.0000,0.0000,0.0000,0.0000,0.0000,0.0000,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,0,0.0000,0.0000),(3,1,'2018-06-18 12:28:54','2018-06-18 12:33:12',NULL,1,0,1,1,1.0000,0,0.0000,0.0000,'EUR','EUR','EUR',39.6400,39.6400,NULL,1,3,1,'[email protected]',NULL,'9qsrd3mi',NULL,'3uisijm2',NULL,'1973-11-11 00:00:00',NULL,1,0,'146.71.11.73',NULL,NULL,NULL,NULL,'EUR',1.0000,1.0000,NULL,'2',32.0000,32.0000,32.0000,32.0000,1,0,NULL,0,0.0000,0.0000,NULL,NULL,0.0000,0.0000,0.0000,0.0000,NULL,NULL,NULL,NULL,0.0000,0.0000,0.0000,0.0000,0.0000,0.0000,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,0,0.0000,0.0000),(4,1,'2018-08-23 08:04:34','2018-10-18 08:01:28',NULL,1,0,0,1,1.0000,0,0.0000,0.0000,'EUR','EUR','EUR',7.0000,7.0000,NULL,4,3,1,'[email protected]',NULL,'lpe3',NULL,'s7u',NULL,NULL,NULL,1,0,'172.252.206.90',NULL,NULL,NULL,NULL,'EUR',1.0000,1.0000,NULL,NULL,7.0000,7.0000,7.0000,7.0000,1,0,NULL,0,0.0000,0.0000,NULL,NULL,0.0000,0.0000,0.0000,0.0000,NULL,NULL,NULL,NULL,0.0000,0.0000,0.0000,0.0000,0.0000,0.0000,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,0,0.0000,0.0000),(5,1,'2018-08-23 08:31:26','2018-08-23 08:31:26',NULL,1,0,0,0,0.0000,0,0.0000,0.0000,'EUR','EUR','EUR',0.0000,0.0000,NULL,6,NULL,1,'[email protected]',NULL,'dgzg',NULL,'17m',NULL,NULL,NULL,1,0,'53.197.54.97',NULL,NULL,NULL,NULL,'EUR',1.0000,1.0000,NULL,'1',0.0000,0.0000,0.0000,0.0000,1,0,NULL,0,0.0000,0.0000,NULL,NULL,0.0000,0.0000,0.0000,0.0000,NULL,NULL,NULL,NULL,0.0000,0.0000,0.0000,0.0000,0.0000,0.0000,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,0,0.0000,0.0000),(6,1,'2018-08-23 08:49:48','2018-08-23 09:26:18',NULL,0,0,0,1,1.0000,0,0.0000,0.0000,'EUR','EUR','EUR',33.9000,33.9000,NULL,7,3,1,'[email protected]',NULL,'zy5t',NULL,'ebm',NULL,NULL,NULL,1,0,'82.148.219.187',NULL,NULL,NULL,NULL,'EUR',1.0000,1.0000,NULL,NULL,28.9000,28.9000,28.9000,28.9000,1,0,NULL,0,0.0000,0.0000,NULL,NULL,0.0000,0.0000,0.0000,0.0000,NULL,NULL,NULL,NULL,0.0000,0.0000,0.0000,0.0000,0.0000,0.0000,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,0,0.0000,0.0000),(7,1,'2018-08-23 08:53:16','2018-08-23 08:53:16',NULL,1,0,0,0,0.0000,0,0.0000,0.0000,'EUR','EUR','EUR',0.0000,0.0000,NULL,8,NULL,1,'[email protected]',NULL,'0w3y1a1',NULL,'c9rfnxwv0pk',NULL,NULL,NULL,1,0,'130.90.148.64',NULL,NULL,NULL,NULL,'EUR',1.0000,1.0000,NULL,NULL,0.0000,0.0000,0.0000,0.0000,1,0,NULL,0,0.0000,0.0000,NULL,NULL,0.0000,0.0000,0.0000,0.0000,NULL,NULL,NULL,NULL,0.0000,0.0000,0.0000,0.0000,0.0000,0.0000,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,0,0.0000,0.0000),(8,1,'2018-08-23 08:59:45','2018-08-23 08:59:45',NULL,1,0,0,0,0.0000,0,0.0000,0.0000,'EUR','EUR','EUR',0.0000,0.0000,NULL,7,NULL,1,'[email protected]',NULL,'6q5g',NULL,'pwl',NULL,NULL,NULL,1,0,'234.44.17.40',NULL,NULL,NULL,NULL,'EUR',1.0000,1.0000,NULL,NULL,0.0000,0.0000,0.0000,0.0000,1,0,NULL,0,0.0000,0.0000,NULL,NULL,0.0000,0.0000,0.0000,0.0000,NULL,NULL,NULL,NULL,0.0000,0.0000,0.0000,0.0000,0.0000,0.0000,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,0,0.0000,0.0000);
Hi @guvra
After some test on my end, I observe that the cache_key feature works as expected on a Magento 2.4.6-p3 Enterprise Edition but it fails on Magento 2.4.6-p4 Community Edition
On the 2.4.6-p4 CE, as I said previously, the cache_key feature works only for the last order (the most recent).
Could you retry after replacing isset with array_key_exists here?
https://github.com/Smile-SA/gdpr-dump/blob/main/src/Converter/Proxy/Cache.php#L36
This is the only potential bug I can think of.
Hi @guvra , Thanks! We found on our end that there are some data on our DB which are inconsistent. We are going to run some tests and if needed we will close this issue.