[PW-6782] Special characters in card holder name cannot be unserialise in Magento from Adyen response

[GRajwani]

Making your own contribution is easy, encouraged and greatly appreciated! We will put effort into reviewing and merging your PR quickly. For more info, please refer to the contribution guidelines: https://github.com/Adyen/adyen-magento2/blob/develop/CONTRIBUTING.md

Describe the bug A payment is issued in adyen using a credit card with a card holder name that contains special characters for e.g. Erica 𝓼𝓲𝓸𝓻𝓹𝓪𝓮𝓼

To Reproduce Steps to reproduce the behavior:

1. Use a card that has a cardholder name containing special characters

Expected behavior I would expect it would be able to deserialise the name

Magento version [e.g. 2.4.2]

Plugin version [e.g. 7.3.7]

Screenshots

Desktop (please complete the following information):

• Chrome Browser

Additional context In Magento, the card holder name cannot be deserialise

AdyenLoggerTest.ADYEN_NOTIFICATION_CRONJOB: Notification 62719 had an error: Unable to unserialize value, string is corrupted. #0 [internal function]: Magento\Framework\Serialize\Serializer\Serialize->Magento\Framework\Serialize\Serializer{closure}(8, 'unserialize(): ...', '/var/www/html/v...', 42, Array) #1 /var/www/html/vendor/magento/framework/Serialize/Serializer/Serialize.php(42): unserialize('a:70:{s:32:"fra...', Array)

[tnaber]

Hi @GRajwani,

Thank you for opening this issue. I will be looking into this. In order to determine the priority, is this something that occurs often?

Cheers, Titus

[GRajwani]

Hi @tnaber , No it does not happen often but we have encounter this issue whenever the card holder name has special characters means 3-4 times and in our system we notify a different ERP system once order becomes in received state but in these occasions they are not notified since the order remains in Payment Review state which cause issues for me.

Thanks, Girdar

[GRajwani]

Hi @tnaber , Any fixes for this issue because I again got the same issue or any temporary fix which I can perform to cater this. Please let me know.

Thanks, Girdar

[Morerice]

Hi @GRajwani,

The fix has been released on v7.3.10. Feel free to re-open this issue if it persists.

Thanks, Jean Adyen

[Morerice]

Hi @GRajwani,

Please ignore my latest message. I incorrectly thought that the linked PR was a fix for this issue but this is not the case. However, when investigating further it seems that our module is calling the unserialize function that is provided to us by the magento platform. Hence, the function on magento side is having issues when trying to unserialize such value.

Does your fix also include a call to this function?

Thanks, Jean Adyen

[GRajwani]

Hi, Well i have no fix for the moment. But yes it calls the unserialize function in Magento which then produce an error. I believe your fix will treat this special characters to normal text so that during unserialize it does not product any issue.

Thanks, Girdar

[RokPopov]

Hey @GRajwani ,

We have reopened this issue and are looking into it. However, using the latest v7.3.x of our plugin -> v7.3.11, Magento platform v2.4.3 and PHP v7.4.30, we are not able to replicate the issue. I am sharing with you the stack trace from the var/log/adyen/cronjob.log of processing the notification of the payment made with the usage of special characters where you can see the notification being processed correctly:

[2022-07-22 14:18:01] AdyenLoggerTest.ADYEN_NOTIFICATION_CRONJOB: Prepare invoice for order [] []
[2022-07-22 14:18:01] AdyenLoggerTest.ADYEN_NOTIFICATION_CRONJOB: Creating invoice for order [] []
[2022-07-22 14:18:01] AdyenLoggerTest.ADYEN_NOTIFICATION_CRONJOB: Capture mode is set to auto capture [] []
[2022-07-22 14:18:02] AdyenLoggerTest.ADYEN_NOTIFICATION_CRONJOB: Notification 1 created an invoice. {"invoiceId":"3","invoiceIncrementId":"000000003","invoiceState":2,"invoiceStateName":"Paid","invoiceWasPayCalled":true,"invoiceCanCapture":false,"invoiceCanCancel":false,"invoiceCanVoid":true,"invoiceCanRefund":true} []
[2022-07-22 14:18:02] AdyenLoggerTest.ADYEN_NOTIFICATION_CRONJOB: Set order to authorised [] []
[2022-07-22 14:18:02] AdyenLoggerTest.ADYEN_NOTIFICATION_CRONJOB: Notification w/amount 5000 has completed the capturing of order 000000003 w/amount 5000 [] []
[2022-07-22 14:18:02] AdyenLoggerTest.ADYEN_NOTIFICATION_CRONJOB: Order 000000003 was finalized. Authorised status not set [] []
[2022-07-22 14:18:02] AdyenLoggerTest.ADYEN_NOTIFICATION_CRONJOB: Send orderconfirmation email to shopper [] []
[2022-07-22 14:18:02] AdyenLoggerTest.ADYEN_NOTIFICATION_CRONJOB: Notification 1 was processed {"orderId":"3","orderIncrementId":"000000003","orderState":"processing","orderStatus":"processing"} []
[2022-07-22 14:18:02] AdyenLoggerTest.ADYEN_NOTIFICATION_CRONJOB: Cronjob updated 1 notification(s) [] []

Could you please clarify which version of PHP are you using on your server?

Thank you in advance.

Kind regards, Rok, Adyen

[GRajwani]

Hi, I am currently using php version 7.3.26 and adyen version 7.3.7.

[2022-05-20 06:46:05] AdyenLoggerTest.ADYEN_NOTIFICATION_CRONJOB: {"entity_id":"62719","pspreference":"WS6CS6CZH37DPK52","merchant_reference":"DODIT00038144","event_code":"AUTHORISATION","success":"true","payment_method":"mc","amount_value":"38200","amount_currency":"EUR","reason":"150487:3050:12\/2024","live":"true","additional_data":"a:70:{s:32:\"fraudCheck-25-CVCAuthResultCheck\";s:1:\"0\";s:48:\"fraudCheck-75-PaymentDetailCarteBancaireRefCheck\";s:1:\"0\";s:33:\"fraudCheck-29-LiabilityShiftCheck\";s:1:\"0\";s:36:\"fraudCheck-63-TransactionAmountCheck\";s:1:\"0\";s:30:\"fraudCheck-8-ShopperEmailUsage\";s:1:\"0\";s:39:\"fraudCheck-64-TransactionAmountVelocity\";s:1:\"0\";s:30:\"fraudCheck-6-ShopperIpRefCheck\";s:1:\"0\";s:49:\"fraudCheck-52-DistinctPaymentDetailUsageByShopper\";s:1:\"0\";s:43:\"fraudCheck-57-BillingAddressDeliveryAddress\";s:1:\"0\";s:27:\"fraudCheck-2-CardChunkUsage\";s:1:\"0\";s:22:\"billingAddress.country\";s:2:\"IT\";s:11:\"fraudOffset\";s:1:\"0\";s:55:\"fraudCheck-54-DistinctSharedPaymentDetailUsageByShopper\";s:1:\"0\";s:30:\"fraudCheck-77-EmailSyntaxCheck\";s:1:\"0\";s:17:\"fraudManualReview\";s:5:\"false\";s:31:\"fraudCheck-3-PaymentDetailUsage\";s:1:\"0\";s:12:\"shopperEmail\";s:25:\"[email protected]\";s:27:\"fraudCheck-7-ShopperIpUsage\";s:2:\"10\";s:16:\"shopperTelephone\";s:10:\"3331579158\";s:43:\"fraudCheck-46-DistinctCountryUsageByShopper\";s:1:\"0\";s:51:\"fraudCheck-53-DistinctSharedShopperIpUsageByShopper\";s:1:\"0\";s:13:\"fundingSource\";s:7:\"PREPAID\";s:16:\"shopperReference\";s:49:\"DODIT00038144dc4718bb-5c2b-480d-857b-9714e7ecd124\";s:40:\"fraudCheck-55-ShopperAuthorisedFrequency\";s:1:\"0\";s:48:\"fraudCheck-50-DistinctShopperEmailUsageByShopper\";s:1:\"0\";s:38:\"fraudCheck-48-ChargebackCountByShopper\";s:1:\"0\";s:9:\"avsResult\";s:17:\"3 AVS unavailable\";s:28:\"fraudCheck-74-EmailNameCheck\";s:1:\"0\";s:43:\"fraudCheck-41-PaymentDetailNonFraudRefCheck\";s:1:\"0\";s:35:\"fraudCheck-56-ShopperReferenceCheck\";s:1:\"0\";s:9:\"shopperIP\";s:12:\"172.30.31.25\";s:15:\"totalFraudScore\";s:2:\"20\";s:31:\"fraudCheck-79-BinAttackRefCheck\";s:1:\"0\";s:10:\"expiryDate\";s:7:\"12\/2024\";s:33:\"fraudCheck-65-EmailDomainRefCheck\";s:1:\"0\";s:11:\"shopperName\";s:15:\"Erica Siorpaes \";s:21:\"billingAddress.street\";s:22:\"Localit\u00e0 Brite de Val\";s:36:\"fraudCheck-15-IssuingCountryReferral\";s:1:\"0\";s:33:\"fraudCheck-37-ShopperDetailsUsage\";s:2:\"10\";s:34:\"fraudCheck-26-ShopperEmailRefCheck\";s:1:\"0\";s:40:\"fraudCheck-47-BlockedCardsUsageByShopper\";s:1:\"0\";s:7:\"cardBin\";s:6:\"533862\";s:52:\"fraudCheck-51-DistinctShopperReferenceUsageByShopper\";s:1:\"0\";s:28:\"fraudCheck-4-HolderNameUsage\";s:1:\"0\";s:17:\"acquirerReference\";s:12:\"214006104525\";s:28:\"fraudCheck-13-IssuerRefCheck\";s:1:\"0\";s:38:\"fraudCheck-10-HolderNameContainsNumber\";s:1:\"0\";s:45:\"fraudCheck-70-ShopperConsecutiveRefusalsCheck\";s:1:\"0\";s:18:\"cardIssuingCountry\";s:2:\"IT\";s:36:\"fraudCheck-69-AutomatedReferralCheck\";s:1:\"0\";s:6:\"150487\";s:15:\"fraudResultType\";s:5:\"GREEN\";s:14:\"cardHolderName\";s:38:\"Erica ","done":"0","processing":true,"error_count":"4","error_message":"[5\/20\/22, 8:42 AM]: Unable to unserialize value, string is corrupted.

The issue is when the cardholdername has special characters which php whatever be the version will not be able to de-serialize it. Before the adyen returns the response in json, it should remove the special characters so in magent ocode php will be able to deserialize it.

Your example does not mentioned you used special characters in the cardholdername. You just mentioned the result of the notification cron job but for which kind of cardholdername used is not specified.

I think its now easier for you to reproduce the issue at your side and make sure the response where the cardholdername containing special characters from the example I provided, it does not give "error_message":"[5/20/22, 8:42 AM]: Unable to unserialize value, string is corrupted.

Thanks, Girdar

[RokPopov]

Hi @GRajwani,

Thank you for your reply and the provided details.

The steps I took for replicating this are the following:

1. Select a product and put it in the shopping cart
2. Proceed to the page where the shopper details need to be filled in and use the string 𝓼𝓲𝓸𝓻𝓹𝓪𝓮𝓼 as the last name
3. Proceed to the checkout page and again use the string with special characters in the card component's last name input field
4. Proceed with the payment and process the notification that comes to Magento.
5. The notification gets processed correctly and an error that is thrown in your case is not thrown

However, checking the database, I can see that the adyen_notification table doesn't contain the special characters that were used for the last name. It stores the last name as a text in the form of question marks => ?????????????????? Therefore, I tried to manually change the entry in the database, however, the value cannot be saved. The issue that I am getting in my IDE is the following:

[22001][1366] Data truncation: Incorrect string value: '\xF0\x9D\x93\xBC\xF0\x9D...' for column `magento`.`adyen_notification`.`additional_data` at row 1

It seems like your database is using a different collation that our is? We are using Magento's default way to unserialize the value. Would you happen to be aware of a better way to "normalize" a string before unserializing it? That way we could maybe explore a new approach for these cases.

Looking forward to your reply,

Kind regards, Rok, Adyen

[GRajwani]

Hi, For this order reference, I can see in the adyen_notifcation table, the error_message column has value

[5/24/22, 4:24 PM]: Unable to unserialize value, string is corrupted. [5/24/22, 4:25 PM]: Unable to unserialize value, string is corrupted. [5/24/22, 4:26 PM]: Unable to unserialize value, string is corrupted. [5/24/22, 4:27 PM]: Unable to unserialize value, string is corrupted. [5/24/22, 4:28 PM]: Unable to unserialize value, string is corrupted.

and additional_data column a:70:{s:32:"fraudCheck-25-CVCAuthResultCheck";s:1:"0";s:48:"fraudCheck-75-PaymentDetailCarteBancaireRefCheck";s:1:"0";s:33:"fraudCheck-29-LiabilityShiftCheck";s:1:"0";s:36:"fraudCheck-63-TransactionAmountCheck";s:1:"0";s:30:"fraudCheck-8-ShopperEmailUsage";s:1:"0";s:39:"fraudCheck-64-TransactionAmountVelocity";s:1:"0";s:30:"fraudCheck-6-ShopperIpRefCheck";s:1:"0";s:49:"fraudCheck-52-DistinctPaymentDetailUsageByShopper";s:1:"0";s:43:"fraudCheck-57-BillingAddressDeliveryAddress";s:1:"0";s:27:"fraudCheck-2-CardChunkUsage";s:1:"0";s:22:"billingAddress.country";s:2:"IT";s:11:"fraudOffset";s:1:"0";s:55:"fraudCheck-54-DistinctSharedPaymentDetailUsageByShopper";s:1:"0";s:30:"fraudCheck-77-EmailSyntaxCheck";s:1:"0";s:17:"fraudManualReview";s:5:"false";s:31:"fraudCheck-3-PaymentDetailUsage";s:1:"0";s:12:"shopperEmail";s:25:"[email protected]";s:27:"fraudCheck-7-ShopperIpUsage";s:2:"10";s:16:"shopperTelephone";s:10:"3331579158";s:43:"fraudCheck-46-DistinctCountryUsageByShopper";s:1:"0";s:51:"fraudCheck-53-DistinctSharedShopperIpUsageByShopper";s:1:"0";s:13:"fundingSource";s:7:"PREPAID";s:16:"shopperReference";s:49:"DODIT00038144dc4718bb-5c2b-480d-857b-9714e7ecd124";s:40:"fraudCheck-55-ShopperAuthorisedFrequency";s:1:"0";s:48:"fraudCheck-50-DistinctShopperEmailUsageByShopper";s:1:"0";s:38:"fraudCheck-48-ChargebackCountByShopper";s:1:"0";s:9:"avsResult";s:17:"3 AVS unavailable";s:28:"fraudCheck-74-EmailNameCheck";s:1:"0";s:43:"fraudCheck-41-PaymentDetailNonFraudRefCheck";s:1:"0";s:11:"cardSummary";s:4:"3050";s:35:"fraudCheck-56-ShopperReferenceCheck";s:1:"0";s:9:"shopperIP";s:12:"172.30.31.25";s:15:"totalFraudScore";s:2:"20";s:31:"fraudCheck-79-BinAttackRefCheck";s:1:"0";s:10:"expiryDate";s:7:"12/2024";s:33:"fraudCheck-65-EmailDomainRefCheck";s:1:"0";s:11:"shopperName";s:15:"Erica Siorpaes ";s:21:"billingAddress.street";s:22:"Località Brite de Val";s:36:"fraudCheck-15-IssuingCountryReferral";s:1:"0";s:33:"fraudCheck-37-ShopperDetailsUsage";s:2:"10";s:34:"fraudCheck-26-ShopperEmailRefCheck";s:1:"0";s:40:"fraudCheck-47-BlockedCardsUsageByShopper";s:1:"0";s:7:"cardBin";s:6:"533862";s:19:"billingAddress.city";s:17:"Cortina d'Ampezzo";s:52:"fraudCheck-51-DistinctShopperReferenceUsageByShopper";s:1:"0";s:28:"fraudCheck-4-HolderNameUsage";s:1:"0";s:17:"acquirerReference";s:12:"214006104525";s:28:"fraudCheck-13-IssuerRefCheck";s:1:"0";s:38:"fraudCheck-10-HolderNameContainsNumber";s:1:"0";s:45:"fraudCheck-70-ShopperConsecutiveRefusalsCheck";s:1:"0";s:18:"cardIssuingCountry";s:2:"IT";s:36:"fraudCheck-69-AutomatedReferralCheck";s:1:"0";s:8:"authCode";s:6:"150487";s:15:"fraudResultType";s:5:"GREEN";s:14:"cardHolderName";s:38:"Erica

You can see the json has stopped at cardholdername where the unseralisation issue occurred in your test, you have assigned the string in different format using font for cardholdername and in the case which happened at my end in Production is different.

With these two columns in adyen_notification table I have sent you, I hope it is more easier to detect the issue.

Thanks, Girdar

[GRajwani]

Hi,

I checked the mysql database collation and it is utf8_general_ci, I tried to change to utf8mb4_unicode_ci, still it does not work. Can you please suggest which collation or which setting to change in mysql to avoid such issues?

Thanks, Girdar

[GRajwani]

Hi, Can you please let me know the settings on the database collation or other settings, we tried to change to collation utf8mb4_unicode_520_ci, still the issue is the same ?

What can the setting of the database so that it accepts the special characters or converts into ???? as it happened at your example?

Thanks, Girdar

[Morerice]

Hi @GRajwani,

We are using utf8mb4 as the character_set_database and utf8mb4_general_ci as the collation_database.

Best regards, Jean Adyen

[Morerice]

Hi @GRajwani,

Did the issue persist after checking out this character set and collation?

Best regards, Jean Adyen

[GRajwani]

Hi, Yes the issue still continue despite changing the collation and character set. I have now disabled special characters to be pasted in the cardholdername to avoid this issue

[Morerice]

Hi @GRajwani,

In that case I'm afraid that at this moment we can't do anything more for this issue from our side given that we weren't able to reproduce it and given the information about the database that we have provided. If however in the future you have some more information about this, feel free to reopen this issue and include any other relevant information.

Regards, Jean Adyen