iconv(): Detected an illegal character in input string at /path/apps/mail/lib/IMAP/Charset/Converter.php#43

[WarpinWolf]

trafficstars

Steps to reproduce

1. Look at log.... Sorry - don't exactly know why and what is causing this in my emails.... I see a recurring multitude of these entries about every 12h at midnight and noon.

3.{"reqId":"wfpt1wIF9x9GPEoXDNP4","level":3,"time":"2025-03-17T09:25:53+01:00","remoteAddr":"","user":"--","app":"PHP","method":"","url":"--","message":"iconv(): Detected an illegal character in input string at /path/apps/mail/lib/IMAP/Charset/Converter.php#43","userAgent":"--","version":"30.0.7.2","data":{"app":"PHP"},"id":"67d7dc91f2b82"}

Expected behavior

No error ?!?! Indication of what was causing the error? Email ID? Text....

Actual behavior

Error

Mail app version

3.7

Nextcloud version

30.0.7.2

Mailserver or service

roundcube

Operating system

Debian Bookworm

PHP engine version

PHP 8.3

Nextcloud memory caching

memcachconfig.php: 'memcache.local' => '\OC\Memcache\APCu', config.php: 'memcache.locking' => '\OC\Memcache\Redis', config.php: 'memcache.distributed' => '\OC\Memcache\Redis', config.php: 'has_rebuilt_cache' => true,

Web server

Apache (supported)

Database

PostgreSQL

Additional info

I did go into mail/lib/IMAP/Charset/Converter.php and fiddled with the code.... Sorry - not really a programmer, didn't understand a lot so did things as I would be doing them - sorry if this is violating anything here. If I replace the original code with mine, no errors pop up anymore....

`<?php

declare(strict_types=1);

/**

• SPDX-FileCopyrightText: 2023 Nextcloud GmbH and Nextcloud contributors
• SPDX-License-Identifier: AGPL-3.0-only */

namespace OCA\Mail\IMAP\Charset;

use Horde_Mime_Part; use OCA\Mail\Exception\ServiceException; use function in_array; use function is_string;

class Converter {

    /**
     * @param Horde_Mime_Part $p
     * @return string
     * @throws ServiceException
     */
    public function convert(Horde_Mime_Part $p): string {
            /** @var null|string $data */
            $returnString = '';

            $data = $p->getContents();

            if ($data !== null) {

                    $charset = $p->getCharset();

                    if ($charset === null) {
                            // let's ask mb if this could be UTF-8
                            $detectedCharset = mb_detect_encoding($data, 'UTF-8', true);
                            if ($detectedCharset === false) {
                                    // Fallback, non UTF-8 - ??????
                                    $detectedCharset = mb_detect_encoding($data, null, true);
                            }
                            // Still UTF8, no need to convert
                            if ($detectedCharset !== false) {
                                    $charset = $detectedCharset;
                            }
                    }

                    if (strtoupper($charset) === 'UTF-8') {
                            // no conversion needed
                            $returnString = $data;
                    } else {
                            // try conversion according to charset - even if null, the conversion routines will try their best...
                            $converted = @mb_convert_encoding($data, 'UTF-8', $charset);
                            if (!is_string($converted)) {
                                try {
                                    // Try using iconv to convert encoding
                                    $converted = iconv($charset, 'UTF-8', $data);
                                } catch (\Exception $e) {
                                    // If iconv throws an exception, log the error and continue without conversion
                                    error_log('iconv failed: ' . $e->getMessage());
                                    // Continue processing with the original data
                                    $converted = false;
                                }
                            }

                            if (is_string($converted)) {
                                    // conversion successfull
                                    $returnString = $converted;
                            } else {
                                    throw new ServiceException('Could not detect message charset');
                            }
                    }
            }

            return $returnString;

    }

} `

[gvargas9]

Same issue here. I am having at least 200 entries. Seems to be related to the message input?

{ "reqId": "6m19wjkjyTshIy1Ot6z8", "level": 3, "time": "2025-03-19T18:28:28+00:00", "remoteAddr": "", "user": false, "app": "PHP", "method": "", "url": "--", "message": "iconv(): Detected an illegal character in input string at /config/www/nextcloud/apps/mail/lib/IMAP/Charset/Converter.php#43", "userAgent": "--", "version": "31.0.1.2", "data": { "app": "PHP" }, "id": "67db0ceea05d6" }

getContents(); if ($data === null) { return ''; } // Only convert encoding if it is explicitly specified in the header because text/calendar // data is utf-8 by default. $charset = $p->getCharset(); if ($charset !== null && strtoupper($charset) === 'UTF-8') { return $data; } // The part specifies a charset if ($charset !== null) { if (in_array($charset, mb_list_encodings(), true)) { $converted = mb_convert_encoding($data, 'UTF-8', $charset); } else { $converted = iconv($charset, 'UTF-8', $data); } if (is_string($converted)) { return $converted; } } // No charset specified, let's ask mb if this could be UTF-8 $detectedCharset = mb_detect_encoding($data, 'UTF-8', true); if ($detectedCharset === false) { // Fallback, non UTF-8 $detectedCharset = mb_detect_encoding($data, null, true); } // Still UTF8, no need to convert if ($detectedCharset !== false && strtoupper($detectedCharset) === 'UTF-8') { return $data; } $converted = @mb_convert_encoding($data, 'UTF-8', $charset); if ($converted === false) { // Might be a charset that PHP mb doesn't know how to handle, fall back to iconv $converted = iconv($charset, 'UTF-8', $data); } if (!is_string($converted)) { throw new ServiceException('Could not detect message charset'); } return $converted; } }

[gvargas9]

This removed the error: (line 43)

Original: $converted = iconv($charset,'UTF-8', $data); Updated: $converted = iconv($charset,'UTF-8//TRANSLIT', $data);

// The part specifies a charset
if ($charset !== null) {
if (in_array($charset, mb_list_encodings(), true)) {
$converted = mb_convert_encoding($data, 'UTF-8', $charset); } else {
$converted = iconv($charset,'UTF-8//TRANSLIT', $data); }

                    if (is_string($converted)) {                                       
                            return $converted;                               
                    }                                                             
            }                                                                            

source: https://www.php.net/manual/en/function.iconv.php

[steffaaannnn]

same here .

[Nextcloud Hub 9 (30.0.8) D.B type Mysql

[codingHahn]

I got this error after receiving a confirmation for an invitation I sent. The other user used google mail to accept.

[hansingt]

Same here on Nextcloud 31.0.2. Mail is just updated to 4.3.6.

Really spams the error log an might make me overlook real problems. :-/

[steffaaannnn]

after update to Nextcloud Hub 10] (31.0.4) , always this problem ... :-/

[mrtreg]

This removed the error: (line 43)

Original: $converted = iconv($charset,'UTF-8', $data); Updated: $converted = iconv($charset,'UTF-8//TRANSLIT', $data);

// The part specifies a charset if ($charset !== null) { if (in_array($charset, mb_list_encodings(), true)) { $converted = mb_convert_encoding($data, 'UTF-8', $charset); } else { $converted = iconv($charset,'UTF-8//TRANSLIT', $data); }

                    if (is_string($converted)) {                                       
                            return $converted;                               
                    }                                                             
            }                                                                            

source: https://www.php.net/manual/en/function.iconv.php

For me, this proposed change above does not work. I need to change line 43 to the following instead:

$converted = iconv($charset, 'UTF-8//IGNORE', $data);

To effectively ignore the "illegal character". This works for me.

[ChristophWurst]

Thank you for the suggestion

Updated: $converted = iconv($charset,'UTF-8//TRANSLIT', $data);

I'm hesitant on this. You can find more details in https://github.com/nextcloud/mail/pull/8327#discussion_r1289715428. Some setups have issues with //IGNORE and //TRANSLIT.

[vawaver]

Hi all,

Just letting you know that on Nextcloud AIO v30.0.11 with Mail v5.1.1 (Ubuntu 22.04, XFS) I still see in the log:

iconv(): Detected an illegal character in input string at
/var/www/html/custom_apps/mail/lib/IMAP/Charset/Converter.php#43

and when attempting DKIM verification:

Undefined array key 1 at
/var/www/html/custom_apps/mail/vendor/phpmailer/dkimvalidator/src/Validator.php#52

In my emails I use Slovak text (with diacritics), so presumably corresponding encoding (e.g. ISO-8859-2 or UTF-8).

[ChristophWurst]

In my emails I use Slovak text (with diacritics), so presumably corresponding encoding (e.g. ISO-8859-2 or UTF-8).

Is there one you could export and share with us for reproduction? Thanks

[kesselb]

Thanks for the reports 🙏

We will look into better handling those errors and enriching the logs with a more context.