glpi-project
Hebrew Characters turn to Gibberish when ticket is received via 0Auth with Office365
Code of Conduct
- [X] I agree to follow this project's Code of Conduct
Is there an existing issue for this?
- [X] I have searched the existing issues
Version
10.0.12
Bug description
I installed GLPI 10.0.12 and after connecting our ticketing office365 tenant mailbox via 0auth, I tried opening tickets via email with Hebrew characters and I get gibberish instead in the subject and the body.
When I open it manually on the platform there's no problem and Hebrew is shown fine.
Relevant log output
No response
Page URL
No response
Steps To reproduce
- Connect receiver office365 mailbox with 0Auth to GLPI
- Setup automatic mailgate to retrieve the new emails as tickets
- Send email with Hebrew characters to this mailbox
- The new ticket is created, but with Hebrew content in Gibberish
Your GLPI setup information
Information about system installation & configuration
GLPI 10.0.12 ( => /var/www/glpi) Installation mode: TARBALL Current language:en_US
Server
Operating system: Linux REDACTED WEBSITE ADDRESS 5.15.0-113-generic #123-Ubuntu SMP Mon Jun 10 08:16:17 UTC 2024 x86_64 PHP 8.1.2-1ubuntu2.18 fpm-fcgi (Core, FFI, PDO, Phar, Reflection, SPL, SimpleXML, Zend OPcache, apcu, bz2, calendar, cgi-fcgi, ctype, curl, date, dom, exif, fileinfo, filter, ftp, gd, gettext, hash, iconv, imagick, imap, intl, json, ldap, libxml, mbstring, memcache, mysqli, mysqlnd, openssl, pcre, pdo_mysql, posix, pspell, readline, session, shmop, sockets, sodium, standard, sysvmsg, sysvsem, sysvshm, tidy, tokenizer, xml, xmlreader, xmlrpc, xmlwriter, xsl, zip, zlib) Setup: max_execution_time="30" memory_limit="128M" post_max_size="8M" safe_mode="" session.save_handler="files" upload_max_filesize="2M" disable_functions="" Software: nginx/1.18.0 Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/126.0.0.0 Safari/537.36 Server Software: Ubuntu 22.04 Server Version: 10.6.18-MariaDB-0ubuntu0.22.04.1 Server SQL Mode: STRICT_TRANS_TABLES,ERROR_FOR_DIVISION_BY_ZERO,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION Parameters: glpi_admin@localhost/glpi_db Host info: Localhost via UNIX socketPHP version (8.1.2-1ubuntu2.18) is supported. Sessions configuration is OK. Allocated memory is sufficient. mysqli extension is installed. Following extensions are installed: dom, fileinfo, filter, libxml, json, simplexml, xmlreader, xmlwriter. curl extension is installed. gd extension is installed. intl extension is installed. zlib extension is installed. The constant SODIUM_CRYPTO_AEAD_XCHACHA20POLY1305_IETF_NPUBBYTES is present. Database engine version (10.6.18) is supported. No files from previous GLPI version detected. The log file has been created successfully. Write access to /var/www/glpi/files/_cache has been validated. Write access to /var/www/glpi/files/_cron has been validated. Write access to /var/www/glpi/files has been validated. Write access to /var/www/glpi/files/_dumps has been validated. Write access to /var/www/glpi/files/_graphs has been validated. Write access to /var/www/glpi/files/_lock has been validated. Write access to /var/www/glpi/files/_pictures has been validated. Write access to /var/www/glpi/files/_plugins has been validated. Write access to /var/www/glpi/files/_rss has been validated. Write access to /var/www/glpi/files/_sessions has been validated. Write access to /var/www/glpi/files/_tmp has been validated. Write access to /var/www/glpi/files/_uploads has been validated.
Web server root directory configuration seems safe. Sessions configuration is secured. OS and PHP are relying on 64 bits integers. exif extension is installed. ldap extension is installed. openssl extension is installed. Following extensions are installed: bz2, Phar, zip. Zend OPcache extension is installed. Following extensions are installed: ctype, iconv, mbstring, sodium. Write access to /var/www/glpi/marketplace has been validated. Timezones seems loaded in database.
GLPI constants
GLPI_ROOT: "/var/www/glpi" GLPI_CONFIG_DIR: "/var/www/glpi/config" GLPI_VAR_DIR: "/var/www/glpi/files" GLPI_MARKETPLACE_DIR: "/var/www/glpi/marketplace" GLPI_USE_CSRF_CHECK: "1" GLPI_CSRF_EXPIRES: "7200" GLPI_CSRF_MAX_TOKENS: "100" GLPI_USE_IDOR_CHECK: "1" GLPI_IDOR_EXPIRES: "7200" GLPI_ALLOW_IFRAME_IN_RICH_TEXT: false GLPI_SERVERSIDE_URL_ALLOWLIST: ["/^(https?|feed):\/\/[^@:]+(\/.*)?$/"] GLPI_TELEMETRY_URI: "https://telemetry.glpi-project.org" GLPI_INSTALL_MODE: "TARBALL" GLPI_NETWORK_MAIL: "[email protected]" GLPI_NETWORK_SERVICES: "https://services.glpi-network.com" GLPI_MARKETPLACE_ALLOW_OVERRIDE: true GLPI_MARKETPLACE_MANUAL_DOWNLOADS: true GLPI_USER_AGENT_EXTRA_COMMENTS: "" GLPI_DISABLE_ONLY_FULL_GROUP_BY_SQL_MODE: "1" GLPI_AJAX_DASHBOARD: "1" GLPI_CALDAV_IMPORT_STATE: 0 GLPI_DEMO_MODE: "0" GLPI_CENTRAL_WARNINGS: "1" GLPI_TEXT_MAXSIZE: "4000" GLPI_DOC_DIR: "/var/www/glpi/files" GLPI_CACHE_DIR: "/var/www/glpi/files/_cache" GLPI_CRON_DIR: "/var/www/glpi/files/_cron" GLPI_DUMP_DIR: "/var/www/glpi/files/_dumps" GLPI_GRAPH_DIR: "/var/www/glpi/files/_graphs" GLPI_LOCAL_I18N_DIR: "/var/www/glpi/files/_locales" GLPI_LOCK_DIR: "/var/www/glpi/files/_lock" GLPI_LOG_DIR: "/var/www/glpi/files/_log" GLPI_PICTURE_DIR: "/var/www/glpi/files/_pictures" GLPI_PLUGIN_DOC_DIR: "/var/www/glpi/files/_plugins" GLPI_RSS_DIR: "/var/www/glpi/files/_rss" GLPI_SESSION_DIR: "/var/www/glpi/files/_sessions" GLPI_TMP_DIR: "/var/www/glpi/files/_tmp" GLPI_UPLOAD_DIR: "/var/www/glpi/files/_uploads" GLPI_INVENTORY_DIR: "/var/www/glpi/files/_inventories" GLPI_NETWORK_REGISTRATION_API_URL: "https://services.glpi-network.com/api/registration/" GLPI_MARKETPLACE_PLUGINS_API_URI: "https://services.glpi-network.com/api/marketplace/" GLPI_I18N_DIR: "/var/www/glpi/locales" GLPI_VERSION: "10.0.12" GLPI_SCHEMA_VERSION: "10.0.12@b902ef5f5da967c1a9d0826ee8a7dce07d57cf3d" GLPI_MARKETPLACE_PRERELEASES: false GLPI_MIN_PHP: "7.4.0" GLPI_MAX_PHP: "8.4.0" GLPI_YEAR: "2024"
Libraries
htmlawed/htmlawed version 1.2.14 in (/var/www/glpi/vendor/htmlawed/htmlawed) phpmailer/phpmailer version 6.8.0 in (/var/www/glpi/vendor/phpmailer/phpmailer/src) simplepie/simplepie version 1.5.8 in (/var/www/glpi/vendor/simplepie/simplepie/library) tecnickcom/tcpdf version 6.6.2 in (/var/www/glpi/vendor/tecnickcom/tcpdf) michelf/php-markdown in (/var/www/glpi/vendor/michelf/php-markdown/Michelf) true/punycode in (/var/www/glpi/vendor/true/punycode/src) iamcal/lib_autolink in (/var/www/glpi/vendor/iamcal/lib_autolink) sabre/dav in (/var/www/glpi/vendor/sabre/dav/lib/DAV) sabre/http in (/var/www/glpi/vendor/sabre/http/lib) sabre/uri in (/var/www/glpi/vendor/sabre/uri/lib) sabre/vobject in (/var/www/glpi/vendor/sabre/vobject/lib) laminas/laminas-i18n in (/var/www/glpi/vendor/laminas/laminas-i18n/src) laminas/laminas-servicemanager in (/var/www/glpi/vendor/laminas/laminas-servicemanager/src) monolog/monolog in (/var/www/glpi/vendor/monolog/monolog/src/Monolog) sebastian/diff in (/var/www/glpi/vendor/sebastian/diff/src) donatj/phpuseragentparser in (/var/www/glpi/vendor/donatj/phpuseragentparser/src/UserAgent) elvanto/litemoji in (/var/www/glpi/vendor/elvanto/litemoji/src) symfony/console in (/var/www/glpi/vendor/symfony/console) scssphp/scssphp in (/var/www/glpi/vendor/scssphp/scssphp/src) laminas/laminas-mail in (/var/www/glpi/vendor/laminas/laminas-mail/src/Protocol) laminas/laminas-mime in (/var/www/glpi/vendor/laminas/laminas-mime/src) rlanvin/php-rrule in (/var/www/glpi/vendor/rlanvin/php-rrule/src) ramsey/uuid in (/var/www/glpi/vendor/ramsey/uuid/src) psr/log in (/var/www/glpi/vendor/psr/log/Psr/Log) psr/simple-cache in (/var/www/glpi/vendor/psr/simple-cache/src) psr/cache in (/var/www/glpi/vendor/psr/cache/src) league/csv in (/var/www/glpi/vendor/league/csv/src) mexitek/phpcolors in (/var/www/glpi/vendor/mexitek/phpcolors/src/Mexitek/PHPColors) guzzlehttp/guzzle in (/var/www/glpi/vendor/guzzlehttp/guzzle/src) guzzlehttp/psr7 in (/var/www/glpi/vendor/guzzlehttp/psr7/src) glpi-project/inventory_format in (/var/www/glpi/vendor/glpi-project/inventory_format/lib/php) wapmorgan/unified-archive in (/var/www/glpi/vendor/wapmorgan/unified-archive/src) paragonie/sodium_compat in (/var/www/glpi/vendor/paragonie/sodium_compat/src) symfony/cache in (/var/www/glpi/vendor/symfony/cache) html2text/html2text in (/var/www/glpi/vendor/html2text/html2text/src) symfony/css-selector in (/var/www/glpi/vendor/symfony/css-selector) symfony/dom-crawler in (/var/www/glpi/vendor/symfony/dom-crawler) twig/twig in (/var/www/glpi/vendor/twig/twig/src) twig/string-extra in (/var/www/glpi/vendor/twig/string-extra) symfony/polyfill-ctype not found symfony/polyfill-iconv not found symfony/polyfill-mbstring not found symfony/polyfill-php80 not found symfony/polyfill-php81 not found symfony/polyfill-php82 in (/var/www/glpi/vendor/symfony/polyfill-php82) league/oauth2-client in (/var/www/glpi/vendor/league/oauth2-client/src/Provider) league/oauth2-google in (/var/www/glpi/vendor/league/oauth2-google/src/Provider) thenetworg/oauth2-azure in (/var/www/glpi/vendor/thenetworg/oauth2-azure/src/Provider) phpCas version 1.3.8 in (/usr/share/php/CAS/source)
SQL replicas
Not active
Notifications
Way of sending emails: SMTP+TLS ([email protected]@smtp.office365.com)
Plugins list
news Name: Alerts Version: 1.12.2 State: Installed / not activated
Install Method: Marketplace behaviors Name: Behaviours Version: 2.7.3 State: Installed / not activated
Install Method: Marketplace datainjection Name: Data injection Version: 2.13.5 State: Installed / not activated
Install Method: Marketplace archimap Name: Diagrams Version: 3.3.4 State: Installed / not activated
Install Method: Marketplace financialreports Name: Financial reports Version: 3.0.0 State: Installed / not activated
Install Method: Marketplace addressing Name: IP Adressing Version: 3.0.2 State: Installed / not activated
Install Method: Marketplace mfa Name: MFA Version: 1.0.2 State: Installed / not activated
Install Method: Marketplace mreporting Name: More Reporting Version: 1.8.6 State: Installed / not activated
Install Method: Marketplace oauthimap Name: Oauth IMAP Version: 1.4.3 State: Enabled
Install Method: Marketplace onetimesecret Name: OneTimeSecret Version: 2.1.3 State: Installed / not activated
Install Method: Marketplace pdf Name: Print to pdf Version: 3.0.0 State: Installed / not activated
Install Method: Marketplace reports Name: Reports Version: 1.16.0 State: Installed / not activated
Install Method: Marketplace screenshot Name: Screenshot Version: 2.0.3 State: Installed / not activated
Install Method: Marketplace ticketfilter Name: Ticketfilter Version: 1.3.2 State: Installed / not activated
Install Method: Marketplace timelineticket Name: Timeline of tickets Version: 10.0+1.2 State: Installed / not activated
Install Method: Marketplace vip Name: VIP Version: 1.8.2 State: Installed / not activated
Install Method: Marketplace
Anything else?
these might be relevant but i dont know if these fixes were implemented into 10.0.12 so i decided to open a new issue here https://github.com/glpi-project/glpi/pull/7748 https://github.com/glpi-project/glpi/pull/4844
this is the closest issue I found on the subject https://github.com/glpi-project/glpi/issues/7709
Hi,
Please check if the problem persists in GLPI 10.0.15. If problem persists, please attache the message into .eml format so we could add it to our test suite and see the results.
Hi Cedric thanks for the quick reply. we upgrade to 10.0.15 and now the gibberish is different haha, still an encoding issue.
any recommendations?
Testing after 10.0.15 בדיקה נוספת אחרי מעבר לגרסה חדשה.eml.zip
it was blocking the .eml file so it's attached inside this compressed zip file.
I validate the issue. It sees that the problem is related to usage of iso-8859-8-i encoding.
*** PHP Warning (2): iconv(): Wrong encoding, conversion from "iso-8859-8-i" to "UTF-8//IGNORE" is not allowed in /var/www/glpi/src/MailCollector.php at line 2470
Backtrace :
src/MailCollector.php:2470 iconv()
src/MailCollector.php:1808 MailCollector->getDecodedContent()
src/MailCollector.php:1150 MailCollector->getBody()
src/MailCollector.php:811 MailCollector->buildTicket()
front/mailcollector.form.php:106 MailCollector->collect()
public/index.php:82 require()
Issue solved.
By editing Mailcollector.php I was able to edit the functions decodeMimeHeader and getDecodedContent to be able to handle Hebrew characters. the decoding function that works for incoming emails from M365 is mb_convert_encoding.
Attaching my code snippets of Mailcollector.php for your consideration. I will test some more next week but you can close this thread if you want.
- Subject wasn't using the subject decoding function - it was going to 'cleanSubject' instead. I replace cleanSubject with my decodeMimeHeader function.
- getDecodedContent had too many conditions at first, then i replaced it with a leaner version which is suitable for our needs. GLPI Mailcollector.php updated functions and code for Hebrew Encoding email messages.zip
Could you please open a pull request with your changes? Test suite will run with your changes; and it's easiest for us to review changes.
I took another look at the issue and proposed patch; and I see just just rely on ISO-8859-8 instead of ISO-8859-8-I (the one PHP cannot handle correctly).
We already handle similar cases with unknown windows encodings, so https://github.com/glpi-project/glpi/pull/19951 should fix the issue.