Reading mail fails if it has attachments with UTF-8 characters in filenames

[APaikens]

I'm reading mail from server, using IMAP storage.

Problems started to show after update to latest version - 2.4.2.
With 2.4.0 everything works as described in example: http://framework.zend.com/manual/current/en/modules/zend.mail.read.html

Seems that this is a bit different than #7501

Problems starts with getting parts of message:
foreach (new RecursiveIteratorIterator($mail->getMessage(1)) as $part) {

Everything gets read and decoded correctly, even headers with UTF-8, but at the end it doesn't pass new validation for only-ASCII characters.

\vendor\zendframework\zendframework\library\Zend\Mail\Header\ContentType.php:145
Message:
Invalid content-type parameter value detected
Stack trace:
#0 vendor\zendframework\zendframework\library\Zend\Mail\Header\ContentType.php(55): Zend\Mail\Header\ContentType->addParameter('name', 'Demo_skulpt??ra...')

...

Currently I'm out of ideas how to solve this

[Maks3w]

Could you paste the exact arguments used in addParameter?

[Maks3w]

/cc @weierophinney

[APaikens]

Yes, sure.
It can be repeated this way:

$testContentType = new \Zend\Mail\Header\ContentType;
$testContentType->addParameter('name', 'Demo_skulptūra_Vecā_Jaunā.jpg');

The same exception

[weierophinney]

@APaikens Values in headers MUST follow RFC 2822, which means only visible ASCII characters. If you supply values that are outside that range, you must encode them first.

My question is: are you creating a mail message from an individual part? If so, we should likely add some logic to allow passing un-encoded characters to the headers, and then encode them on-the-fly when retrieving them. I need a full use case, however, before we look into anything like this.

[APaikens]

A little bit more details - i follow examples from this link: http://framework.zend.com/manual/current/en/modules/zend.mail.read.html

My aim is to process email messages as files, not to send them again.

1. create new storage (imap) with credentials
2. get particular message (like in example) - $message = $mail->getMessage($messageNum);
3. Access message parts to get content and attachments:

foreach (new RecursiveIteratorIterator($mail->getMessage(1)) as $part) {

It stops at this point.

I checked source email - to me it seems, that attachments are encoded correctly.

Simple test case - sent myself empty email with small text file, named Bādsīmbols.txt
Email sources from web client shows, that it is encoded correctly:

Content-Type: text/plain; charset=US-ASCII; name="=?UTF-8?B?QsSBZHPEq21ib2xzLnR4dA==?="
Content-Disposition: attachment; filename="=?UTF-8?B?QsSBZHPEq21ib2xzLnR4dA==?="

But - reading that email using previously described method gives same error.

To me it seems that there is some hook inside logic - correct header names get decoded and then again passed to headers objects, where it fails - because they are already decoded.

[APaikens]

Hi there again, created a gist with example - https://gist.github.com/APaikens/40641d977fd5f05fbbf5
Is this more useful?

[yurikuzn]

I have problem with this too. @APaikens, have you found any solution?

[APaikens]

No, not yet

[croensch]

Well Zend\Mail\Headers\ContentType should not decode the value in fromString and then have addParameter() expect it encoded.
Mime decoding should be somewhere outside where its actually needed.

[homeempire]

Any solution?

[yurikuzn]

Fixed in 2.5.2

[APaikens]

Works in 2.5.2

[pillex]

@weierophinney

I need a full use case, however, before we look into anything like this.

I have similar problems in ZF1. Full email below, that throws the exception when read and parsed with ZF. It was created with Thunderbid.

Emails cannot be parsed, if they have attachments containing utf8 characters in the filnames.

I'm using ZF 1.12.17

for me the exception is thrown in

foreach( new RecursiveIteratorIterator($message) as $part )

the error is:
Invalid header value detected

the exception happens in
/Mail/Header/HeaderValue.php

public static function isValid($value)
returns false for filenames with utf8 characters.

example $value that throws:
text/plain; charset=UTF-8; name="testä.txt"

here is a complete test email that throws the exception.
the filename of the attachment is testä.txt

To: [email protected]
From: "[email protected]" <[email protected]>
Subject: test
Date: Fri, 13 Apr 2016 19:29:37 +0200
MIME-Version: 1.0
Content-Type: multipart/mixed;
 boundary="------------050706090708030904000509"

This is a multi-part message in MIME format.
--------------050706090708030904000509
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: 7bit

Just some test text body

--------------050706090708030904000509
Content-Type: text/plain; charset=UTF-8;
 name="=?UTF-8?Q?test=c3=a4.txt?="
Content-Transfer-Encoding: base64
Content-Disposition: attachment;
 filename*=utf-8''%74%65%73%74%C3%A4%2E%74%78%74

YXR0YWNobWVudCB0ZXN0
--------------050706090708030904000509--