Package home | Report new bug | New search | Development Roadmap Status: Open | Feedback | All | Closed Since Version 1.10.12

Bug #11238 From address encoding
Submitted: 2007-06-06 13:34 UTC
From: cederstrom Assigned: cipri
Status: Closed Package: Mail_Mime (version 1.4.0)
PHP Version: 5.2.1 OS: FreeBSD
Roadmaps: 1.5.3    
Subscription  


 [2007-06-06 13:34 UTC] cederstrom (Lasse Cederstrom)
Description: ------------ When the From name contain a danish char æ,ø or å, then the sending fails with the error "Validation failed for: =?ISO-8859-1?Q?Test=F..." If I change the 'From'=>"Testø Testå <me@example.com>" into 'From'=>"Test Test <me@example.com>" it works fine. Also if I change it into 'From'=>"Testø Testå", and add the "<me@example.com>" after the $mime->headers(..) line it also works fine. The error was not in version 1.36. Test script: --------------- <? include('Mail.php'); include('Mail/mime.php'); $crlf = "\n"; $hdrs = array( 'From' => "Testø Testå <me@example.com>", 'Subject' => "Example"); $mime = new Mail_mime($crlf); $mime->setTXTBody("Very short email!"); //do not ever try to call these lines in reverse order $body = $mime->get(); $hdrs = $mime->headers($hdrs); $mail =& Mail::factory('mail'); $err = $mail->send("lasse@novicell.dk", $hdrs, $body); print_r($err); ?> Expected result: ---------------- The mail to be sent. Actual result: -------------- PEAR_Error Object - containing a validation error.

Comments

 [2007-06-17 21:47 UTC] cipri (Cipriano Groenendal)
THe problem is that the entire From header is encoded, including the email address. I've created a test case in CVS, and'll fix this bug in 1.6.0
 [2007-06-18 08:01 UTC] cipri (Cipriano Groenendal)
Got a possible fix running locally, will commit and release soon.
 [2007-06-26 16:10 UTC] jtacon (Javier Tacón)
Is there some news about the fix?
 [2007-07-02 19:02 UTC] mkone (Martin Körner)
Two other problems with the display-name: 1. Display-Name as a Quoted String: 'From' => '"Testä Testö" <me@example.com>' You will receive a "Validation failed for ..." error from the Mail_RFC822 class while trying to use $mail->send(). 2. Special Chars in Display-Name In the Mail_RFC822 class special characters are specified as ()<>@,;\:". If any of the From, CC, To headers has a display name that contains one of the special characters, you could use a Quoted String - but then No. 1 happens. If you don't quote: 'From' => 'Testä@home <me@example.com>' you will receive also a "Validation failed for ..." error. The _encodeHeaders method could encode the special characters (not only non-us-ascii), too. And additionally handle quoted strings correctly.
 [2007-07-03 10:26 UTC] jtacon (Javier Tacón)
Cipriano, is there some news about the fix?
 [2007-07-06 10:05 UTC] maksimus (Maks Maks)
If "From" contains national letters Your convert is wrong: =?koi8-r?Q?=F4=C5=D3=CC=C5=CE=CB=CF_<mail@mail.ru>?= Header must be =?koi8-?Q?=F4=C5=D3=CC=C5=CE=CB=CF?= <mail@mail.ru> Please fix it!!!!
 [2007-07-12 08:51 UTC] vzeman (Viktor Zeman)
This problem is not related to Free BSD ! It is problem for all platforms. When do you think we can wait fix of this problem ?
 [2007-07-13 10:49 UTC] vzeman (Viktor Zeman)
I have made a really short hack for this problem until you will find real solution. Here is function, which you have to replace in original database. It doesn't support all types of mail formats ! Just following mail format is supported: "special characters here" <any@any.com> function _encodeHeaders($input, $params = array()) { $build_params = $this->_build_params; while (list($key, $value) = each($params)) { $build_params[$key] = $value; } //$hdr_name: Name of the heaer //$hdr_value: Full line of header value. //$hdr_value_out: The recombined $hdr_val-atoms, or the encoded string. $useIconv = true; if (isset($build_params['ignore-iconv'])) { $useIconv = !$build_params['ignore-iconv']; } foreach ($input as $hdr_name => $hdr_value) { $hdr_value_suffix = ""; if (in_array($hdr_name, array('Reply-To', 'From'))) { if (preg_match('/^(".*")( <.*>)$/', $hdr_value, $match)) { $hdr_value_suffix = $match[2]; $hdr_value = $match[1]; } } if (preg_match('#([\x80-\xFF]){1}#', $hdr_value)) { if (function_exists('iconv_mime_encode') && $useIconv) { $imePrefs = array(); if ($build_params['head_encoding'] == 'base64') { $imePrefs['scheme'] = 'B'; } else { $imePrefs['scheme'] = 'Q'; } $imePrefs['input-charset'] = $build_params['head_charset']; $imePrefs['output-charset'] = $build_params['head_charset']; $imePrefs['line-length'] = 74; $imePrefs['line-break-chars'] = "\r\n"; //Specified in RFC2047 $hdr_value = iconv_mime_encode($hdr_name, $hdr_value, $imePrefs); $hdr_value = preg_replace("#^{$hdr_name}\:\ #", "", $hdr_value); } elseif ($build_params['head_encoding'] == 'base64') { //Base64 encoding has been selected. //Base64 encode the entire string $hdr_value = base64_encode($hdr_value); //Generate the header using the specified params and dynamicly //determine the maximum length of such strings. //75 is the value specified in the RFC. The first -2 is there so //the later regexp doesn't break any of the translated chars. //The -2 on the first line-regexp is to compensate for the ": " //between the header-name and the header value $prefix = '=?' . $build_params['head_charset'] . '?B?'; $suffix = '?='; $maxLength = 75 - strlen($prefix . $suffix) - 2; $maxLength1stLine = $maxLength - strlen($hdr_name) - 2; //We can cut base4 every 4 characters, so the real max //we can get must be rounded down. $maxLength = $maxLength - ($maxLength % 4); $maxLength1stLine = $maxLength1stLine - ($maxLength1stLine % 4); $cutpoint = $maxLength1stLine; $hdr_value_out = $hdr_value; $output = ""; while ($hdr_value_out) { //Split translated string at every $maxLength $part = substr($hdr_value_out, 0, $cutpoint); $hdr_value_out = substr($hdr_value_out, $cutpoint); $cutpoint = $maxLength; //RFC 2047 specifies that any split header should //be seperated by a CRLF SPACE. if ($output) { $output .= "\r\n "; } $output .= $prefix . $part . $suffix; } $hdr_value = $output; } else { //quoted-printable encoding has been selected //Fix for Bug #10298, Ota Mares <om@viazenetti.de> //Check if there is a double quote at beginning or end of //the string to prevent that an open or closing quote gets //ignored because it is encapsuled by an encoding pre/suffix. //Remove the double quote and set the specific prefix or //suffix variable so that we can concat the encoded string and //the double quotes back together to get the intended string. $quotePrefix = $quoteSuffix = ''; if ($hdr_value{0} == '"') { $hdr_value = substr($hdr_value, 1); $quotePrefix = '"'; } if ($hdr_value{strlen($hdr_value)-1} == '"') { $hdr_value = substr($hdr_value, 0, -1); $quoteSuffix = '"'; } //Generate the header using the specified params and dynamicly //determine the maximum length of such strings. //75 is the value specified in the RFC. The -2 is there so //the later regexp doesn't break any of the translated chars. //The -2 on the first line-regexp is to compensate for the ": " //between the header-name and the header value $prefix = '=?' . $build_params['head_charset'] . '?Q?'; $suffix = '?='; $maxLength = 75 - strlen($prefix . $suffix) - 2 - 1; $maxLength1stLine = $maxLength - strlen($hdr_name) - 2; $maxLength = $maxLength - 1; //Replace all special characters used by the encoder. $search = array('=', '_', '?', ' '); $replace = array('=3D', '=5F', '=3F', '_'); $hdr_value = str_replace($search, $replace, $hdr_value); //Replace all extended characters (\x80-xFF) with their //ASCII values. $hdr_value = preg_replace('#([\x80-\xFF])#e', '"=" . strtoupper(dechex(ord("\1")))', $hdr_value); //This regexp will break QP-encoded text at every $maxLength //but will not break any encoded letters. $reg1st = "|(.{0,$maxLength1stLine}[^\=][^\=])|"; $reg2nd = "|(.{0,$maxLength}[^\=][^\=])|"; //Fix for Bug #10298, Ota Mares <om@viazenetti.de> //Concat the double quotes and encoded string together $hdr_value = $quotePrefix . $hdr_value . $quoteSuffix; $hdr_value_out = $hdr_value; $realMax = $maxLength1stLine + strlen($prefix . $suffix); if (strlen($hdr_value_out) >= $realMax) { //Begin with the regexp for the first line. $reg = $reg1st; $output = ""; while ($hdr_value_out) { //Split translated string at every $maxLength //But make sure not to break any translated chars. $found = preg_match($reg, $hdr_value_out, $matches); //After this first line, we need to use a different //regexp for the first line. $reg = $reg2nd; //Save the found part and encapsulate it in the //prefix & suffix. Then remove the part from the //$hdr_value_out variable. if ($found) { $part = $matches[0]; $len = strlen($matches[0]); $hdr_value_out = substr($hdr_value_out, $len); } else { $part = $hdr_value_out; $hdr_value_out = ""; } //RFC 2047 specifies that any split header should //be seperated by a CRLF SPACE if ($output) { $output .= "\r\n "; } $output .= $prefix . $part . $suffix; } $hdr_value_out = $output; } else { $hdr_value_out = $prefix . $hdr_value_out . $suffix; } $hdr_value = $hdr_value_out; } } $input[$hdr_name] = $hdr_value . $hdr_value_suffix; } return $input; }
 [2007-07-13 10:54 UTC] vzeman (Viktor Zeman)
Here is also diff of my change: @@ -923,7 +923,16 @@ if (isset($build_params['ignore-iconv'])) { $useIconv = !$build_params['ignore-iconv']; } - foreach ($input as $hdr_name => $hdr_value) { + foreach ($input as $hdr_name => $hdr_value) { + + $hdr_value_suffix = ""; + if (in_array($hdr_name, array('Reply-To', 'From'))) { + if (preg_match('/^(".*")( <.*>)$/', $hdr_value, $match)) { + $hdr_value_suffix = $match[2]; + $hdr_value = $match[1]; + } + } + if (preg_match('#([\x80-\xFF]){1}#', $hdr_value)) { if (function_exists('iconv_mime_encode') && $useIconv) { $imePrefs = array(); @@ -1069,7 +1078,7 @@ $hdr_value = $hdr_value_out; } } - $input[$hdr_name] = $hdr_value; + $input[$hdr_name] = $hdr_value . $hdr_value_suffix; } return $input; }
 [2007-08-29 12:27 UTC] oliver (Oliver Jusinger)
Thanks to vzeman for the patch for this critical bug. Unfortunately iconv_mime_encode isn't available for php4. It would be great if a new version would be available soon.
 [2007-08-31 15:02 UTC] vzeman (Viktor Zeman)
Oliver, your are right, this function is supported from php5. In this module is implemented workaround for this function. You can see check: if (function_exists('iconv_mime_encode') && $useIconv)
 [2007-10-12 16:18 UTC] dlublink2 (David Lublink)
I tried your patch, and it gives the same error, but it only includes the words inside the quotes instead of the entire header. Additionally, when there are no accents it seems to drop the email address and take the name that was in quotes only. When postfix receives this it appends the domain name so I end up with " Sender@internal-server-name.mydomain.tld ".
 [2007-12-11 22:16 UTC] mick (Michael Heuberger)
This bug is really annoying me. Is there any progress already made? Or does a safe workaround already exists? Regards Michael
 [2008-02-11 14:42 UTC] ameyer (André Meyer)
Half a year ago since this bug was submitted, and still no solution?
 [2008-02-19 23:37 UTC] lego (Le Go)
Seriously, this package needs some active maintainers - this bug is a deal-breaker. Does anybody know of any decent alternatives?
 [2008-03-11 11:08 UTC] knikke (Niko Lehtonen)
Hey, Any progress here? I installed that newest Mail_mime and there is still that sender encode problem. Regards, Niko
 [2008-03-25 17:51 UTC] dlublink (David Lublink)
Looks like Pear Mail relies completely on iconv_mime_encode which does not distinguish between a header with an email address and a header without. echo iconv_mime_encode('Subject','Hello world!éé'); echo "\n"; echo iconv_mime_encode('To','\'Hello world!éé\' <david@banana.org>',array('scheme'=>'Q')); echo "\n"; echo iconv_mime_encode('from','"Hello world!éé" <david@banana.org>',array('scheme'=>'Q')); echo "\n";
 [2008-07-07 21:16 UTC] ain (Ain Tohvri)
Yes, this is a serious drawback. Fix would be highly appreciated.
 [2008-07-07 21:33 UTC] brubla (Brubla Brubla)
Fixed version of mime.php can be downloaded from here: http://www.presentum.cz/mime.zip Works with following addresses: mail@example.com Test ě¨čř¸ýáíé<mail@example.com> 'Test ě¨čř¸ýáíé'<mail@example.com> "Test ě¨čř¸ýáíé"<mail@example.com> Test ě¨čř¸ýáíé<mail@example.com>, Test2<mail2@example.com> "Test ě¨čř¸ýáíé" <mail@example.com> Also provides separate function _encode() for any string conversion requested by some other bug entry. Changes in mime.php labelled with string: Edited by MP for SF
 [2008-08-10 14:28 UTC] obengelb (Fake Name)
It seems the problem is still not solved. Any suggestions on how to cope with it? I have mail names like Spezial "Gemüse" Ltd. <info@gemuese-ltd.de> entered by users. In this case the packages breaks mail sending completely.
 [2008-12-27 18:05 UTC] avb (Alexey Borzov)
I've read RFC 2047 and it has quite explicit rules in section 5 on where encoded words may appear. This unfortunately means that a proper implementation will need to distinguish between headers that are simply defined as '*text' and structured ones. It will also need to parse the values of the latter ones and properly handle at least * comments * quoted strings * address specs Unfortunately, none of the proposed solutions do this yet, some even break more than they fix. In particular, I tried using the version from http://www.presentum.cz/mime.zip and number of failed tests for Mail_mime went from 4 to 16.
 [2009-06-04 03:36 UTC] erne100 (Ernesto Che)
I use the version from http://www.presentum.cz/mime.zip and it works for me!
 [2009-12-21 14:23 UTC] alec (Aleksander Machniak)
-Status: Analyzed +Status: Closed
This bug has been fixed in SVN. If this was a documentation problem, the fix will appear on pear.php.net by the end of next Sunday (CET). If this was a problem with the pear.php.net website, the change should be live shortly. Otherwise, the fix will appear in the package's next release. Thank you for the report and for helping us make PEAR better.