[gentoo-user] Mutt and C\x05C%C\x04C$C\x16C6 characters

[gentoo-user] Mutt and C\x05C%C\x04C$C\x16C6 characters

[Juho Rosqvist]

Now, the subject _should_ read:
Mutt and ÅåÄäÖö characters

The problem is that when I begin composing a message in mutt and it asks
for address and subject, the scandinavian characters (among others) are
garbled as I type them in. This is puzzling, because mutt's pager displays
them just fine on received mail, vim works like a charm, and the terminal
has no problem with those characters either. Situation is the same under
pure console and X. I have found no help on the net; this does not seem
to be a common grief. Mutt's input line seems to be the only thing
affected.

I'm running an utf-8 environment, and the muttrc are in utf-8 according
to vim. Output of `locale` is:
LANG=
LC_CTYPE=fi_FI.utf8
LC_NUMERIC="POSIX"
LC_TIME="POSIX"
LC_COLLATE=en_US.utf8
LC_MONETARY="POSIX"
LC_MESSAGES="POSIX"
LC_PAPER=fi_FI.utf8
LC_NAME="POSIX"
LC_ADDRESS="POSIX"
LC_TELEPHONE="POSIX"
LC_MEASUREMENT="POSIX"
LC_IDENTIFICATION="POSIX"
LC_ALL=

Setting LC_ALL=fi_FI.utf8 won't remove the problem. I have even played
with $send_charset, which is set to "us-ascii:iso-8859-1:utf-8", but
changing it to "iso-8859-1" or "utf-8" made no difference.

I'm really at a loss as to what causes this problem, or how to fix it.
Help would be appreciated.

Regards, Juho
-- 
gentoo-user@gentoo.org mailing list

Re: [gentoo-user] Mutt and C\x05C%C\x04C$C\x16C6 characters

[Graham Murray]

Juho Rosqvist <juho.rosqvist@mbnet.fi> writes:

> Now, the subject _should_ read:
> Mutt and ÅåÄäÖö characters
[snip]
> I'm really at a loss as to what causes this problem, or how to fix it.
> Help would be appreciated.

The problem is that if mail headers (especially to, from and subject)
contain non US-Ascii characters (as your email does), then the mail
program is supposed to use the mechanism described in RFC2047. Your
emailer is not doing this, but is sending the 'raw' Scandinavian
characters. As to how to fix it, I am sorry but I cannot be of any
help.  
--
gentoo-user@gentoo.org mailing list

Re: [gentoo-user] Mutt and C\x05C%C\x04C$C\x16C6 characters

[Juho Rosqvist]

On Sat, Mar 31, 2007 at 06:31:34 +0100, Graham Murray wrote:
> Juho Rosqvist <juho.rosqvist@mbnet.fi> writes:
> 
> > Now, the subject _should_ read:
> > Mutt and ÅåÄäÖö characters
> [snip]
> > I'm really at a loss as to what causes this problem, or how to fix it.
> > Help would be appreciated.
> 
> The problem is that if mail headers (especially to, from and subject)
> contain non US-Ascii characters (as your email does), then the mail
> program is supposed to use the mechanism described in RFC2047. Your
> emailer is not doing this, but is sending the 'raw' Scandinavian
> characters. As to how to fix it, I am sorry but I cannot be of any
> help.  

Thanks for the info on RFC2047. However, I'd like to believe that this is
not the cause of the problem. I may not have stressed enough the first 
time, that the problematic characters are garbled to what you see in the
subject line _as I type them_. So I see the subject in its 'final form'
immediately. Also, when I reply to a mail containing ä, ö et al in the
subject, and do not erase those characters, they get sent away just
fine.

So I'm thinking this is an input problem of mutt -- actually, I just 
noticed that it is present in each and every input request mutt makes.
-- 
gentoo-user@gentoo.org mailing list

Re: [gentoo-user] Mutt and C\x05C%C\x04C$C\x16C6 characters

[Juho Rosqvist]

On Mon, Apr 02, 2007 at 13:49:46 +0200, Michal 'vorner' Vaner wrote:
> Hello
> 
> On Sat, Mar 31, 2007 at 06:31:34AM +0100, Graham Murray wrote:
> > Juho Rosqvist <juho.rosqvist@mbnet.fi> writes:
> > 
> > > Now, the subject _should_ read:
> > > Mutt and ÅåÄäÖö characters
> > [snip]
> > > I'm really at a loss as to what causes this problem, or how to fix it.
> > > Help would be appreciated.
> > 
> > The problem is that if mail headers (especially to, from and subject)
> > contain non US-Ascii characters (as your email does), then the mail
> > program is supposed to use the mechanism described in RFC2047. Your
> > emailer is not doing this, but is sending the 'raw' Scandinavian
> > characters. As to how to fix it, I am sorry but I cannot be of any
> > help.  
> 
> I do not think this would be the problem, since MUTT does encode them
> (at last my mutt with Czech characters and utf-8 charset). I would try
> some other TUI application like mc or links. Vim handles the input
> directly AFAIK, but these use readline library for it.

I agree that my problem is probably input related. By your reply I
presume that mutt uses readline for input; is this correct?

Midnight Commander displays åäö correctly, but not the corresponding
capital letters ÅÄÖ, which is confusing to say the least. Links works 
after switching to UTF-8 I/O and ISO 8859-1 character set (UTF-8 is not
available?), although some symbols are not displayed, e.g. the euro
currency symbol € is replaced by EUR -- but this is due to the symbol
missing from the charset, I believe.
-- 
gentoo-user@gentoo.org mailing list

Re: [gentoo-user] Mutt and C\x05C%C\x04C$C\x16C6 characters

[Michal 'vorner' Vaner]

[-- Attachment #1: Type: text/plain, Size: 1577 bytes --]

On Mon, Apr 02, 2007 at 09:10:58PM +0300, Juho Rosqvist wrote:
> On Mon, Apr 02, 2007 at 13:49:46 +0200, Michal 'vorner' Vaner wrote:
> > Hello
> > 
> > On Sat, Mar 31, 2007 at 06:31:34AM +0100, Graham Murray wrote:
> > > Juho Rosqvist <juho.rosqvist@mbnet.fi> writes:
> > > 
> > > > Now, the subject _should_ read:
> > > > Mutt and ÅåÄäÖö characters
> > > [snip]
> > > > I'm really at a loss as to what causes this problem, or how to fix it.
> > > > Help would be appreciated.
> > > 
> > > The problem is that if mail headers (especially to, from and subject)
> > > contain non US-Ascii characters (as your email does), then the mail
> > > program is supposed to use the mechanism described in RFC2047. Your
> > > emailer is not doing this, but is sending the 'raw' Scandinavian
> > > characters. As to how to fix it, I am sorry but I cannot be of any
> > > help.  
> > 
> > I do not think this would be the problem, since MUTT does encode them
> > (at last my mutt with Czech characters and utf-8 charset). I would try
> > some other TUI application like mc or links. Vim handles the input
> > directly AFAIK, but these use readline library for it.
> 
> I agree that my problem is probably input related. By your reply I
> presume that mutt uses readline for input; is this correct?

I'm not sure, it is just a guess. It is only the direction I would try
first if I had this problem, nothing definite...

-- 
Wait few minutes before opening this email. The temperature difference 
could lead to vapour condensation.

Michal "vorner" Vaner

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-user] Mutt and C\x05C%C\x04C$C\x16C6 characters

[Juho Rosqvist]

On Mon, Apr 02, 2007 at 22:18:46 +0200, Michal 'vorner' Vaner wrote:
> On Mon, Apr 02, 2007 at 09:10:58PM +0300, Juho Rosqvist wrote:
> > On Mon, Apr 02, 2007 at 13:49:46 +0200, Michal 'vorner' Vaner wrote:
> > 
> > I agree that my problem is probably input related. By your reply I
> > presume that mutt uses readline for input; is this correct?
> 
> I'm not sure, it is just a guess. It is only the direction I would try
> first if I had this problem, nothing definite...

I checked the ebuilds, and sys-libs/readline doesn't appear to be a
dependency of mutt. Not even an indirect dep. Although mutt's input
behaves a _lot_ like readline.
-- 
gentoo-user@gentoo.org mailing list

Re: [gentoo-user] Mutt and C\x05C%C\x04C$C\x16C6 characters

[Juho Rosqvist]

OK, I solved the problem.

For the record: it's worth checking whether the problem persists with a
near-empty muttrc. It did not, so I went through the rc file with a fine
comb once again. The culprit turned out to be this line:

set meta_key = yes

Unsetting the variable removes the problem. In hindsight, it's pretty
obvious, as explained by the Mutt manual:

--- quote ---
meta_key
Type: boolean
Default: no
If set, forces Mutt to interpret keystrokes with the high bit (bit 8) set
as if the user had pressed the ESC key and whatever key remains after having
the high bit removed. For example, if the key pressed has an ASCII value of
0xf4, then this is treated as if the user had pressed ESC then ``x''. This is
because the result of removing the high bit from ``0xf4'' is ``0x74'', which
is the ASCII character ``x''.
--- end quote ---

Thank you for pushing me towards the solution. I might have switched to
a mail client that 'sucks more' without your support ;-)

Regards, Juho

-- 
When you have eliminated the impossible, whatever remains, however
improbable, must be the truth. -- Sir Arthur Conan Doyle
-- 
gentoo-user@gentoo.org mailing list