[gentoo-portage-dev] changelog encoding

[gentoo-portage-dev] changelog encoding

[Brian]

What is the official encoding method(s) for the changelogs.   It has
been reported that porthole often fails getting the changelogs due to
the encoding.  Currently it is assuming ascii.   Many are reported to be
iso-8859-1.
-- 
Brian <dol-sen@telus.net>


--
gentoo-portage-dev@gentoo.org mailing list

Re: [gentoo-portage-dev] changelog encoding

[Marius Mauch]

[-- Attachment #1: Type: text/plain, Size: 367 bytes --]

On 10/07/04  Brian wrote:

> What is the official encoding method(s) for the changelogs.   It has
> been reported that porthole often fails getting the changelogs due to
> the encoding.  Currently it is assuming ascii.   Many are reported to
> be iso-8859-1.

I don't think we have an official encoding, but I think ciaranm knows a
bit more about that issue.

Marius

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-portage-dev] changelog encoding

[Ciaran McCreesh]

[-- Attachment #1: Type: text/plain, Size: 751 bytes --]

[ hopefully on the list and able to post to it now, if not please
forward ]

On Fri, 8 Oct 2004 16:26:33 +0000 Luke-Jr <luke-jr@utopios.org> wrote:
| > Yup. We *need* to have an official encoding. Reason being, at least
| > one developer has a non-(ASCII as in characters 0..126 only)
| 
| ASCII defines 128 characters: 0-127
| Why cut the last off?

As I recall, 127 is flaky. Come to think of it, so is 0-31ish as well.
So maybe I should've said [a-zA-Z0-9\-_,.<>?/\\;:'@#~\]{}\+="$%^&* ] or
something... Basically, anything even the slightest bit flaky, plus
newlines, is prone to explode.

-- 
Ciaran McCreesh : Gentoo Developer (Sparc, MIPS, Vim, Fluxbox)
Mail            : ciaranm at gentoo.org
Web             : http://dev.gentoo.org/~ciaranm


[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-portage-dev] changelog encoding

[George Shapovalov]

Well, all the docs are gotta be UTF-8, and that was official since that big 
doc reorganization back 2+ years ago :). This would be a good precedent to 
apply the same policy" for the rest of the dok-like and even generic stuff 
(why not even ebuilds). Besides, while I don't this this was "officially 
stamped" yet, I seem to remember UTF-8 mentioned in extended 
metadata/ChangeLog discussions..

George

On Thursday 07 October 2004 21:19, Brian wrote:
> What is the official encoding method(s) for the changelogs.   It has
> been reported that porthole often fails getting the changelogs due to
> the encoding.  Currently it is assuming ascii.   Many are reported to be
> iso-8859-1.


--
gentoo-portage-dev@gentoo.org mailing list

Re: [gentoo-portage-dev] changelog encoding

[Ed Grimm]

On Fri, 8 Oct 2004, Ciaran McCreesh wrote:
> On Fri, 8 Oct 2004 16:26:33 +0000 Luke-Jr <luke-jr@utopios.org> wrote:
>|> Yup. We *need* to have an official encoding. Reason being, at least
>|> one developer has a non-(ASCII as in characters 0..126 only)
>|
>| ASCII defines 128 characters: 0-127
>| Why cut the last off?
>
> As I recall, 127 is flaky. Come to think of it, so is 0-31ish as well.
> So maybe I should've said [a-zA-Z0-9\-_,.<>?/\\;:'@#~\]{}\+="$%^&* ] or
> something... Basically, anything even the slightest bit flaky, plus
> newlines, is prone to explode.

I hope you like long lines.

Might I propose [\n\t\r -~]?

However, I suggest that any programs that cannot deal with characters
0-127 are broken and should be fixed.  Admittedly, outputing 27 to the
terminal raw could trigger a security hole in many terminals, and
several other terminals have problems with another character (I'm
vaguely thinking 17, but not sure.)

(The preceeding paragraph should not be taken as an alternate solution,
but one to combine with restricting characters to the above recommended
9, 10, 13, 32-126.)

Ed

--
gentoo-portage-dev@gentoo.org mailing list

Re: [gentoo-portage-dev] changelog encoding

[Ciaran McCreesh]

[-- Attachment #1: Type: text/plain, Size: 880 bytes --]

On Fri, 8 Oct 2004 20:05:16 -0500 (EST) Ed Grimm
<paranoid@gentoo.evolution.tgape.org> wrote:
| On Fri, 8 Oct 2004, Ciaran McCreesh wrote:
| > As I recall, 127 is flaky. Come to think of it, so is 0-31ish as
| > well. So maybe I should've said
| > [a-zA-Z0-9\-_,.<>?/\\;:'@#~\]{}\+="$%^&* ] or something...
| > Basically, anything even the slightest bit flaky, plus newlines, is
| > prone to explode.
| 
| I hope you like long lines.

Well, remember that there's no standard way of doing newlines. Or that
there're at least three standards, depending upon how you look at it. 

Eh, not that this is really relevant anyway. All that matters is that
ASCII is insufficient and that UTF-8 is most likely the best
alternative.

-- 
Ciaran McCreesh : Gentoo Developer (Sparc, MIPS, Vim, Fluxbox)
Mail            : ciaranm at gentoo.org
Web             : http://dev.gentoo.org/~ciaranm


[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-portage-dev] changelog encoding

[Brian]

On Sat, 2004-10-09 at 06:53, Ciaran McCreesh wrote:
> On Fri, 8 Oct 2004 20:05:16 -0500 (EST) Ed Grimm
> <paranoid@gentoo.evolution.tgape.org> wrote:
> | On Fri, 8 Oct 2004, Ciaran McCreesh wrote:
> | > As I recall, 127 is flaky. Come to think of it, so is 0-31ish as
> | > well. So maybe I should've said
> | > [a-zA-Z0-9\-_,.<>?/\\;:'@#~\]{}\+="$%^&* ] or something...
> | > Basically, anything even the slightest bit flaky, plus newlines, is
> | > prone to explode.
> | 
> | I hope you like long lines.
> 
> Well, remember that there's no standard way of doing newlines. Or that
> there're at least three standards, depending upon how you look at it. 
> 
> Eh, not that this is really relevant anyway. All that matters is that
> ASCII is insufficient and that UTF-8 is most likely the best
> alternative.

Thank you.

   I have changed porthole's code to try decode(utf_8), then try
decode(iso-8859-1).  Failing either of those Porthole will display an
unknown encoding error and to please report it to bugs.gentoo.org as
well as porthole's bug tracker.


-- 
Brian <dol-sen@telus.net>


--
gentoo-portage-dev@gentoo.org mailing list

Re: [gentoo-portage-dev] changelog encoding

[Ciaran McCreesh]

[-- Attachment #1: Type: text/plain, Size: 766 bytes --]

On Sat, 09 Oct 2004 08:24:38 -0700 Brian <dol-sen@telus.net> wrote:
|    I have changed porthole's code to try decode(utf_8), then try
| decode(iso-8859-1).  Failing either of those Porthole will display an
| unknown encoding error and to please report it to bugs.gentoo.org as
| well as porthole's bug tracker.

Since there's no documented standard... Could one of you portage people
please post a message to -dev saying "as of now, use UTF-8 for ebuilds
and changelogs if at all possible"? I can update app-vim/gentoo-syntax
and the default vimrc to encourange fileencoding to be uft-8 for these
things...

-- 
Ciaran McCreesh : Gentoo Developer (Sparc, MIPS, Vim, Fluxbox)
Mail            : ciaranm at gentoo.org
Web             : http://dev.gentoo.org/~ciaranm


[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-portage-dev] changelog encoding

[Marius Mauch]

[-- Attachment #1: Type: text/plain, Size: 869 bytes --]

On 10/09/04  Ciaran McCreesh wrote:

> On Sat, 09 Oct 2004 08:24:38 -0700 Brian <dol-sen@telus.net> wrote:
> |    I have changed porthole's code to try decode(utf_8), then try
> | decode(iso-8859-1).  Failing either of those Porthole will display
> | an unknown encoding error and to please report it to bugs.gentoo.org
> | as well as porthole's bug tracker.
> 
> Since there's no documented standard... Could one of you portage
> people please post a message to -dev saying "as of now, use UTF-8 for
> ebuilds and changelogs if at all possible"? I can update
> app-vim/gentoo-syntax and the default vimrc to encourange fileencoding
> to be uft-8 for these things...

No problem for ChangeLogs, but for ebuilds we probably need to make a
difference between the portage-visible parts and the normal bash stuff
as portage itself can only handle ascii internally.

Marius

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]