[gentoo-dev] Some suggestions

[gentoo-dev] Some suggestions

[David Sankel]

Hello gentoo developers,

  I would like to say great job to all of you.  Gentoo is an exceptional
distribution.  I have been using it for about 7 months now.  You all
should be very proud of yourselves for creating a very solid software
product.

  I am an independent contractor with several years experience in user
interaction with software systems.  I came up with a few suggestions for
the gentoo system.  It is my hope that you will find them interesting or
informative.

1)  etc-update changes for a more automated system update

etc-update allows you to automatically update (noted) etc files that one
never changed from their last emerge.  This could save a lot of 
maintenance time if it was put in a cron job to routinely do that after 
an "emerge sync;emerge -u world"
  The user then wouldn't have to look at all of the etc files to see 
if they were changed after every "emerge sync; emerge-u world".  

2)  make.conf updates to be more automated

Most gentoo users, I believe, modify this file.  This specific file changes
quite often with updates.  Since most users only modify the "USE" and "CFLAGS"
components, having an update that is automatic is plausible.  This feature
is a trade off between the integrity and consistency of the system verses
end-user maintenance time.

3) emerge -u world "NOTICE:" output changes.

When doing an "emerge -u world" several "NOTICE:"'s fly by on the screen. 
For many users, they go unread although they contain important information
in a lot of cases.  If these "NOTICE:"'s are cached and output at the end
of an "emerge -u world", their readership would have a dramatic increase. 
This would allow interested gentoo users to be more informed of their
system.

4) emerge -u world progress bar option

Most users probably care less about the compilation stages of their update
in comparison to the percentage of completion.  A progress bar in this
area would be a nice aesthetic and informative addition for most users. 
In order for this to happen, the output of the "emerge -u world" command
would probably have to be standardized with some flags to mark the
beginning and end of a compile.

5) A simple graphical front end to maintenance commands such as "emerge
sync", "emerge -u world", and "etc-update".

It would be a nice feature if users didn't have to commit these commands
to memory for regular maintenance.  The maintenance menu might have an
icon on all of the default desktops.  If this type of program was
implemented, it could be prominently displayed and made known for all new
users.  Perhaps the install documentation could use this tool as much as
possible.

6) A streamlined GUI install.

I'm sure this one has been brought up before.  I consider this the "1.0"
maker of the gentoo distribution.  In such an installer, I suggest that
the CFLAGS should not be modified by default.  It has been shown in
several places that optimizing these does not give a significant enough
performance increase.  It should stay, of course, as an option though.

7) Emerge -S and Emerge -s speed improvements.

I don't know why these commands perform as slow as they do.  My intuition
says that they could be an order of magnitude faster.  Perhaps a
reimplementation in C/C++ or a data format change could help.

Thanks for reading my comments and taking them into consideration.  Again,
I want to thank you, the gentoo developers, for doing such a great job on
the gentoo system.  I welcome any thoughts or comments on my suggestions
in this newsgroup or otherwise.

Sincerely,

David J. Sankel

camio@yahoo.com


--
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Some suggestions

[Douglas Russell]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Saturday 06 September 2003 7:05 pm, David Sankel wrote:
> Hello gentoo developers,
>
>   I would like to say great job to all of you.  Gentoo is an exceptional
> distribution.  I have been using it for about 7 months now.  You all
> should be very proud of yourselves for creating a very solid software
> product.
>
>   I am an independent contractor with several years experience in user
> interaction with software systems.  I came up with a few suggestions for
> the gentoo system.  It is my hope that you will find them interesting or
> informative.
>
> 1)  etc-update changes for a more automated system update
>
> etc-update allows you to automatically update (noted) etc files that one
> never changed from their last emerge.  This could save a lot of
> maintenance time if it was put in a cron job to routinely do that after
> an "emerge sync;emerge -u world"
>   The user then wouldn't have to look at all of the etc files to see
> if they were changed after every "emerge sync; emerge-u world".

You can't be serious!? If you don't examine those etc-update changes you could
seriously screw up your system.

> 2)  make.conf updates to be more automated
>
> Most gentoo users, I believe, modify this file.  This specific file changes
> quite often with updates.  Since most users only modify the "USE" and
> "CFLAGS" components, having an update that is automatic is plausible.  This
> feature is a trade off between the integrity and consistency of the system
> verses end-user maintenance time.

What about those of use with a load of other stuff in there. I just use the
interactive merge function of etc-update. Takes about 30 seconds to do
make.conf.

>
> 3) emerge -u world "NOTICE:" output changes.
>
> When doing an "emerge -u world" several "NOTICE:"'s fly by on the screen.
> For many users, they go unread although they contain important information
> in a lot of cases.  If these "NOTICE:"'s are cached and output at the end
> of an "emerge -u world", their readership would have a dramatic increase.
> This would allow interested gentoo users to be more informed of their
> system.

I believe their is a patch to do this somewhere around. Sorry I can't remember
where.

> 4) emerge -u world progress bar option
>
> Most users probably care less about the compilation stages of their update
> in comparison to the percentage of completion.  A progress bar in this
> area would be a nice aesthetic and informative addition for most users.
> In order for this to happen, the output of the "emerge -u world" command
> would probably have to be standardized with some flags to mark the
> beginning and end of a compile.

A progress bar would be difficult as its difficult to estimate how much code
is left to compile and/or how long that is going to take. It would probably
end up just saying 20 our of 24, 21 out of 24 etc. I personally prefer to see
the compilation so when it breaks I can see exactly where it was before it
broke.

> 5) A simple graphical front end to maintenance commands such as "emerge
> sync", "emerge -u world", and "etc-update".
>
> It would be a nice feature if users didn't have to commit these commands
> to memory for regular maintenance.  The maintenance menu might have an
> icon on all of the default desktops.  If this type of program was
> implemented, it could be prominently displayed and made known for all new
> users.  Perhaps the install documentation could use this tool as much as
> possible.

Well, there are programs whcih do some of the stuff you might be referring to
like kportage. They don't work on all desktops though, and they don't do all
of that.

> 6) A streamlined GUI install.
>
> I'm sure this one has been brought up before.  I consider this the "1.0"
> maker of the gentoo distribution.  In such an installer, I suggest that
> the CFLAGS should not be modified by default.  It has been shown in
> several places that optimizing these does not give a significant enough
> performance increase.  It should stay, of course, as an option though.

Your right, GUI discussion is all over the place. It would be useful to some
users certainly.

What places? Not modifying the CFLAGS? What do you suggest they should be left
at. I can assure you, your Athlon XP 2800+ or whatever will comparitivly
crawl with mcpu=i686.

> 7) Emerge -S and Emerge -s speed improvements.
>
> I don't know why these commands perform as slow as they do.  My intuition
> says that they could be an order of magnitude faster.  Perhaps a
> reimplementation in C/C++ or a data format change could help.

Their are faster versions. http://bugs.gentoo.org/show_bug.cgi?id=24756 for
example.

> Thanks for reading my comments and taking them into consideration.  Again,
> I want to thank you, the gentoo developers, for doing such a great job on
> the gentoo system.  I welcome any thoughts or comments on my suggestions
> in this newsgroup or otherwise.
>
> Sincerely,
>
> David J. Sankel
>
> camio@yahoo.com
>
>
> --
> gentoo-dev@gentoo.org mailing list

Puggy
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.2 (GNU/Linux)

iD8DBQE/WjO1XYnvgFdTojMRAn/uAKCbyE9q3OSY2E6Ckqgr+XTt81hcMwCgrN4C
VbMZGO4LLKRHVCGcNHr+5WY=
=bZ0l
-----END PGP SIGNATURE-----


--
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Some suggestions

[Douglas Russell]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

> > 3) emerge -u world "NOTICE:" output changes.
> >
> > When doing an "emerge -u world" several "NOTICE:"'s fly by on the screen.
> > For many users, they go unread although they contain important
> > information in a lot of cases.  If these "NOTICE:"'s are cached and
> > output at the end of an "emerge -u world", their readership would have a
> > dramatic increase. This would allow interested gentoo users to be more
> > informed of their system.
>
> I believe their is a patch to do this somewhere around. Sorry I can't
> remember where.

Here:
http://bugs.gentoo.org/show_bug.cgi?id=11359

Puggy...again. :-D

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.2 (GNU/Linux)

iD8DBQE/WjR1XYnvgFdTojMRApWDAKCeumfgkPxFunh2MhTaBhHhZfwBTQCg0eLm
H0QtLbZA0CqlO8LAxD7M8eg=
=zHGT
-----END PGP SIGNATURE-----


--
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Some suggestions

[Thomas de Grenier de Latour]

On Sat, 06 Sep 2003 13:05:16 -0500
David Sankel <sankeld@nonconformity.net> wrote:

> 1)  etc-update changes for a more automated system update

Try dispatch-conf (from recent portage versions). It does unmodified
files auto-upgrade, plus some other goodies.

> 2)  make.conf updates to be more automated

Why something special for this one? With dispatch-conf, you already have
automation for cvs headers. For the rest of the file, it's ok to do it
manually. If you've only changed USE and CFLAGS, then the merge should
really be straightforward (2 press on "l", the others on "r"), and it
has the advantage to show you what's new in portage (new features, etc.)

> 3) emerge -u world "NOTICE:" output changes.

I use a small script that parse emerge log files for that, but it's true
that a report at this end of an emerge would be nice.

> 4) emerge -u world progress bar option
> 
> the output of the "emerge -u world" command would probably have to be
> standardized with some flags to mark the beginning and end of a
> compile.

I would imagine one progress bar for the overall process, and one for
the current compilation. There is already some work that have been done
here for a "make" progress meter:
http://forums.gentoo.org/viewtopic.php?t=42346
But it was still buggy last time I checked.

> 5) A simple graphical front end to maintenance commands such as
> "emerge sync", "emerge -u world", and "etc-update".

I think there are already frontends for emerge (one for gnome and one
for kde if I remember well) that exist. Seen that somewhere on the forum
I guess.

> 6) A streamlined GUI install.

http://glis.sourceforge.net/ 

> 7) Emerge -S and Emerge -s speed improvements.
> 
> I don't know why these commands perform as slow as they do.  My
> intuition says that they could be an order of magnitude faster. 
> Perhaps a reimplementation in C/C++ or a data format change could
> help.

Caching the info from thousand files into a single one is enough. 
There are 2 approach: portage patch or separate tool:
http://gentoo.devel-net.org/download/emerge-fastsearch.patch
http://bugs.gentoo.org/show_bug.cgi?id=24756
 
-- 
TGL.

--
gentoo-dev@gentoo.org mailing list

[gentoo-dev] Re: Some suggestions

[David Sankel]

On Sat, 06 Sep 2003 20:21:20 +0100, Douglas Russell wrote:

Hello Douglas.  Thanks for your informed responses.

>> etc-update allows you to automatically update (noted) etc files that one
>> never changed from their last emerge.  This could save a lot of
>> maintenance time if it was put in a cron job to routinely do that after
>> an "emerge sync;emerge -u world"
>>   The user then wouldn't have to look at all of the etc files to see
>> if they were changed after every "emerge sync; emerge-u world".
> 
> You can't be serious!? If you don't examine those etc-update changes you could
> seriously screw up your system.

I was afraid that it wouldn't be clear what I meant by this.  I am not
suggesting that all files could be automatically overwritten, only files
that were not modified from the default by the user.  For example the
file:

/etc/X11/chooser.sh

was never changed by a given user on a given system.  It is the default for
the version they have.  When a revision to the default is discovered, the
system will flag that file as being one that the user hasn't specialized. 
In etc-update, there could be an option to update all flagged files
automatically.  Does that make more sense?

>> Most gentoo users, I believe, modify this file.  This specific file changes
>> quite often with updates.  Since most users only modify the "USE" and
>> "CFLAGS" components, having an update that is automatic is plausible.  This
>> feature is a trade off between the integrity and consistency of the system
>> verses end-user maintenance time.
> 
> What about those of use with a load of other stuff in there. I just use the
> interactive merge function of etc-update. Takes about 30 seconds to do
> make.conf.

I certainly would not suggest that the flexibility of the current system
be undermined by such a feature.  There should be an option for users such
as yourself to use the system as is.

>> Most users probably care less about the compilation stages of their update
>> in comparison to the percentage of completion.  A progress bar in this
>> area would be a nice aesthetic and informative addition for most users.
>> In order for this to happen, the output of the "emerge -u world" command
>> would probably have to be standardized with some flags to mark the
>> beginning and end of a compile.
> 
> A progress bar would be difficult as its difficult to estimate how much code
> is left to compile and/or how long that is going to take. It would probably
> end up just saying 20 our of 24, 21 out of 24 etc. I personally prefer to see
> the compilation so when it breaks I can see exactly where it was before it
> broke.

You are absolutely correct.  Having something simple like 20/24 (currently
working on package XXX) is probably the best such a progress bar could do.
If the compilation does break, and that has been very rare in my
experience, the program could output the log of that failed compile for 
inspection.  I think this feature, being only an option, would be an 
enhancement that wouldn't remove any features you are interested in.

> What places? Not modifying the CFLAGS? What do you suggest they should be left
> at. I can assure you, your Athlon XP 2800+ or whatever will comparitivly
> crawl with mcpu=i686.

I suggest they be left at whatever the non-source based distributions
leave them at.  Perhaps I am misinformed about how much improvement one
gets with aggressive optimization flags.  Could you point me somewhere in
the right direction?

David J. Sankel


--
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Some suggestions

[Brian Jackson]

On Saturday 06 September 2003 01:05 pm, David Sankel wrote:
> Hello gentoo developers,
> 
>   I would like to say great job to all of you.  Gentoo is an exceptional
> distribution.  I have been using it for about 7 months now.  You all
> should be very proud of yourselves for creating a very solid software
> product.
> 
>   I am an independent contractor with several years experience in user
> interaction with software systems.  I came up with a few suggestions for
> the gentoo system.  It is my hope that you will find them interesting or
> informative.
> 
> 1)  etc-update changes for a more automated system update
> 
> etc-update allows you to automatically update (noted) etc files that one
> never changed from their last emerge.  This could save a lot of 
> maintenance time if it was put in a cron job to routinely do that after 
> an "emerge sync;emerge -u world"
>   The user then wouldn't have to look at all of the etc files to see 
> if they were changed after every "emerge sync; emerge-u world".  

been suggested, don't know what the status is, there's probably a bug that's 
tracking the progress

> 
> 2)  make.conf updates to be more automated
> 
> Most gentoo users, I believe, modify this file.  This specific file changes
> quite often with updates.  Since most users only modify the "USE" and 
"CFLAGS"
> components, having an update that is automatic is plausible.  This feature
> is a trade off between the integrity and consistency of the system verses
> end-user maintenance time.
> 
> 3) emerge -u world "NOTICE:" output changes.
> 
> When doing an "emerge -u world" several "NOTICE:"'s fly by on the screen. 
> For many users, they go unread although they contain important information
> in a lot of cases.  If these "NOTICE:"'s are cached and output at the end
> of an "emerge -u world", their readership would have a dramatic increase. 
> This would allow interested gentoo users to be more informed of their
> system.

been suggested numerous times, I believe it's being worked on

> 
> 4) emerge -u world progress bar option
> 
> Most users probably care less about the compilation stages of their update
> in comparison to the percentage of completion.  A progress bar in this
> area would be a nice aesthetic and informative addition for most users. 
> In order for this to happen, the output of the "emerge -u world" command
> would probably have to be standardized with some flags to mark the
> beginning and end of a compile.

it's hard to say what progress something is at, this would require some way to 
measure a packages relative size, but we do have an "emerging package x/t" 
message

> 
> 5) A simple graphical front end to maintenance commands such as "emerge
> sync", "emerge -u world", and "etc-update".
> 
> It would be a nice feature if users didn't have to commit these commands
> to memory for regular maintenance.  The maintenance menu might have an
> icon on all of the default desktops.  If this type of program was
> implemented, it could be prominently displayed and made known for all new
> users.  Perhaps the install documentation could use this tool as much as
> possible.

kind of tough considering the sheer number of desktop environments/window 
managers out there, but I know there is one for kde and one for gnome

> 
> 6) A streamlined GUI install.
> 
> I'm sure this one has been brought up before.  I consider this the "1.0"
> maker of the gentoo distribution.  In such an installer, I suggest that
> the CFLAGS should not be modified by default.  It has been shown in
> several places that optimizing these does not give a significant enough
> performance increase.  It should stay, of course, as an option though.

being worked on
glis.sf.net
they are doing a scripted backend, that eventually can have whatever kind of 
front end

> 
> 7) Emerge -S and Emerge -s speed improvements.
> 
> I don't know why these commands perform as slow as they do.  My intuition
> says that they could be an order of magnitude faster.  Perhaps a
> reimplementation in C/C++ or a data format change could help.

also been mentioned on too many occasions to count. portage is not going to be 
rewritten in c/c++, even if it was it wouldn't help, most of the time is i/o. 
there are people working on modularizing the portage code to make it easier 
to change out the backend, so some day you may be able to use a db, sql, 
whatever backend.

> 
> Thanks for reading my comments and taking them into consideration.  Again,
> I want to thank you, the gentoo developers, for doing such a great job on
> the gentoo system.  I welcome any thoughts or comments on my suggestions
> in this newsgroup or otherwise.

We always welcome new suggestions, thanks for your input.

--iggy

> 
> Sincerely,
> 
> David J. Sankel

-- 
Home -- http://www.brianandsara.net
Gentoo -- http://gentoo.brianandsara.net

--
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Some suggestions

[Thomas de Grenier de Latour]

On Sat, 6 Sep 2003 21:42:17 +0200
Thomas de Grenier de Latour <degrenier@easyconnect.fr> wrote:

> > 7) Emerge -S and Emerge -s speed improvements.

Oh, and also "emerge esearch".

-- 
TGL.

--
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Some suggestions

[Marius Mauch]

[-- Attachment #1: Type: text/plain, Size: 5592 bytes --]

On 09/06/03  David Sankel wrote:

> Hello gentoo developers,
> 
>   I would like to say great job to all of you.  Gentoo is an
>   exceptional
> distribution.  I have been using it for about 7 months now.  You all
> should be very proud of yourselves for creating a very solid software
> product.
> 
>   I am an independent contractor with several years experience in user
> interaction with software systems.  I came up with a few suggestions
> for the gentoo system.  It is my hope that you will find them
> interesting or informative.
> 
> 1)  etc-update changes for a more automated system update
> 
> etc-update allows you to automatically update (noted) etc files that
> one never changed from their last emerge.  This could save a lot of 
> maintenance time if it was put in a cron job to routinely do that
> after an "emerge sync;emerge -u world"
>   The user then wouldn't have to look at all of the etc files to see 
> if they were changed after every "emerge sync; emerge-u world".  

You don't really use emerge -u world in a cron job? emerge -u world and
etc-update can bring severe changes to your system and might create
problems if used without caution, so I strongly discourage anyone from
running them in a cron-job.
But I agree that etc-update needs some improvements, haven't looked at
dispatch-conf yet, maybe it does the job better.

> 2)  make.conf updates to be more automated
> 
> Most gentoo users, I believe, modify this file.  This specific file
> changes quite often with updates.  Since most users only modify the
> "USE" and "CFLAGS" components, having an update that is automatic is
> plausible.  This feature is a trade off between the integrity and
> consistency of the system verses end-user maintenance time.

make.conf updates are annoying, maybe we can make an exception in
etc-update for it to handle it different (only look for new variables).

> 3) emerge -u world "NOTICE:" output changes.
> 
> When doing an "emerge -u world" several "NOTICE:"'s fly by on the
> screen. For many users, they go unread although they contain important
> information in a lot of cases.  If these "NOTICE:"'s are cached and
> output at the end of an "emerge -u world", their readership would have
> a dramatic increase. This would allow interested gentoo users to be
> more informed of their system.

There are several bugs open about this.

> 4) emerge -u world progress bar option
> 
> Most users probably care less about the compilation stages of their
> update in comparison to the percentage of completion.  A progress bar
> in this area would be a nice aesthetic and informative addition for
> most users. In order for this to happen, the output of the "emerge -u
> world" command would probably have to be standardized with some flags
> to mark the beginning and end of a compile.

A accurate progress bar is nearly impossible as compile times differ
from package to package (and are independent of package size too). The
only thing that would be rather easy to implement is a progrssbar that
only advance when a package is finished and even then we need something
to keep it on the screen, so ncurses or something similar would be
needed.

> 5) A simple graphical front end to maintenance commands such as
> "emerge sync", "emerge -u world", and "etc-update".

kportage, portagemaster and there are several others not in the tree
yet.
 
> It would be a nice feature if users didn't have to commit these
> commands to memory for regular maintenance.  The maintenance menu
> might have an icon on all of the default desktops.  If this type of
> program was implemented, it could be prominently displayed and made
> known for all new users.  Perhaps the install documentation could use
> this tool as much as possible.

I disagree, every user should know the basic system management commands.
We really should not rely on GUI tools. It would just add another
possible point of failure.

> 6) A streamlined GUI install.
> 
> I'm sure this one has been brought up before.  I consider this the
> "1.0" maker of the gentoo distribution.  In such an installer, I
> suggest that the CFLAGS should not be modified by default.  It has
> been shown in several places that optimizing these does not give a
> significant enough performance increase.  It should stay, of course,
> as an option though.

Most people (including me) don't like that, there are other distros for
that. One approach I like is glis (gentoo linux install script), beause
it doesn't dumb down the install process.
Search the forums, there are several threads about this issue.

> 7) Emerge -S and Emerge -s speed improvements.
> 
> I don't know why these commands perform as slow as they do.  My
> intuition says that they could be an order of magnitude faster. 
> Perhaps a reimplementation in C/C++ or a data format change could
> help.

They are slow because they have to access several thousand files. Search
the forums or bugzilla for esearch, that is a script using a static
index to improve search times to about one second.
A rewrite in C/C++ would only cost a lot of time and won't bring much in
terms of speed (this has come up several times). One thing that has some
potential to improve the speed is to use a database for the portage
tree, but that won't happen anytime soon. There is the portagesql
project at breakmygentoo.net that tries to achieve such a solution,
haven't tried it yet.

Marius

-- 
Public Key at http://www.genone.de/info/gpg-key.pub

In the beginning, there was nothing. And God said, 'Let there be
Light.' And there was still nothing, but you could see a bit better.

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-dev] Some suggestions

[Phil Richards]

On 2003-09-06, Thomas de Grenier de Latour <degrenier@easyconnect.fr> wrote:
>  On Sat, 06 Sep 2003 13:05:16 -0500
>  David Sankel <sankeld@nonconformity.net> wrote:
> > 1)  etc-update changes for a more automated system update
>  Try dispatch-conf (from recent portage versions). It does unmodified
>  files auto-upgrade, plus some other goodies.

Erm, dispatch-conf?  What is it?  What does it do?

I don't like running commands as root that neither have a man page nor
respond to "-h" or "--help".  Nor am I particularly informed when I
run it (after a cursory look at the code) and get:

dispatch-conf: Config archive dir [/etc/config-archive] must exist; fatal

Ok, I'm off to the forums to see if I can find out what the command
is actually supposed to do.  (10 minute pause.)  Nope, still none the
wiser.

phil
-- 
change name to "phil" for email


--
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Some suggestions

[Thomas de Grenier de Latour]

On Sat,  6 Sep 2003 21:23:24 +0100 (BST)
Phil Richards <news@derived-software.ltd.uk> wrote:

> Erm, dispatch-conf?  What is it?  What does it do?

Python replacement for etc-update:
http://bugs.gentoo.org/show_bug.cgi?id=14079

It's true that some man page / --help output would be nice.

-- 
TGL.

--
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Some suggestions

[Thomas de Grenier de Latour]

On Sat, 6 Sep 2003 21:50:42 +0200
Marius Mauch <genone@genone.de> wrote:

> 
> A accurate progress bar is nearly impossible as compile times differ
> from package to package 

Maybe maintainers could fill in the ebuilds a kind of approximative
compile time from their experience, which would be relative to a
well know reference time (a kernel compilation with default options, or
something like this). It doesn't need to be very accurate.


-- 
TGL.

--
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Some suggestions

[Douglas Russell]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Saturday 06 September 2003 9:46 pm, Thomas de Grenier de Latour wrote:
> On Sat, 6 Sep 2003 21:50:42 +0200
>
> Marius Mauch <genone@genone.de> wrote:
> > A accurate progress bar is nearly impossible as compile times differ
> > from package to package
>
> Maybe maintainers could fill in the ebuilds a kind of approximative
> compile time from their experience, which would be relative to a
> well know reference time (a kernel compilation with default options, or
> something like this). It doesn't need to be very accurate.

This has also been discussed recently, I'm not sure if there was a resolution. 
I think it came down to what was happening about gentoo stats...as of course, 
an approx value on a Athlon XP 2800+ with a gig of RAM is not going to help 
someone on a P133 with 32 MBs of RAM. Stats would allow a lookup against 
previous emerges done on similar spec'd machines.

Puggy
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.2 (GNU/Linux)

iD8DBQE/WknzXYnvgFdTojMRAsHGAKDOcLOYbrkeFw8D7sQ69EeW/GLUkwCcC4Zx
nE1woeHGDeT1Y0LAdxbIvTM=
=hCJf
-----END PGP SIGNATURE-----


--
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Some suggestions

[Marius Mauch]

On 09/06/03  Douglas Russell wrote:

> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
> 
> On Saturday 06 September 2003 9:46 pm, Thomas de Grenier de Latour
> wrote:
> > On Sat, 6 Sep 2003 21:50:42 +0200
> >
> > Marius Mauch <genone@genone.de> wrote:
> > > A accurate progress bar is nearly impossible as compile times
> > > differ from package to package
> >
> > Maybe maintainers could fill in the ebuilds a kind of approximative
> > compile time from their experience, which would be relative to a
> > well know reference time (a kernel compilation with default options,
> > or something like this). It doesn't need to be very accurate.
> 
> This has also been discussed recently, I'm not sure if there was a
> resolution. I think it came down to what was happening about gentoo
> stats...as of course, an approx value on a Athlon XP 2800+ with a gig
> of RAM is not going to help someone on a P133 with 32 MBs of RAM.
> Stats would allow a lookup against previous emerges done on similar
> spec'd machines.

And don't forget the impact of USE flags.

-- 
Public Key at http://www.genone.de/info/gpg-key.pub

In the beginning, there was nothing. And God said, 'Let there be
Light.' And there was still nothing, but you could see a bit better.

--
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Re: Some suggestions

[Chris Gianelloni]

[-- Attachment #1: Type: text/plain, Size: 2817 bytes --]

On Sat, 2003-09-06 at 15:45, David Sankel wrote:
> I was afraid that it wouldn't be clear what I meant by this.  I am not
> suggesting that all files could be automatically overwritten, only files
> that were not modified from the default by the user.  For example the
> file:
> 
> /etc/X11/chooser.sh
> 
> was never changed by a given user on a given system.  It is the default for
> the version they have.  When a revision to the default is discovered, the
> system will flag that file as being one that the user hasn't specialized. 
> In etc-update, there could be an option to update all flagged files
> automatically.  Does that make more sense?

I think this is a great idea.  I am tired of having to update files
which I haven't even touched simply because they have changed a little
since the last merge.

> You are absolutely correct.  Having something simple like 20/24 (currently
> working on package XXX) is probably the best such a progress bar could do.
> If the compilation does break, and that has been very rare in my
> experience, the program could output the log of that failed compile for 
> inspection.  I think this feature, being only an option, would be an 
> enhancement that wouldn't remove any features you are interested in.

This is somewhat done now via the xterm titles.  I personally find it
annoying and turn it off.  Having a progress bar for portage would annoy
me to no end since as a developer I like to see what is going on with my
compiles.  If it were implemented as a FEATURE, I would have no problem
with it.  I also don't think it would be useful at all except in the
case of merging multiple ebuilds at once.

> I suggest they be left at whatever the non-source based distributions
> leave them at.  Perhaps I am misinformed about how much improvement one
> gets with aggressive optimization flags.  Could you point me somewhere in
> the right direction?

http://www.linuxgazette.com/issue88/piszcz.html

Quite simply, test it yourself.  The best way to test is to use a x86
GRP CD and install Gentoo.  This is approximately equivalent to the
build flags on other binary distributions.  Run a bunch of benchmarks. 
Next reinstall the same machine using aggressive CFLAGS and run the same
benchmarks.  You'll see quite a dramatic difference in many things.  The
real thing is to realize what could possibly change by optimization. 
Binary size is usually larger with optimized code, since it is designed
for fast execution rather than binary size.  For example, a I/O
benchmark would be generally worthless to test the speed increases of
optimization, since you would be limited much more by the hardware than
the code itself.

-- 
Chris Gianelloni
Developer, Gentoo Linux
Games Team

Is your power animal a penguin?

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-dev] Some suggestions

[Chris Gianelloni]

[-- Attachment #1: Type: text/plain, Size: 713 bytes --]

On Sat, 2003-09-06 at 16:46, Thomas de Grenier de Latour wrote:
> On Sat, 6 Sep 2003 21:50:42 +0200
> Marius Mauch <genone@genone.de> wrote:
> 
> > 
> > A accurate progress bar is nearly impossible as compile times differ
> > from package to package 
> 
> Maybe maintainers could fill in the ebuilds a kind of approximative
> compile time from their experience, which would be relative to a
> well know reference time (a kernel compilation with default options, or
> something like this). It doesn't need to be very accurate.

We could always use LFS's BU (bash units) system of measure for
progress.

-- 
Chris Gianelloni
Developer, Gentoo Linux
Games Team

Is your power animal a penguin?

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-dev] Some suggestions

[Brian Harring]

On Saturday, September 6, 2003, at 03:46 PM, Thomas de Grenier de 
Latour wrote:

> On Sat, 6 Sep 2003 21:50:42 +0200
> Marius Mauch <genone@genone.de> wrote:
>
>>
>> A accurate progress bar is nearly impossible as compile times differ
>> from package to package
>
> Maybe maintainers could fill in the ebuilds a kind of approximative
> compile time from their experience, which would be relative to a
> well know reference time (a kernel compilation with default options, or
> something like this). It doesn't need to be very accurate.
Eh, wouldn't hold or be particularly accurate, mainly since I/O, proc 
speed, and available memory (let alone if another job is running in the 
background and hogging cycles) are too many variables (imo) to try and 
factor out.
Someone a while back had a setup such that they parsed the makefile, 
figuring out the number of actions (gcc calls, ar calls, mv/cp/install 
commands), and tracked progress that way.  Strikes me as the better 
way, although some packages weren't able to be parsed correctly 
resulting in a compilation progress reading at rather off values like 
1100% and counting...
~bdh

>
>
> -- 
> TGL.
>
> --
> gentoo-dev@gentoo.org mailing list
>


--
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Some suggestions

[Thomas de Grenier de Latour]

On Sat, 06 Sep 2003 16:58:39 -0500
Brian Harring <bdharring@wisc.edu> wrote:

> Eh, wouldn't hold or be particularly accurate, mainly since I/O, proc 
> speed, and available memory (let alone if another job is running in
> the background and hogging cycles) are too many variables (imo) to try
> and factor out.

I still think it it would be accurate enough for an "emerge -u world"
progress indicator. The goal is only to show that sometimes, while you
are still on update 1/10 after 2 hours, it doesn't mean 18 hours remain.

> Someone a while back had a setup such that they parsed the makefile, 
> figuring out the number of actions (gcc calls, ar calls, mv/cp/install
> commands), and tracked progress that way.  Strikes me as the better 
> way, although some packages weren't able to be parsed correctly 

I've post a link my previous to a forum thread about this feature. Sure
it is what should be used (if it can be debugged) for a particular
package compilation progress bar.

-- 
TGL.

--
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Some suggestions

[Steven Elling]

On Saturday 06 September 2003 13:05, David Sankel wrote:
> 2)  make.conf updates to be more automated
>
> Most gentoo users, I believe, modify this file.  This specific file
> changes quite often with updates.  Since most users only modify the "USE"
> and "CFLAGS" components, having an update that is automatic is plausible.
>  This feature is a trade off between the integrity and consistency of the
> system verses end-user maintenance time.

Requiring portage updates to make.conf at all has always bugged me.  The 
file is meant to contain custom settings for portage and to append to or 
override variables in make.globals and the defaults.  It should not hold 
all the documentation for make.conf.  It should not hold all the 
defaults... that's what make.globals and the defaults are for.

Why is all the documentation on make.conf in make.conf anyway?  Shouldn't it 
be in make.globals or better yet the man page?

make.conf is used for system customization and, as such, portage should 
leave it alone.  When portage is installed on the drive for the first time 
it should not create make.conf.  Portage should leave it up to the 
admin/user of the box to create the file.


--
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Some suggestions (SUMMARY?)

[Jason Stubbs]

I'm just a user, not even running a server with Gentoo. My thoughts on your 
suggestions are aligned with most of the devs on almost every point, so I'll 
put my own thoughts and add relevant bits from other threads.

On Sunday 07 September 2003 03:05, David Sankel wrote:
> 1)  etc-update changes for a more automated system update
>
> etc-update allows you to automatically update (noted) etc files that one
> never changed from their last emerge.  This could save a lot of
> maintenance time if it was put in a cron job to routinely do that after
> an "emerge sync;emerge -u world"
>   The user then wouldn't have to look at all of the etc files to see
> if they were changed after every "emerge sync; emerge-u world".

I've had this same idea myself. It should be fairly easy to do given 
/var/db/pkg/*/*/CONTENTS - that's got the modification date/time in there 
encoded, right?

A python script to address this issue named dispatch-conf has been written and 
is already included in sys-apps/portage. Details can be found at 
http://bugs.gentoo.org/show_bug.cgi?id=14079.

> 2)  make.conf updates to be more automated
>
> Most gentoo users, I believe, modify this file.  This specific file changes
> quite often with updates.  Since most users only modify the "USE" and
> "CFLAGS" components, having an update that is automatic is plausible.  This
> feature is a trade off between the integrity and consistency of the system
> verses end-user maintenance time.

Personally, I'm against automatic updates of make.conf, the main reason being 
that important updates to portage's functionality are usually documented 
there (eventually). While some users would like this functionality, the work 
to implement it does not match the benefit.

Functionality such as this would belong in a (optional!) script that would be 
able to update all configuration files. That sort of thing would be way down 
the track, though. My reasoning on this is that automatic updates of 
make.conf should only be necessary for the layman whom would have trouble 
updating almost any configuration file. Presently, Gentoo does not support 
this class of user.

> 3) emerge -u world "NOTICE:" output changes.
>
> When doing an "emerge -u world" several "NOTICE:"'s fly by on the screen.
> For many users, they go unread although they contain important information
> in a lot of cases.  If these "NOTICE:"'s are cached and output at the end
> of an "emerge -u world", their readership would have a dramatic increase.
> This would allow interested gentoo users to be more informed of their
> system.

There is a patch to portage available at 
http://bugs.gentoo.org/show_bug.cgi?id=11359. Once all the bugs are worked 
out, integration into official portage is likely.

> 4) emerge -u world progress bar option
>
> Most users probably care less about the compilation stages of their update
> in comparison to the percentage of completion.  A progress bar in this
> area would be a nice aesthetic and informative addition for most users.
> In order for this to happen, the output of the "emerge -u world" command
> would probably have to be standardized with some flags to mark the
> beginning and end of a compile.

There are two attempts at this at 
http://forums.gentoo.org/viewtopic.php?t=42346 and 
. Both are buggy and work does not seem to be continuing.

> 5) A simple graphical front end to maintenance commands such as "emerge
> sync", "emerge -u world", and "etc-update".
>
> It would be a nice feature if users didn't have to commit these commands
> to memory for regular maintenance.  The maintenance menu might have an
> icon on all of the default desktops.  If this type of program was
> implemented, it could be prominently displayed and made known for all new
> users.  Perhaps the install documentation could use this tool as much as
> possible.

There is fairly full-featured work done on this already. Two packages for kde 
are app-portage/kemerge and app-portage/kportage. There is also one in java - 
app-portage/portagemaster. I believe there to be work done on a gnome 
front-end but I cannot find it in portage.

As to having it on the desktop by default, this is definately out of line with 
Gentoo's philosophy - Gentoo would have to make a decision for the user. 
There are also some technical aspects; some WMs don't even have a desktop, 
many installations don't have X11, etc.

Slightly unrelated but work is being done on a (optional!) unified menu 
system. Once that is completed, an emerge of one of the above portage 
front-ends would add an icon to the menus of all installed WMs. That is about 
the closest you'll ever get to a default desktop icon.

> 6) A streamlined GUI install.
>
> I'm sure this one has been brought up before.  I consider this the "1.0"
> maker of the gentoo distribution.  In such an installer, I suggest that
> the CFLAGS should not be modified by default.  It has been shown in
> several places that optimizing these does not give a significant enough
> performance increase.  It should stay, of course, as an option though.

http://glis.sourceforge.net/
A mostly complete and bug-free curses version has already been written. The 
project is now undergoing a code restructuring to allow for easy rewriting of 
the front-end so that an X11 version can be written as well.

> 7) Emerge -S and Emerge -s speed improvements.
>
> I don't know why these commands perform as slow as they do.  My intuition
> says that they could be an order of magnitude faster.  Perhaps a
> reimplementation in C/C++ or a data format change could help.

This has been discussed many times. Faster implementations can be found at 
http://bugs.gentoo.org/show_bug.cgi?id=24756 or
http://gentoo.devel-net.org/download/emerge-fastsearch.patch

"emerge esearch" was also suggested but I cannot find any information on it.

Portage is currently undergoing a rewrite to allow for pluggable back-ends. 
DB-based backends will address problems such as these. For a current DB-based 
portage, check portagesql as breakmygentoo.net.


Regards,
Jason

--
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Some suggestions

[Jason Stubbs]

On Sunday 07 September 2003 08:48, Steven Elling wrote:
> On Saturday 06 September 2003 13:05, David Sankel wrote:
> > 2)  make.conf updates to be more automated
> >
> > Most gentoo users, I believe, modify this file.  This specific file
> > changes quite often with updates.  Since most users only modify the "USE"
> > and "CFLAGS" components, having an update that is automatic is plausible.
> >  This feature is a trade off between the integrity and consistency of the
> > system verses end-user maintenance time.
>
> Requiring portage updates to make.conf at all has always bugged me.  The
> file is meant to contain custom settings for portage and to append to or
> override variables in make.globals and the defaults.  It should not hold
> all the documentation for make.conf.  It should not hold all the
> defaults... that's what make.globals and the defaults are for.
>
> Why is all the documentation on make.conf in make.conf anyway?  Shouldn't
> it be in make.globals or better yet the man page?
>
> make.conf is used for system customization and, as such, portage should
> leave it alone.  When portage is installed on the drive for the first time
> it should not create make.conf.  Portage should leave it up to the
> admin/user of the box to create the file.

Damn - right after I made a summary! These are excellent points though. I 
agree wholeheartedly all the way!

Regards,
Jason

--
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Some suggestions

[Jon Portnoy]

On Sat, Sep 06, 2003 at 06:48:54PM -0500, Steven Elling wrote:
> On Saturday 06 September 2003 13:05, David Sankel wrote:
> > 2)  make.conf updates to be more automated
> >
> > Most gentoo users, I believe, modify this file.  This specific file
> > changes quite often with updates.  Since most users only modify the "USE"
> > and "CFLAGS" components, having an update that is automatic is plausible.
> >  This feature is a trade off between the integrity and consistency of the
> > system verses end-user maintenance time.
> 
> Requiring portage updates to make.conf at all has always bugged me.  The 
> file is meant to contain custom settings for portage and to append to or 
> override variables in make.globals and the defaults.  It should not hold 
> all the documentation for make.conf.  It should not hold all the 
> defaults... that's what make.globals and the defaults are for.
> 
> Why is all the documentation on make.conf in make.conf anyway?  Shouldn't it 
> be in make.globals or better yet the man page?
> 
> make.conf is used for system customization and, as such, portage should 
> leave it alone.  When portage is installed on the drive for the first time 
> it should not create make.conf.  Portage should leave it up to the 
> admin/user of the box to create the file.
> 

Most users find it very convienent to have all of that information 
available to them, especially on an initial install. Additionally, it's 
necessary to have that file present for Portage to operate properly, 
even if it hasn't been edited.


-- 
Jon Portnoy
avenj/irc.freenode.net

--
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Some suggestions

[Luke-Jr]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Saturday 06 September 2003 06:05 pm, David Sankel wrote:
> 1)  etc-update changes for a more automated system update
>
> etc-update allows you to automatically update (noted) etc files that one
> never changed from their last emerge.  This could save a lot of
> maintenance time if it was put in a cron job to routinely do that after
> an "emerge sync;emerge -u world"
>   The user then wouldn't have to look at all of the etc files to see
> if they were changed after every "emerge sync; emerge-u world".
I agree. etc-update should, at very least, check modify dates on files and if 
they haven't been changed, simply replace them...
>
> 2)  make.conf updates to be more automated
>
> Most gentoo users, I believe, modify this file.  This specific file changes
> quite often with updates.  Since most users only modify the "USE" and
> "CFLAGS" components, having an update that is automatic is plausible.  This
> feature is a trade off between the integrity and consistency of the system
> verses end-user maintenance time.
It might be a good idea to setup some kind of system where etc-update can be 
told *how* to update certain files. Maybe something like 
"/etc/.__update_make.conf" could be run as a script...
>
> 3) emerge -u world "NOTICE:" output changes.
>
> When doing an "emerge -u world" several "NOTICE:"'s fly by on the screen.
> For many users, they go unread although they contain important information
> in a lot of cases.  If these "NOTICE:"'s are cached and output at the end
> of an "emerge -u world", their readership would have a dramatic increase.
> This would allow interested gentoo users to be more informed of their
> system.
I believe something like this is already being setup to some extent. Likely it 
will simply be sent to a new stream or something so it can be easilly logged 
and displayed later.
>
> 4) emerge -u world progress bar option
>
> Most users probably care less about the compilation stages of their update
> in comparison to the percentage of completion.  A progress bar in this
> area would be a nice aesthetic and informative addition for most users.
> In order for this to happen, the output of the "emerge -u world" command
> would probably have to be standardized with some flags to mark the
> beginning and end of a compile.
I believe the -v option provides stuff scripts can look for...
>
> 5) A simple graphical front end to maintenance commands such as "emerge
> sync", "emerge -u world", and "etc-update".
>
> It would be a nice feature if users didn't have to commit these commands
> to memory for regular maintenance.  The maintenance menu might have an
> icon on all of the default desktops.  If this type of program was
> implemented, it could be prominently displayed and made known for all new
> users.  Perhaps the install documentation could use this tool as much as
> possible.
I'm not sure, but wouldn't KPortage have something like this?
>
> 6) A streamlined GUI install.
>
> I'm sure this one has been brought up before.  I consider this the "1.0"
> maker of the gentoo distribution.  In such an installer, I suggest that
> the CFLAGS should not be modified by default.  It has been shown in
> several places that optimizing these does not give a significant enough
> performance increase.  It should stay, of course, as an option though.
Gentoo is currently at 1.4. 1.0 was quite a while ago. I am working on a GUI 
install for computer newbies, and I believe the GLIS project is working on a 
GUI for more experienced users...
>
> 7) Emerge -S and Emerge -s speed improvements.
>
> I don't know why these commands perform as slow as they do.  My intuition
> says that they could be an order of magnitude faster.  Perhaps a
> reimplementation in C/C++ or a data format change could help.
I doubt the language would improve the speed at all. I'm not sure why it's 
slow, but it could very well be I/O access... grepping the Portage tree isn't 
any faster.
- -- 
Luke-Jr
Developer, Gentoo Linux
http://www.gentoo.org/
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.3 (GNU/Linux)

iD8DBQE/WnYtZl/BHdU+lYMRAhKLAJ9b7SRMKCfy5qfH5myFkmaBKWarrgCfXFaq
546MXDUZCIIhu7D5A2EFo9o=
=ZNeH
-----END PGP SIGNATURE-----


--
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Some suggestions

[Steven Elling]

On Saturday 06 September 2003 16:58, Brian Harring wrote:
> Eh, wouldn't hold or be particularly accurate, mainly since I/O, proc
> speed, and available memory (let alone if another job is running in the
> background and hogging cycles) are too many variables (imo) to try and
> factor out.
> Someone a while back had a setup such that they parsed the makefile,
> figuring out the number of actions (gcc calls, ar calls, mv/cp/install
> commands), and tracked progress that way.  Strikes me as the better
> way, although some packages weren't able to be parsed correctly
> resulting in a compilation progress reading at rather off values like
> 1100% and counting...
> ~bdh

What about using a bogomips seconds factor to determine a `close estimate' 
of how long a package will take to compile?  Here is something I posted to 
another mailing list:

====================
Subject: OpenOffice build on Gentoo / Determining how long it will take?
Date: 2003-05-28 14:07

I timed the build of OpenOffice on Gentoo yesturday using the following 
command:

time emerge -Du openoffice


The build of version OpenOffice 1.0.3-r1 completed today and time spit out 
the following:

real    844m6.053s
user    704m12.660s
sys     31m30.340s


Overall, the build of OpenOffice on my system (Athlon 900 MHz) took 14 hours 
4 minutes and 6.053 seconds.

I would think a good way to determine how long a build will take on your 
system is to divide a bogomips seconds factor by the bogomips from 
/proc/cpuinfo ( x / bogomips = time ) then throw in a fudge factor of 10%.  
Any thoughts anyone?

The bogomips seconds factor could be determined by compiling OpenOffice on 
several systems using the time command to record the amount of time it 
takes, calculating the bogomips seconds factor for each system using the 
formula "time * bogomips = x" and then averaging the numbers.

On my system, the bogomips are 1782.57 so the bogomips seconds factor for 
OpenOffice would be 50646.053 seconds * 1782.57 bogomips = 90280134.6962 
bogomips seconds.

Using the bogomips seconds factor from above, my laptop should take 
90280134.6962 bogomips seconds / 730.72 bogomips = 123549.5602 seconds (34 
hours 19 minutes and 9.5602 seconds) +- 10%.  I'll find out how long it 
actually takes sometime in the next week because I'm currently updating 
Gentoo on it.
====================


--
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Some suggestions

[Thomas de Grenier de Latour]

On Sun, 7 Sep 2003 08:53:25 +0900
Jason Stubbs <jasonbstubbs@mailandnews.com> wrote:


> "emerge esearch" was also suggested but I cannot find any information
> on it.

Oops, my fault. "esearch" is the name of a script for fast searching the
tree. I've thought it was in portage, but in fact it is not:
http://bugs.gentoo.org/show_bug.cgi?id=26532

-- 
TGL.

--
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Some suggestions

[Steven Elling]

On Saturday 06 September 2003 18:56, Jon Portnoy wrote:
> On Sat, Sep 06, 2003 at 06:48:54PM -0500, Steven Elling wrote:
> > On Saturday 06 September 2003 13:05, David Sankel wrote:
> > > 2)  make.conf updates to be more automated
> > >
> > > Most gentoo users, I believe, modify this file.  This specific file
> > > changes quite often with updates.  Since most users only modify the
> > > "USE" and "CFLAGS" components, having an update that is automatic is
> > > plausible. This feature is a trade off between the integrity and
> > > consistency of the system verses end-user maintenance time.
> >
> > Requiring portage updates to make.conf at all has always bugged me. 
> > The file is meant to contain custom settings for portage and to append
> > to or override variables in make.globals and the defaults.  It should
> > not hold all the documentation for make.conf.  It should not hold all
> > the defaults... that's what make.globals and the defaults are for.
> >
> > Why is all the documentation on make.conf in make.conf anyway? 
> > Shouldn't it be in make.globals or better yet the man page?
> >
> > make.conf is used for system customization and, as such, portage should
> > leave it alone.  When portage is installed on the drive for the first
> > time it should not create make.conf.  Portage should leave it up to the
> > admin/user of the box to create the file.
>
> Most users find it very convienent to have all of that information
> available to them, especially on an initial install. Additionally, it's
> necessary to have that file present for Portage to operate properly,
> even if it hasn't been edited.

IMHO, it is just as convienent to have the information in the man page.  
Also, maybe there needs to be a make.conf doc on www.gentoo.org and 
referenced from the install docs.  That way anyone doing an install could 
print out or view the install and make.conf docs.  Even though I've done 
several installs, I still like to have the install instructions in front of 
me and a make.conf doc would help.

Does portage bomb out if make.conf is not present?  If so, maybe portage 
needs to be changed so that it will work without the file.


--
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Some suggestions

[Luke-Jr]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

I did a bogomips test a while ago and it wasn't anywhere close, IIRC.
There could have been some other factor I missed though, as I was basing my 
test on time an emerge took on my system and other one I don't have root 
on... (via the gest script)
For all I know, another emerge was going on concurrently or the task was 
suspended at one point.

On Sunday 07 September 2003 12:10 am, Steven Elling wrote:
> On Saturday 06 September 2003 16:58, Brian Harring wrote:
> > Eh, wouldn't hold or be particularly accurate, mainly since I/O, proc
> > speed, and available memory (let alone if another job is running in the
> > background and hogging cycles) are too many variables (imo) to try and
> > factor out.
> > Someone a while back had a setup such that they parsed the makefile,
> > figuring out the number of actions (gcc calls, ar calls, mv/cp/install
> > commands), and tracked progress that way.  Strikes me as the better
> > way, although some packages weren't able to be parsed correctly
> > resulting in a compilation progress reading at rather off values like
> > 1100% and counting...
> > ~bdh
>
> What about using a bogomips seconds factor to determine a `close estimate'
> of how long a package will take to compile?  Here is something I posted to
> another mailing list:
>
> ====================
> Subject: OpenOffice build on Gentoo / Determining how long it will take?
> Date: 2003-05-28 14:07
>
> I timed the build of OpenOffice on Gentoo yesturday using the following
> command:
>
> time emerge -Du openoffice
>
>
> The build of version OpenOffice 1.0.3-r1 completed today and time spit out
> the following:
>
> real    844m6.053s
> user    704m12.660s
> sys     31m30.340s
>
>
> Overall, the build of OpenOffice on my system (Athlon 900 MHz) took 14
> hours 4 minutes and 6.053 seconds.
>
> I would think a good way to determine how long a build will take on your
> system is to divide a bogomips seconds factor by the bogomips from
> /proc/cpuinfo ( x / bogomips = time ) then throw in a fudge factor of 10%.
> Any thoughts anyone?
>
> The bogomips seconds factor could be determined by compiling OpenOffice on
> several systems using the time command to record the amount of time it
> takes, calculating the bogomips seconds factor for each system using the
> formula "time * bogomips = x" and then averaging the numbers.
>
> On my system, the bogomips are 1782.57 so the bogomips seconds factor for
> OpenOffice would be 50646.053 seconds * 1782.57 bogomips = 90280134.6962
> bogomips seconds.
>
> Using the bogomips seconds factor from above, my laptop should take
> 90280134.6962 bogomips seconds / 730.72 bogomips = 123549.5602 seconds (34
> hours 19 minutes and 9.5602 seconds) +- 10%.  I'll find out how long it
> actually takes sometime in the next week because I'm currently updating
> Gentoo on it.
> ====================
>
>
> --
> gentoo-dev@gentoo.org mailing list

- -- 
Luke-Jr
Developer, Gentoo Linux
http://www.gentoo.org/
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.3 (GNU/Linux)

iD8DBQE/WoB0Zl/BHdU+lYMRAn/5AKCQeVExUv2nInxCgtuyLcOEkSoDQwCcCLtD
4quU+ILFri7aKRLfrdIiDRg=
=Cbd5
-----END PGP SIGNATURE-----


--
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Some suggestions

[Chris Gianelloni]

[-- Attachment #1: Type: text/plain, Size: 831 bytes --]

On Sat, 2003-09-06 at 20:26, Steven Elling wrote:
> Does portage bomb out if make.conf is not present?  If so, maybe portage 
> needs to be changed so that it will work without the file.

I definitely like the idea of creating make.conf docs and a link from
the install docs.  Also, why can't the portage ebuild contain a 0 byte
make.conf file?  After all, the file CAN be empty and portage will still
work from the make.globals since make.conf serves no purpose but to
override the system defaults.  We could include a make.conf file in the
stage tarballs with a few settings (depending on the settings of the
stage) and a comment telling the user where the docs for make.conf are
located both locally and on Gentoo.org.

-- 
Chris Gianelloni
Developer, Gentoo Linux
Games Team

Is your power animal a penguin?

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-dev] Some suggestions

[Martin Schlemmer]

[-- Attachment #1: Type: text/plain, Size: 1659 bytes --]

On Sun, 2003-09-07 at 02:57, Chris Gianelloni wrote:
> On Sat, 2003-09-06 at 20:26, Steven Elling wrote:
> > Does portage bomb out if make.conf is not present?  If so, maybe portage 
> > needs to be changed so that it will work without the file.
> 
> I definitely like the idea of creating make.conf docs and a link from
> the install docs.  Also, why can't the portage ebuild contain a 0 byte
> make.conf file?  After all, the file CAN be empty and portage will still
> work from the make.globals since make.conf serves no purpose but to
> override the system defaults.  We could include a make.conf file in the
> stage tarballs with a few settings (depending on the settings of the
> stage) and a comment telling the user where the docs for make.conf are
> located both locally and on Gentoo.org.

I really do not see how having to either:

1) Copy and paste everything

2) Type if from a printout/whatever

is efficient or helps the average user?  It is way easier
to just uncomment and change as needed with the help, etc
there in front of you.

What happened to CONFIG_PROTECT and you having control?
Just mv the thing to make.conf.orig and edit a clean
file, or if you really want it to stay the same to see
what additions there is in future, leave as is, and
just put a 'source /etc/make.conf.foo' in there, and
add your changes to /etc/make.conf.foo.

Come on guys, think what is best for the *distro* (meaning,
what will work best for the other 90% of users, and not
necessary for you ...).


Thanks, 

-- 

Martin Schlemmer
Gentoo Linux Developer, Desktop/System Team Developer
Cape Town, South Africa



[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-dev] Some suggestions

[Jan Krueger]

On Sunday 07 September 2003 03:08, Martin Schlemmer wrote:
> Come on guys, think what is best for the *distro* (meaning,
> what will work best for the other 90% of users,

From my point of view the best for the 90% of users in this case (make.conf) 
would be:
1. a very precise documentation with examples about the user settable things 
for make.conf thats accessable via a standard command, like man make.conf or 
info make.conf

2. a clean, easy to read configuration file without mess and things the user 
doesnt care about so it is easy for the user (even easier for tools) to 
change exactly the setting the user wants to change because it is easier to 
identify the place where the change must happen and easier to identify the 
values that already are there.

You may try this yourself:
nano -w make.conf as it gets installed

and
nano -w make.conf with just the settings you actually use, everything else 
thrown out.

Which one is easier to modify? 
Especially try to change a FEATURE setting. Or even better, let a new gentoo 
user try to change a FEATURE setting.

If the user is not sure about the variables/values to put there the user may 
at anytime suspend nano or open another terminal or use another virtual 
console and execute man make.conf

most users probably use just 4 or 5 settings:
use flags
c(xx) flags
sync
mirrors
features

the rest is for the majority of users just useless in make.conf. So my 
expirience (and assumption).


So i strongly support:
On Saturday 06 September 2003 23:48, Steven Elling wrote:
> Requiring portage updates to make.conf at all has always bugged me.  The
> file is meant to contain custom settings for portage and to append to or
> override variables in make.globals and the defaults.  It should not hold
> all the documentation for make.conf.  It should not hold all the
> defaults... that's what make.globals and the defaults are for.
>
> Why is all the documentation on make.conf in make.conf anyway?  Shouldn't
> it be in make.globals or better yet the man page?
>
> make.conf is used for system customization and, as such, portage should
> leave it alone.  When portage is installed on the drive for the first time
> it should not create make.conf.  Portage should leave it up to the
> admin/user of the box to create the file.
Thats sound like a clean solution to me. Thats the way it should be.

I refuse to update my customised and over the time grown settings in 
/etc/make.conf with /etc/make.conf with comments for things i never intend to 
use. That doesnt make any sense to me to put such useless comments with 
documentation that has to be in the man page anyway in a file thats so 
important for my system.
I refuse to let anything automaticly update this file.
I refuse to touch this file until there is a strong need to edit it because i 
want a feature/useflag or whatever. So then, and only then, i edit this file 
or let a tool edit it (eg: euse).

If a change, because of a new advanced portage version, to my existing 
settings is needed, this change should be delayed as other software does it:
mark the old thing as deprecated and warn the user for some time|versions to 
give the user time to get informed and do the change manually or by using a 
dedicated tool.

Jan


--
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Some suggestions

[Rutger Lubbers]

> On Sat, 6 Sep 2003 21:50:42 +0200
> Marius Mauch <genone@genone.de> wrote:
>
>>
>> A accurate progress bar is nearly impossible as compile times differ
>> from package to package
>
> Maybe maintainers could fill in the ebuilds a kind of approximative
> compile time from their experience, which would be relative to a
> well know reference time (a kernel compilation with default options, or
> something like this). It doesn't need to be very accurate.
>

One could easily obtain those compile times from the emerge log file. 
See http://ripat.xs4all.nl/gentoo/stats/ (parse_log.py) for an 
implementation.
Perhaps this can be made part of the gentoo-stats package.

Rutger


--
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Some suggestions

[Troy Dack]

[-- Attachment #1: Type: text/plain, Size: 6707 bytes --]

On Sun, 2003-09-07 at 15:59, Jan Krueger wrote:
> On Sunday 07 September 2003 03:08, Martin Schlemmer wrote:
> > Come on guys, think what is best for the *distro* (meaning,
> > what will work best for the other 90% of users,

I'd say 95% of user, but that's just me :P

> From my point of view the best for the 90% of users in this case (make.conf) 
> would be:
> 1. a very precise documentation with examples about the user settable things 
> for make.conf thats accessable via a standard command, like man make.conf or 
> info make.conf

How imprecise and unaccessible is:

	nano -w /etc/make.conf

All settings are commented, and commented well, and it's available using
standard commands (you could even use ed or a combination of cat, less,
head & tail if you really wanted to)

> 2. a clean, easy to read configuration file without mess and things the user 
> doesnt care about so it is easy for the user (even easier for tools) to 
> change exactly the setting the user wants to change because it is easier to 
> identify the place where the change must happen and easier to identify the 
> values that already are there.

OK, I've installed Gentoo on my boxes that many times I know the process
pretty much off the top of my head, however I can't remember all of the
FEATURES that are available, having them listed in make.conf is
beneficial.

> You may try this yourself:
> nano -w make.conf as it gets installed
> 
> and
> nano -w make.conf with just the settings you actually use, everything else 
> thrown out.

grep -v "#" /etc/make.conf|wc -l && wc -l < /etc/make.conf
     31
    290

Fair enough, I've got 259 lines of comments, I can live with that, the
file isn't confusing and the options are grouped sensibly.

It seems to me to be fairly standard Linux practice to provide a well
commented config file that can be modified slightly as suited, eg:

grep -v "#" /etc/squid/squid.conf | wc -l && wc -l <
/etc/squid/squid.conf
    258
   3231

So, I've got nearly 3000 lines of comments in my squid.conf, I bet you
won't see anyone suggesting that squid gets installed with an empty
config file and you have to roll one by hand.

> Which one is easier to modify? 
> Especially try to change a FEATURE setting. Or even better, let a new gentoo 
> user try to change a FEATURE setting.
>
> If the user is not sure about the variables/values to put there the user may 
> at anytime suspend nano or open another terminal or use another virtual 
> console and execute man make.conf

So it is easier to explain to a newbie that they need to switch to a
new virtual console, start up the documentation viewer (man, less,
lynx), read through the docs, switch back to the other virtual console,
make a few changes in the file, lather, rinse, repeat.

I'll differ with your opinion here, I think it is much easier to have a
well commented file (same as reading code, if the comments are in the
code, it's easier than referring to a description of it and trying to
read the code).

> most users probably use just 4 or 5 settings:
> use flags
> c(xx) flags
> sync
> mirrors
> features

True, those are probably the only ones people set, but they are some of
the most important ones in the system.

Having a config file that lists only the variables to be changed will
lead to people swapping conf files ("Here, use this one it is all set
and works well for me") which could in turn lead to more problems
because people have settings enabled/disabled and they don't know why,
or what those things do.

> the rest is for the majority of users just useless in make.conf. So my 
> expirience (and assumption).
> 
> 
> So i strongly support:
> On Saturday 06 September 2003 23:48, Steven Elling wrote:
> > Requiring portage updates to make.conf at all has always bugged me.  The
> > file is meant to contain custom settings for portage and to append to or
> > override variables in make.globals and the defaults.  It should not hold
> > all the documentation for make.conf.  It should not hold all the
> > defaults... that's what make.globals and the defaults are for.
> >
> > Why is all the documentation on make.conf in make.conf anyway?  Shouldn't
> > it be in make.globals or better yet the man page?
> >
> > make.conf is used for system customization and, as such, portage should
> > leave it alone.  When portage is installed on the drive for the first time
> > it should not create make.conf.  Portage should leave it up to the
> > admin/user of the box to create the file.
> Thats sound like a clean solution to me. Thats the way it should be.

Given that a very, very, very large portion of the questions that get
fielded on #gentoo are the result of a significant lack of RTF'ing M I
don't think having an empty file is the way to go, #gentoo would just
degenerate into "RTFM" responses (of which I am pleased to say that
generally there are not *that* many).

[see above re: swapping of config files as well]

> I refuse to update my customised and over the time grown settings in 
> /etc/make.conf with /etc/make.conf with comments for things i never intend to 
> use. That doesnt make any sense to me to put such useless comments with 
> documentation that has to be in the man page anyway in a file thats so 
> important for my system.
> I refuse to let anything automaticly update this file.
> I refuse to touch this file until there is a strong need to edit it because i 
> want a feature/useflag or whatever. So then, and only then, i edit this file 
> or let a tool edit it (eg: euse).

etc-update, merge files, half a dozen l's, a couple of r's, the odd eb
and it's all done, new FEATURES merged in, old settings left intact.

Not hard, plus I am now aware of what new things are available.

> If a change, because of a new advanced portage version, to my existing 
> settings is needed, this change should be delayed as other software does it:
> mark the old thing as deprecated and warn the user for some time|versions to 
> give the user time to get informed and do the change manually or by using a 
> dedicated tool.
> 

"Gentoo moves pretty fast; if you don't stop and look around once and
awhile, you could miss out."
-- 
Troy Dack        "Yes, yes, I know that, Sydney ... Everybody knows that!
tad@gentoo.org    ... But look: Four wrongs squared, minus two wrongs to 
                  the fourth power, divided by this formula, do make a
                  right." -- Gary Larson

Public Key: http://pgp.mit.edu:11371/pks/lookup?op=get&search=0x4D90BE3C
Key fingerprint = 1F3D 6C15 16AA 09D5 0C96  92E5 FD89 16F9 4D90 BE3C
 

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-dev] Some suggestions

[Jason Stubbs]

Hmmm... Next time I'll think before posting.

After reading this discussion, I'd have to say that I'm in the 95%. However, 
if we get back to the root of the problem, there'll probably be a little bit 
less tension.

I think we all know what the problem is - maybe annoyance is a better word. 
I've had several times where I've had to update make.conf but the only thing 
that had changed is comments. One time there was not even any new 
functionality listed, only clarification in the existing comments.

Having said that, I can't offer a solution that isn't more trouble than it's 
worth. In fact, most solutions would break the standard way of configuring 
other files.

Nothing really to offer except to bring (some of) you guys back in to focus. 
Toodles for now.

On Sunday 07 September 2003 17:19, Troy Dack wrote:
> On Sun, 2003-09-07 at 15:59, Jan Krueger wrote:
> > On Sunday 07 September 2003 03:08, Martin Schlemmer wrote:
> > > Come on guys, think what is best for the *distro* (meaning,
> > > what will work best for the other 90% of users,
>
> I'd say 95% of user, but that's just me :P
>
> > From my point of view the best for the 90% of users in this case
> > (make.conf) would be:
> > 1. a very precise documentation with examples about the user settable
> > things for make.conf thats accessable via a standard command, like man
> > make.conf or info make.conf
>
> How imprecise and unaccessible is:
>
> 	nano -w /etc/make.conf
>
> All settings are commented, and commented well, and it's available using
> standard commands (you could even use ed or a combination of cat, less,
> head & tail if you really wanted to)
>
> > 2. a clean, easy to read configuration file without mess and things the
> > user doesnt care about so it is easy for the user (even easier for tools)
> > to change exactly the setting the user wants to change because it is
> > easier to identify the place where the change must happen and easier to
> > identify the values that already are there.
>
> OK, I've installed Gentoo on my boxes that many times I know the process
> pretty much off the top of my head, however I can't remember all of the
> FEATURES that are available, having them listed in make.conf is
> beneficial.
>
> > You may try this yourself:
> > nano -w make.conf as it gets installed
> >
> > and
> > nano -w make.conf with just the settings you actually use, everything
> > else thrown out.
>
> grep -v "#" /etc/make.conf|wc -l && wc -l < /etc/make.conf
>      31
>     290
>
> Fair enough, I've got 259 lines of comments, I can live with that, the
> file isn't confusing and the options are grouped sensibly.
>
> It seems to me to be fairly standard Linux practice to provide a well
> commented config file that can be modified slightly as suited, eg:
>
> grep -v "#" /etc/squid/squid.conf | wc -l && wc -l <
> /etc/squid/squid.conf
>     258
>    3231
>
> So, I've got nearly 3000 lines of comments in my squid.conf, I bet you
> won't see anyone suggesting that squid gets installed with an empty
> config file and you have to roll one by hand.
>
> > Which one is easier to modify?
> > Especially try to change a FEATURE setting. Or even better, let a new
> > gentoo user try to change a FEATURE setting.
> >
> > If the user is not sure about the variables/values to put there the user
> > may at anytime suspend nano or open another terminal or use another
> > virtual console and execute man make.conf
>
> So it is easier to explain to a newbie that they need to switch to a
> new virtual console, start up the documentation viewer (man, less,
> lynx), read through the docs, switch back to the other virtual console,
> make a few changes in the file, lather, rinse, repeat.
>
> I'll differ with your opinion here, I think it is much easier to have a
> well commented file (same as reading code, if the comments are in the
> code, it's easier than referring to a description of it and trying to
> read the code).
>
> > most users probably use just 4 or 5 settings:
> > use flags
> > c(xx) flags
> > sync
> > mirrors
> > features
>
> True, those are probably the only ones people set, but they are some of
> the most important ones in the system.
>
> Having a config file that lists only the variables to be changed will
> lead to people swapping conf files ("Here, use this one it is all set
> and works well for me") which could in turn lead to more problems
> because people have settings enabled/disabled and they don't know why,
> or what those things do.
>
> > the rest is for the majority of users just useless in make.conf. So my
> > expirience (and assumption).
> >
> >
> > So i strongly support:
> >
> > On Saturday 06 September 2003 23:48, Steven Elling wrote:
> > > Requiring portage updates to make.conf at all has always bugged me. 
> > > The file is meant to contain custom settings for portage and to append
> > > to or override variables in make.globals and the defaults.  It should
> > > not hold all the documentation for make.conf.  It should not hold all
> > > the defaults... that's what make.globals and the defaults are for.
> > >
> > > Why is all the documentation on make.conf in make.conf anyway? 
> > > Shouldn't it be in make.globals or better yet the man page?
> > >
> > > make.conf is used for system customization and, as such, portage should
> > > leave it alone.  When portage is installed on the drive for the first
> > > time it should not create make.conf.  Portage should leave it up to the
> > > admin/user of the box to create the file.
> >
> > Thats sound like a clean solution to me. Thats the way it should be.
>
> Given that a very, very, very large portion of the questions that get
> fielded on #gentoo are the result of a significant lack of RTF'ing M I
> don't think having an empty file is the way to go, #gentoo would just
> degenerate into "RTFM" responses (of which I am pleased to say that
> generally there are not *that* many).
>
> [see above re: swapping of config files as well]
>
> > I refuse to update my customised and over the time grown settings in
> > /etc/make.conf with /etc/make.conf with comments for things i never
> > intend to use. That doesnt make any sense to me to put such useless
> > comments with documentation that has to be in the man page anyway in a
> > file thats so important for my system.
> > I refuse to let anything automaticly update this file.
> > I refuse to touch this file until there is a strong need to edit it
> > because i want a feature/useflag or whatever. So then, and only then, i
> > edit this file or let a tool edit it (eg: euse).
>
> etc-update, merge files, half a dozen l's, a couple of r's, the odd eb
> and it's all done, new FEATURES merged in, old settings left intact.
>
> Not hard, plus I am now aware of what new things are available.
>
> > If a change, because of a new advanced portage version, to my existing
> > settings is needed, this change should be delayed as other software does
> > it: mark the old thing as deprecated and warn the user for some
> > time|versions to give the user time to get informed and do the change
> > manually or by using a dedicated tool.
>
> "Gentoo moves pretty fast; if you don't stop and look around once and
> awhile, you could miss out."

--
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Some suggestions

[Martin Schlemmer]

[-- Attachment #1: Type: text/plain, Size: 3588 bytes --]

On Sun, 2003-09-07 at 07:59, Jan Krueger wrote:
> On Sunday 07 September 2003 03:08, Martin Schlemmer wrote:
> > Come on guys, think what is best for the *distro* (meaning,
> > what will work best for the other 90% of users,
> 
> >From my point of view the best for the 90% of users in this case (make.conf) 
> would be:
> 1. a very precise documentation with examples about the user settable things 
> for make.conf thats accessable via a standard command, like man make.conf or 
> info make.conf
> 

 $ man make.conf

does work .. tried it lately ?

> 2. a clean, easy to read configuration file without mess and things the user 
> doesnt care about so it is easy for the user (even easier for tools) to 
> change exactly the setting the user wants to change because it is easier to 
> identify the place where the change must happen and easier to identify the 
> values that already are there.
> 
> You may try this yourself:
> nano -w make.conf as it gets installed
> 
> and
> nano -w make.conf with just the settings you actually use, everything else 
> thrown out.
> 
> Which one is easier to modify? 

Personally if it is a new app/whatever I try, than not having to search
for the howto to setup it is usually the easiest.


> So i strongly support:
> On Saturday 06 September 2003 23:48, Steven Elling wrote:
> > Requiring portage updates to make.conf at all has always bugged me.  The
> > file is meant to contain custom settings for portage and to append to or
> > override variables in make.globals and the defaults.  It should not hold
> > all the documentation for make.conf.  It should not hold all the
> > defaults... that's what make.globals and the defaults are for.
> >
> > Why is all the documentation on make.conf in make.conf anyway?  Shouldn't
> > it be in make.globals or better yet the man page?
> >
> > make.conf is used for system customization and, as such, portage should
> > leave it alone.  When portage is installed on the drive for the first time
> > it should not create make.conf.  Portage should leave it up to the
> > admin/user of the box to create the file.
> Thats sound like a clean solution to me. Thats the way it should be.
> 
> I refuse to update my customised and over the time grown settings in 
> /etc/make.conf with /etc/make.conf with comments for things i never intend to 
> use. That doesnt make any sense to me to put such useless comments with 
> documentation that has to be in the man page anyway in a file thats so 
> important for my system.
> I refuse to let anything automaticly update this file.
> I refuse to touch this file until there is a strong need to edit it because i 
> want a feature/useflag or whatever. So then, and only then, i edit this file 
> or let a tool edit it (eg: euse).
> 

Yes and ?  I still use a make.conf from portage 1.8 or there abouts on
some of my systems.

> If a change, because of a new advanced portage version, to my existing 
> settings is needed, this change should be delayed as other software does it:
> mark the old thing as deprecated and warn the user for some time|versions to 
> give the user time to get informed and do the change manually or by using a 
> dedicated tool.
> 

That is the point of *having* to update make.globals.


The problem it seems with most people, is that they do not just want to
run 'etc-update' and just press '2' when coming to updating make.conf,
or whatever.


-- 

Martin Schlemmer
Gentoo Linux Developer, Desktop/System Team Developer
Cape Town, South Africa



[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-dev] Some suggestions

[Martin Schlemmer]

[-- Attachment #1: Type: text/plain, Size: 6747 bytes --]

On Sun, 2003-09-07 at 10:19, Troy Dack wrote:
> On Sun, 2003-09-07 at 15:59, Jan Krueger wrote:
> > On Sunday 07 September 2003 03:08, Martin Schlemmer wrote:
> > > Come on guys, think what is best for the *distro* (meaning,
> > > what will work best for the other 90% of users,
> 
> I'd say 95% of user, but that's just me :P
> 
> > From my point of view the best for the 90% of users in this case (make.conf) 
> > would be:
> > 1. a very precise documentation with examples about the user settable things 
> > for make.conf thats accessable via a standard command, like man make.conf or 
> > info make.conf
> 
> How imprecise and unaccessible is:
> 
> 	nano -w /etc/make.conf
> 
> All settings are commented, and commented well, and it's available using
> standard commands (you could even use ed or a combination of cat, less,
> head & tail if you really wanted to)
> 
> > 2. a clean, easy to read configuration file without mess and things the user 
> > doesnt care about so it is easy for the user (even easier for tools) to 
> > change exactly the setting the user wants to change because it is easier to 
> > identify the place where the change must happen and easier to identify the 
> > values that already are there.
> 
> OK, I've installed Gentoo on my boxes that many times I know the process
> pretty much off the top of my head, however I can't remember all of the
> FEATURES that are available, having them listed in make.conf is
> beneficial.
> 
> > You may try this yourself:
> > nano -w make.conf as it gets installed
> > 
> > and
> > nano -w make.conf with just the settings you actually use, everything else 
> > thrown out.
> 
> grep -v "#" /etc/make.conf|wc -l && wc -l < /etc/make.conf
>      31
>     290
> 
> Fair enough, I've got 259 lines of comments, I can live with that, the
> file isn't confusing and the options are grouped sensibly.
> 
> It seems to me to be fairly standard Linux practice to provide a well
> commented config file that can be modified slightly as suited, eg:
> 
> grep -v "#" /etc/squid/squid.conf | wc -l && wc -l <
> /etc/squid/squid.conf
>     258
>    3231
> 
> So, I've got nearly 3000 lines of comments in my squid.conf, I bet you
> won't see anyone suggesting that squid gets installed with an empty
> config file and you have to roll one by hand.
> 
> > Which one is easier to modify? 
> > Especially try to change a FEATURE setting. Or even better, let a new gentoo 
> > user try to change a FEATURE setting.
> >
> > If the user is not sure about the variables/values to put there the user may 
> > at anytime suspend nano or open another terminal or use another virtual 
> > console and execute man make.conf
> 
> So it is easier to explain to a newbie that they need to switch to a
> new virtual console, start up the documentation viewer (man, less,
> lynx), read through the docs, switch back to the other virtual console,
> make a few changes in the file, lather, rinse, repeat.
> 
> I'll differ with your opinion here, I think it is much easier to have a
> well commented file (same as reading code, if the comments are in the
> code, it's easier than referring to a description of it and trying to
> read the code).
> 
> > most users probably use just 4 or 5 settings:
> > use flags
> > c(xx) flags
> > sync
> > mirrors
> > features
> 
> True, those are probably the only ones people set, but they are some of
> the most important ones in the system.
> 
> Having a config file that lists only the variables to be changed will
> lead to people swapping conf files ("Here, use this one it is all set
> and works well for me") which could in turn lead to more problems
> because people have settings enabled/disabled and they don't know why,
> or what those things do.
> 
> > the rest is for the majority of users just useless in make.conf. So my 
> > expirience (and assumption).
> > 
> > 
> > So i strongly support:
> > On Saturday 06 September 2003 23:48, Steven Elling wrote:
> > > Requiring portage updates to make.conf at all has always bugged me.  The
> > > file is meant to contain custom settings for portage and to append to or
> > > override variables in make.globals and the defaults.  It should not hold
> > > all the documentation for make.conf.  It should not hold all the
> > > defaults... that's what make.globals and the defaults are for.
> > >
> > > Why is all the documentation on make.conf in make.conf anyway?  Shouldn't
> > > it be in make.globals or better yet the man page?
> > >
> > > make.conf is used for system customization and, as such, portage should
> > > leave it alone.  When portage is installed on the drive for the first time
> > > it should not create make.conf.  Portage should leave it up to the
> > > admin/user of the box to create the file.
> > Thats sound like a clean solution to me. Thats the way it should be.
> 
> Given that a very, very, very large portion of the questions that get
> fielded on #gentoo are the result of a significant lack of RTF'ing M I
> don't think having an empty file is the way to go, #gentoo would just
> degenerate into "RTFM" responses (of which I am pleased to say that
> generally there are not *that* many).
> 
> [see above re: swapping of config files as well]
> 
> > I refuse to update my customised and over the time grown settings in 
> > /etc/make.conf with /etc/make.conf with comments for things i never intend to 
> > use. That doesnt make any sense to me to put such useless comments with 
> > documentation that has to be in the man page anyway in a file thats so 
> > important for my system.
> > I refuse to let anything automaticly update this file.
> > I refuse to touch this file until there is a strong need to edit it because i 
> > want a feature/useflag or whatever. So then, and only then, i edit this file 
> > or let a tool edit it (eg: euse).
> 
> etc-update, merge files, half a dozen l's, a couple of r's, the odd eb
> and it's all done, new FEATURES merged in, old settings left intact.
> 
> Not hard, plus I am now aware of what new things are available.
> 
> > If a change, because of a new advanced portage version, to my existing 
> > settings is needed, this change should be delayed as other software does it:
> > mark the old thing as deprecated and warn the user for some time|versions to 
> > give the user time to get informed and do the change manually or by using a 
> > dedicated tool.
> > 
> 
> "Gentoo moves pretty fast; if you don't stop and look around once and
> awhile, you could miss out."

Amen


-- 

Martin Schlemmer
Gentoo Linux Developer, Desktop/System Team Developer
Cape Town, South Africa



[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-dev] Some suggestions

[Alexander Gretencord]

On Sunday 07 September 2003 10:19, Troy Dack wrote:
> > 1. a very precise documentation with examples about the user settable
> > things for make.conf thats accessable via a standard command, like man
> > make.conf or info make.conf
>
> How imprecise and unaccessible is:
>
> 	nano -w /etc/make.conf
>
> All settings are commented, and commented well, and it's available using
> standard commands (you could even use ed or a combination of cat, less,
> head & tail if you really wanted to)

You are right, make.conf is quite well commented in the file itself. But Jan 
is also right :) man is _the_ standard way of looking something up. I am all 
for documentation in the config file itself, that's especially useful if you 
have a big file with many options _but_ gentoo has a general problem with 
documentation. Nothing responds to a 'man xyz' in gentoo or if it does, the 
information is outdated. I think that's one serious disadvantage.

Nobody likes to write documentation, I know, but it is necessary. As Phil 
Richards pointed out somewhere else in this thread: "I don't like running 
commands as root that neither have a man page nor respond to "-h" or 
"--help"."

> "Gentoo moves pretty fast; if you don't stop and look around once and
> awhile, you could miss out."

Oh, too true. Somewhere in time fixpackages was added. I was told to run it. 
Ok but what's it good for? Either portage should run it on its own, if it is 
necessary and I can't do anything about it and _not_ tell me anything _or_ if 
it tells me about it and even tells *me* to run it, then have some 
documentation. It has no man page, not even one telling to use info, nor does 
it respond to -h or --help. No documentation at all and that sucks elephants 
through key holes. I had to search through the forum to find out what it 
does. Documentation is one of the big problems of gentoo.


Alex

--
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Some suggestions

[Martin Schlemmer]

[-- Attachment #1: Type: text/plain, Size: 824 bytes --]

On Sun, 2003-09-07 at 16:29, Jan Krueger wrote:

> > The problem it seems with most people, is that they do not just want to
> > run 'etc-update' and just press '2' when coming to updating make.conf,
> > or whatever.
> Yes, you understand me and those people :)
> There is no need to update this file, so why should i?
> 

heh.  Choose '2' for etc-update 8)

Alternative then is to have your /etc/make.conf.example, and have the
full one in stage1, with none being installed with portage.  New guys
or those that like it will have a nice commented one, while those that
prune it, will not be bothered again.

I still think though that this is really a non issue - look at squid,
samba, etc =)


-- 

Martin Schlemmer
Gentoo Linux Developer, Desktop/System Team Developer
Cape Town, South Africa



[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-dev] Some suggestions

[Martin Schlemmer]

[-- Attachment #1: Type: text/plain, Size: 2964 bytes --]

On Sun, 2003-09-07 at 16:56, Jan Krueger wrote:
> On Sunday 07 September 2003 10:48, Martin Schlemmer wrote:
> > On Sun, 2003-09-07 at 10:19, Troy Dack wrote:
> > > "Gentoo moves pretty fast; if you don't stop and look around once and
> > > awhile, you could miss out."
> >
> > Amen
> 
> Thats not just Amen, that actually _is_ a big Problem if you are 
> administrating some servers that are supposed to provide high quality 
> services.
> 
> One of the reasons for this is, as good as it is and i like it very much, 
> portage. And a very timeconsuming thing is having to update things when there 
> is no need to do so. I could easily bypass this problem by creating my own 
> portage tree and modifying ebuilds. Some do so. That can get timeconsuming, 
> especially when the official portage tree changes a lot in basic ebuilds, and 
> additionally i would loose some very nice portage features. So far i came to 
> the conclusion not to use gentoo on my servers and instead join gentoo-dev. I 
> am happy to see that there is some progess with portage as can be verified on 
> the project pages :)
> 
> Having to update comments in some configuration files on each of the servers 
> is silly and a waste of time that someone has to pay for. No need for this.
> 
> If i dont update the comments after some time i will end up with differing 
> documentation (the one in man make.conf and the comments in make.conf)
> that can cause more harm then good. No, i dont want comments in configuration 
> files. And yes, i deleted all cruft from my apache.conf, squid.conf, 
> whatever.conf.
> 

Yes, but the point is that you usually can just delete the update on
the config files of stuff like portage, squid, apache, etc.  Most of
the times, the only stuff I do update is like the X config files,
gnome file, etc.  As I have said before, nothing really major have
changed since portage-1.x to make.conf to force you to update it
as long as you update /etc/make.globals.

If all the changes is to .example files, you are anyhow going to get
fed up with updating them - same as for the real ones now ... will
that make you notice actual critical config changes ?  No.

Most of the times ebuild do state that you should check some or other
change, so usually just throwing the update away should work (although
the bigger problem is that you do not see the announcement of the
ebuild anyhow if there is a few builds that was updated).

That, with the fact that changing this way of doing things, will break
the 'should sorda work out of the box' policy that we support (or try
to last time I checked).

And face it, even without all the comments (stripped down versions that
some suggested), if you do not want to take the effort to check what
my have changed now, you anyhow never will.


Regards,

-- 

Martin Schlemmer
Gentoo Linux Developer, Desktop/System Team Developer
Cape Town, South Africa



[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-dev] Some suggestions

[Thomas de Grenier de Latour]

On Sun, 7 Sep 2003 15:02:27 +0000
Jan Krueger <jk@microgalaxy.net> wrote:

> On Sunday 07 September 2003 12:44, Martin Schlemmer wrote:
> 
> > heh.  Choose '2' for etc-update 8)
> 
> Normally i do so. But i dont want to press this key!
> 

Have you tried addind /etc/make.conf to your CONFIG_PROTECT_MASK?

-- 
TGL.

--
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Some suggestions

[Martin Schlemmer]

[-- Attachment #1: Type: text/plain, Size: 1313 bytes --]

On Sun, 2003-09-07 at 17:02, Jan Krueger wrote:
> On Sunday 07 September 2003 12:44, Martin Schlemmer wrote:
> 
> > heh.  Choose '2' for etc-update 8)
> 
> Normally i do so. But i dont want to press this key!
> 
> Please understand: Its not about pressing a key. Its about:
> 
> Requiring expensive human interaction when there is no need for it.
> 

And not spending a minute or two every now and then might cause a
few hours of debugging.

The other side of the issue that nobody really touched (or wanted to)
up to now, is that the way of doing things as we do is with a reason.
What about proposing (with maybe prototype) a new way of doing what
we do now via CONFIG_PROTECT and etc-update/dispatch-conf that will
also fit the requirements that you guys want ?

Lastly, Nick, what happened to the auto merging of config files ?
At some stage, if the file was not changed by the user
(/var/cache/edb/config), they should be updated automatically.
I am thinking now of the postfix example stuff (/etc/postfix/sample)
and the gnome stuff in /etc/sound/events/ and /etc/gnome/, the
X kbd stuff, etc have to be merged manually every time, although I
never change them ... 


-- 

Martin Schlemmer
Gentoo Linux Developer, Desktop/System Team Developer
Cape Town, South Africa



[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-dev] Some suggestions

[Thomas de Grenier de Latour]

On Sun, 7 Sep 2003 15:23:03 +0000
Jan Krueger <jk@microgalaxy.net> wrote:

> On Sunday 07 September 2003 13:17, Thomas de Grenier de Latour wrote:
> > On Sun, 7 Sep 2003 15:02:27 +0000
> >
> > Jan Krueger <jk@microgalaxy.net> wrote:
> > > On Sunday 07 September 2003 12:44, Martin Schlemmer wrote:
> > > > heh.  Choose '2' for etc-update 8)
> > >
> > > Normally i do so. But i dont want to press this key!
> >
> > Have you tried addind /etc/make.conf to your CONFIG_PROTECT_MASK?
> So my settings get overridden?
> Thats also what i surely not want.
> 

Damn, I should wait the coffee is ready before posting. Sorry about
that.

-- 
TGL.

--
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Some suggestions

[Martin Schlemmer]

[-- Attachment #1: Type: text/plain, Size: 1946 bytes --]

On Sun, 2003-09-07 at 18:07, Jan Krueger wrote:
> On Sunday 07 September 2003 13:21, Martin Schlemmer wrote:
> > The other side of the issue that nobody really touched (or wanted to)
> > up to now, is that the way of doing things as we do is with a reason.
> > What about proposing (with maybe prototype) a new way of doing what
> > we do now via CONFIG_PROTECT and etc-update/dispatch-conf that will
> > also fit the requirements that you guys want ?
> 
> Ok, as is understand this would be the variable:
> CONFIG_EXCLUDE in /etc/make.conf
> 
> This variable would accept a list of directories/files for which the behaviour 
> of portage would be like follows:
> 
> whenever portage has the image of the to install software ready it scans this 
> image for the values in CONFIG_EXCLUDE.
> 
> whenever it finds such a directory/file in the image it moves the 
> directory/file to the doc-directory (eg: 
> /usr/share/doc/${PF}/excluded_config/ ) of the image (and maybe writes a 
> message to the user/log)
> 
> after that portage continues normally.
> 

Ok, this sounds like an good alternative, and it is flexible.

Might add a bug and add us all to it after its been discussed some more.

> (btw: i really dont like the possibility an ebuild can change the live 
> filesystem in pkg_postinst. that somehow makes the sandbox useless. it 
> shudders me, when i think of an ebuild that has a complicated shell code in 
> pkg_postinst with rm/cp/mv/cat/(other potentially dangerous commands) in it. 
> I just can hope that this shell code works as expected on the wide variations 
> of gentoo installations. but thats another story and another reason why i 
> dont use gentoo on my servers any longer)
> 

But you trust the daemons/programs running with root privs all the
time ? :D


Cheers,

-- 

Martin Schlemmer
Gentoo Linux Developer, Desktop/System Team Developer
Cape Town, South Africa



[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-dev] Some suggestions

[Martin Schlemmer]

[-- Attachment #1: Type: text/plain, Size: 876 bytes --]

On Sun, 2003-09-07 at 16:13, Martin Schlemmer wrote:
> On Sun, 2003-09-07 at 18:07, Jan Krueger wrote:
> > On Sunday 07 September 2003 13:21, Martin Schlemmer wrote:
> > > The other side of the issue that nobody really touched (or wanted to)
> > > up to now, is that the way of doing things as we do is with a reason.
> > > What about proposing (with maybe prototype) a new way of doing what
> > > we do now via CONFIG_PROTECT and etc-update/dispatch-conf that will
> > > also fit the requirements that you guys want ?
> > 
> > Ok, as is understand this would be the variable:
> > CONFIG_EXCLUDE in /etc/make.conf
> > 

Btw, I think that getting ebuild messages recorded an displayed at the
end of the merge is also a big part of this whole issue.


-- 

Martin Schlemmer
Gentoo Linux Developer, Desktop/System Team Developer
Cape Town, South Africa



[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-dev] Some suggestions

[Jan Krueger]

On Sunday 07 September 2003 10:44, Martin Schlemmer wrote:
>  $ man make.conf
>
> does work .. tried it lately ?

Yes, its a good man page.
I did not mean to critisize the existin man page. I 

> The problem it seems with most people, is that they do not just want to
> run 'etc-update' and just press '2' when coming to updating make.conf,
> or whatever.
Yes, you understand me and those people :)
There is no need to update this file, so why should i?

Jan


--
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Some suggestions

[Jan Krueger]

On Sunday 07 September 2003 10:48, Martin Schlemmer wrote:
> On Sun, 2003-09-07 at 10:19, Troy Dack wrote:
> > "Gentoo moves pretty fast; if you don't stop and look around once and
> > awhile, you could miss out."
>
> Amen

Thats not just Amen, that actually _is_ a big Problem if you are 
administrating some servers that are supposed to provide high quality 
services.

One of the reasons for this is, as good as it is and i like it very much, 
portage. And a very timeconsuming thing is having to update things when there 
is no need to do so. I could easily bypass this problem by creating my own 
portage tree and modifying ebuilds. Some do so. That can get timeconsuming, 
especially when the official portage tree changes a lot in basic ebuilds, and 
additionally i would loose some very nice portage features. So far i came to 
the conclusion not to use gentoo on my servers and instead join gentoo-dev. I 
am happy to see that there is some progess with portage as can be verified on 
the project pages :)

Having to update comments in some configuration files on each of the servers 
is silly and a waste of time that someone has to pay for. No need for this.

If i dont update the comments after some time i will end up with differing 
documentation (the one in man make.conf and the comments in make.conf)
that can cause more harm then good. No, i dont want comments in configuration 
files. And yes, i deleted all cruft from my apache.conf, squid.conf, 
whatever.conf.

Jan


--
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Some suggestions

[Jan Krueger]

On Sunday 07 September 2003 12:44, Martin Schlemmer wrote:

> heh.  Choose '2' for etc-update 8)

Normally i do so. But i dont want to press this key!

Please understand: Its not about pressing a key. Its about:

Requiring expensive human interaction when there is no need for it.

Jan


--
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Some suggestions

[Sami Näätänen]

On Sunday 07 September 2003 16:21, Martin Schlemmer wrote:
> On Sun, 2003-09-07 at 17:02, Jan Krueger wrote:
> > On Sunday 07 September 2003 12:44, Martin Schlemmer wrote:
> > > heh.  Choose '2' for etc-update 8)
> >
> > Normally i do so. But i dont want to press this key!
> >
> > Please understand: Its not about pressing a key. Its about:
> >
> > Requiring expensive human interaction when there is no need for it.
>
> And not spending a minute or two every now and then might cause a
> few hours of debugging.
>
> The other side of the issue that nobody really touched (or wanted to)
> up to now, is that the way of doing things as we do is with a reason.
> What about proposing (with maybe prototype) a new way of doing what
> we do now via CONFIG_PROTECT and etc-update/dispatch-conf that will
> also fit the requirements that you guys want ?
>
> Lastly, Nick, what happened to the auto merging of config files ?
> At some stage, if the file was not changed by the user
> (/var/cache/edb/config), they should be updated automatically.
> I am thinking now of the postfix example stuff (/etc/postfix/sample)
> and the gnome stuff in /etc/sound/events/ and /etc/gnome/, the
> X kbd stuff, etc have to be merged manually every time, although I
> never change them ...

I get only couple of files from even big world -Du updates, because I 
use CONFIG_PROTECT_MASK to get rid of unnecessary file updates.

CONFIG_PROTECT_MASK="/etc/gconf /etc/init.d /etc/X11/xkb /etc/X11/xdm 
/etc/X11/xserver /etc/X11/xsm /etc/X11/xinit /etc/env.d"

And if I need global env settings I simply add my own file to env.d 
directory.

So I for example have 00locale file in there, which set LC_CTYPE for my 
system.




--
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Some suggestions

[Martin Schlemmer]

[-- Attachment #1: Type: text/plain, Size: 1414 bytes --]

On Sun, 2003-09-07 at 19:55, Jan Krueger wrote:
> On Sunday 07 September 2003 13:12, Martin Schlemmer wrote:
> > That, with the fact that changing this way of doing things, will break
> > the 'should sorda work out of the box' policy that we support (or try
> > to last time I checked).
> Thats good policy and will work for most (Desktops).
> Windows does it, even for servers, and it works pretty well, especially for 
> lovesan.
> 

I did say 'sorda'.  Mostly it is just a fairly usable skeleton.

> On Sunday 07 September 2003 13:12, Martin Schlemmer wrote:
> > [...] work out of the box'
> Thats an illusion if gentoo-core would like to enter the serious server 
> market. There is no such thing as 'out of the box'.
> 

Once again, refer to top.

> Taken from http://www.gentoo.org/main/en/about.xml:
> "Thanks to a technology called Portage, Gentoo Linux can become an ideal 
> secure server"
> 
> is not really true. portage, beside the terrible security holes like 
> "pkg_postinst", has, up to now, nothing to do with the security of a server.
> Please clearify the statement on the page.
> 

Example since you keep bringing it up, and if possible how you would
resolve it.  Also, anything that might be harmful, could also be done
in src_install/where ever.


-- 

Martin Schlemmer
Gentoo Linux Developer, Desktop/System Team Developer
Cape Town, South Africa



[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-dev] Some suggestions

[Jan Krueger]

On Sunday 07 September 2003 13:21, Martin Schlemmer wrote:
> The other side of the issue that nobody really touched (or wanted to)
> up to now, is that the way of doing things as we do is with a reason.
> What about proposing (with maybe prototype) a new way of doing what
> we do now via CONFIG_PROTECT and etc-update/dispatch-conf that will
> also fit the requirements that you guys want ?

Ok, as is understand this would be the variable:
CONFIG_EXCLUDE in /etc/make.conf

This variable would accept a list of directories/files for which the behaviour 
of portage would be like follows:

whenever portage has the image of the to install software ready it scans this 
image for the values in CONFIG_EXCLUDE.

whenever it finds such a directory/file in the image it moves the 
directory/file to the doc-directory (eg: 
/usr/share/doc/${PF}/excluded_config/ ) of the image (and maybe writes a 
message to the user/log)

after that portage continues normally.

(btw: i really dont like the possibility an ebuild can change the live 
filesystem in pkg_postinst. that somehow makes the sandbox useless. it 
shudders me, when i think of an ebuild that has a complicated shell code in 
pkg_postinst with rm/cp/mv/cat/(other potentially dangerous commands) in it. 
I just can hope that this shell code works as expected on the wide variations 
of gentoo installations. but thats another story and another reason why i 
dont use gentoo on my servers any longer)

Jan


--
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Some suggestions

[Chris Gianelloni]

[-- Attachment #1: Type: text/plain, Size: 3222 bytes --]

On Sat, 2003-09-06 at 23:08, Martin Schlemmer wrote:
> On Sun, 2003-09-07 at 02:57, Chris Gianelloni wrote:
> > On Sat, 2003-09-06 at 20:26, Steven Elling wrote:
> > > Does portage bomb out if make.conf is not present?  If so, maybe portage 
> > > needs to be changed so that it will work without the file.
> > 
> > I definitely like the idea of creating make.conf docs and a link from
> > the install docs.  Also, why can't the portage ebuild contain a 0 byte
> > make.conf file?  After all, the file CAN be empty and portage will still
> > work from the make.globals since make.conf serves no purpose but to
> > override the system defaults.  We could include a make.conf file in the
> > stage tarballs with a few settings (depending on the settings of the
> > stage) and a comment telling the user where the docs for make.conf are
> > located both locally and on Gentoo.org.
> 
> I really do not see how having to either:
> 
> 1) Copy and paste everything
> 
> 2) Type if from a printout/whatever
> 
> is efficient or helps the average user?  It is way easier
> to just uncomment and change as needed with the help, etc
> there in front of you.

A link from the installation docs has already proved its worth.  Look at
the USE section of the install docs.  They point to the use.xml file. 
Why would make.conf be any harder?  I also said that the defaults used
in building the stage would be included IN THE STAGE tarball, as it is
now.  Only the portage ebuild would contain the "blank" make.conf.  If
you used a pentium4 stage3 to install from, then the settings in
make.conf would be the USE, CHOST, and CFLAGS used in building that
stage and nothing more.  I don't see how that makes anything harder on
anyone.  It puts all the documentation in a single place and makes it
easier.  As it is now the ONLY good documentation on make.conf is
included in make.conf.  This is unfortunate, since it requires users to
use the slightly complex interactive feature of etc-update just to see
documentation changes.  I find that to be counter-intuitive, especially
if everything were documented on gentoo.org and even in
/usr/share/doc/portage-<version>.

> 
> What happened to CONFIG_PROTECT and you having control?
> Just mv the thing to make.conf.orig and edit a clean
> file, or if you really want it to stay the same to see
> what additions there is in future, leave as is, and
> just put a 'source /etc/make.conf.foo' in there, and
> add your changes to /etc/make.conf.foo.

It has nothing to do with a clean file and more to do with the ease of
changing make.conf and make.globals.  Having the documentation for a
file that needs to be changed in the file itself just seems sloppy to
me.

> Come on guys, think what is best for the *distro* (meaning,
> what will work best for the other 90% of users, and not
> necessary for you ...).

I am thinking what is best for the distribution.  I use alternate
methods on my machines.  I speak here mostly on feedback I have gotten
from users I know personally and have talked to online, along with my my
own feelings.

-- 
Chris Gianelloni
Developer, Gentoo Linux
Games Team

Is your power animal a penguin?

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-dev] Some suggestions

[Jan Krueger]

On Sunday 07 September 2003 14:13, Martin Schlemmer wrote:
> But you trust the daemons/programs running with root privs all the
> time ? :D
No, they may contain unknown security holes. So i try hard to limit the amount 
of root daemons first and second try to run only root deamons that drop 
privileges after they did what they had to do as root. And there are options 
available to jail them pretty tight.

On Sun, 2003-09-07 at 16:13, Martin Schlemmer wrote:
> Btw, I think that getting ebuild messages recorded an displayed at the
> end of the merge is also a big part of this whole issue.

Yes, agreed :)

Jan


--
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Some suggestions

[Thomas de Grenier de Latour]

On Sun, 7 Sep 2003 18:21:19 +0000
Jan Krueger <jk@microgalaxy.net> wrote:

> put
> rm -rf /
> in pkg_postinst
> 
> put
> rm -rf /
> in src_install
> 
> See the difference?
> 

In any system package "foo", put in src_install():
cat << EOF > ${D}/usr/sbin/foo
 #!/bin/sh 
 rm -rf /
EOF

Not that better...

I think if you don't trust ebuilds, then you should not use them, or at
least read them before. The same apply to any distribution package. 

What is done in pkg_postinst is supposed to be good on every system. If
you find an ebuild in which it is not true, then report it as a bug and
if there is no safe way to fix it, then the command will probably be
turned into some einfo message asking you to do it by hand.


-- 
TGL.

--
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Some suggestions

[Chris Gianelloni]

[-- Attachment #1: Type: text/plain, Size: 525 bytes --]

On Sun, 2003-09-07 at 08:44, Martin Schlemmer wrote:
> Alternative then is to have your /etc/make.conf.example, and have the
> full one in stage1, with none being installed with portage.  New guys
> or those that like it will have a nice commented one, while those that
> prune it, will not be bothered again.

This was exactly what I was suggesting.  It would solve the "problem"
and I think would cater to both camps.

-- 
Chris Gianelloni
Developer, Gentoo Linux
Games Team

Is your power animal a penguin?

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-dev] Some suggestions

[Jon Portnoy]

On Sun, Sep 07, 2003 at 06:45:58PM +0200, Thomas de Grenier de Latour wrote:
> On Sun, 7 Sep 2003 18:21:19 +0000
> Jan Krueger <jk@microgalaxy.net> wrote:
> 
> > put
> > rm -rf /
> > in pkg_postinst
> > 
> > put
> > rm -rf /
> > in src_install
> > 
> > See the difference?
> > 
> 
> In any system package "foo", put in src_install():
> cat << EOF > ${D}/usr/sbin/foo
>  #!/bin/sh 
>  rm -rf /
> EOF
> 
> Not that better...
> 
> I think if you don't trust ebuilds, then you should not use them, or at
> least read them before. The same apply to any distribution package. 
> 
> What is done in pkg_postinst is supposed to be good on every system. If
> you find an ebuild in which it is not true, then report it as a bug and
> if there is no safe way to fix it, then the command will probably be
> turned into some einfo message asking you to do it by hand.
> 
> 

The only real vulnerability is if rsync mirrors are compromised. This is 
a major issue, and one that needs to be tackled with GPG signing of 
ebuilds - something that seems to be on hold for whatever reason.

CC'ing portage-dev.

-- 
Jon Portnoy
avenj/irc.freenode.net

--
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Some suggestions

[Jon Portnoy]

On Sun, Sep 07, 2003 at 12:55:49PM -0400, Jon Portnoy wrote:
> On Sun, Sep 07, 2003 at 06:45:58PM +0200, Thomas de Grenier de Latour wrote:
> > On Sun, 7 Sep 2003 18:21:19 +0000
> > Jan Krueger <jk@microgalaxy.net> wrote:
[snip]
> 
> The only real vulnerability is if rsync mirrors are compromised. This is 
> a major issue, and one that needs to be tackled with GPG signing of 
> ebuilds - something that seems to be on hold for whatever reason.
> 

Oops - manifests, not ebuilds.

-- 
Jon Portnoy
avenj/irc.freenode.net

--
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Some suggestions

[Martin Schlemmer]

[-- Attachment #1: Type: text/plain, Size: 570 bytes --]

On Sun, 2003-09-07 at 20:31, Jan Krueger wrote:
> On Sunday 07 September 2003 18:21, Jan Krueger wrote:
> > put
> > rm -rf /
> > in src_install
> >
> > See the difference?
> 
> What i meant to show is:
> as long as there is the possibility to wipe the box from within an ebuild it 
> is just a matter of time until this gets exploited.
> 

Ok, but this is the same for .rpm, .deb, etc.  Heck, even the kernel
could be exploited like this ...


-- 

Martin Schlemmer
Gentoo Linux Developer, Desktop/System Team Developer
Cape Town, South Africa



[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-dev] Some suggestions

[Martin Schlemmer]

[-- Attachment #1: Type: text/plain, Size: 1636 bytes --]

On Sun, 2003-09-07 at 18:43, Chris Gianelloni wrote:

> > 
> > What happened to CONFIG_PROTECT and you having control?
> > Just mv the thing to make.conf.orig and edit a clean
> > file, or if you really want it to stay the same to see
> > what additions there is in future, leave as is, and
> > just put a 'source /etc/make.conf.foo' in there, and
> > add your changes to /etc/make.conf.foo.
> 
> It has nothing to do with a clean file and more to do with the ease of
> changing make.conf and make.globals.  Having the documentation for a
> file that needs to be changed in the file itself just seems sloppy to
> me.
> 

This is not the only issue - check some of my later posts.

> > Come on guys, think what is best for the *distro* (meaning,
> > what will work best for the other 90% of users, and not
> > necessary for you ...).
> 
> I am thinking what is best for the distribution.  I use alternate
> methods on my machines.  I speak here mostly on feedback I have gotten
> from users I know personally and have talked to online, along with my my
> own feelings.

I will ask again - if you did not physically have to merge 10/20+ config
files every time, would it still be such an issue ? 

As I posted in other mails - there are a few good reasons why we do
things as we do, and the CONFIG_EXCLUDE might be a solution to the
'problem' (which in my opinion is still not CONFIG_PROTECT or 100%
the comments in the config files, but rather the 'merge process needed
for config files).


-- 

Martin Schlemmer
Gentoo Linux Developer, Desktop/System Team Developer
Cape Town, South Africa



[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-dev] Some suggestions

[Thomas de Grenier de Latour]

On Sun, 7 Sep 2003 19:07:03 +0000
Jan Krueger <jk@microgalaxy.net> wrote:

> The notable difference is:
> /usr/sbin/foo is not executed automaticly while emerge. 

You lack imagination: the bash scripts used by emerge are just
as easy to corrupt using a src_install only ebuild.

> On the other hand i try discuss on g-hardened how to detect malicious
> code.

Cryptographic signature as suggested by avenj would be a much more
realistic approach here. Since I do my phd in the security-oriented
program analysis domain, it breaks my heart to say that, but it's a
fact.

> > What is done in pkg_postinst is supposed to be good on every system.
> For sure, Windows is supposed to be good on every system too.
> However its deficencies make it from time to time a threat for the
> internet: code red, nimda to name just 2 of them.

I said "good", not "enough". I don't believe that src_postinst commands
will make your server super secure, but only that they do things that
are usefull and as safe as other parts of the ebuilds.

> I see the potential for gentoo to join windows on its way to bring the
> internet down.

I withdraw what I've said, you do have imagination.  :)
That would be a nice success story though.

-- 
TGL.

--
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Some suggestions

[Jan Krueger]

On Sunday 07 September 2003 13:12, Martin Schlemmer wrote:
> That, with the fact that changing this way of doing things, will break
> the 'should sorda work out of the box' policy that we support (or try
> to last time I checked).
Thats good policy and will work for most (Desktops).
Windows does it, even for servers, and it works pretty well, especially for 
lovesan.

On Sunday 07 September 2003 13:12, Martin Schlemmer wrote:
> [...] work out of the box'
Thats an illusion if gentoo-core would like to enter the serious server 
market. There is no such thing as 'out of the box'.

Taken from http://www.gentoo.org/main/en/about.xml:
"Thanks to a technology called Portage, Gentoo Linux can become an ideal 
secure server"

is not really true. portage, beside the terrible security holes like 
"pkg_postinst", has, up to now, nothing to do with the security of a server.
Please clearify the statement on the page.

Jan


--
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] suggestion pkg_postinst

[Martin Schlemmer]

[-- Attachment #1: Type: text/plain, Size: 1495 bytes --]

On Sun, 2003-09-07 at 20:12, Jan Krueger wrote:
> As is i already mentioned in mail before i see
> pkg_postinst and collegues as a risk that makes the sandbox of portage pretty 
> useless.
> 
> I understand that after transfering an image to the life filesystem sometimes 
> additional steps are required to make the software function well.
> 
> If this tasks are very special, this task should be triggered manually
> (eg. via ebuild bla.ebuild config or such)
> It should be possible to preview what task this command would execeute.
> 
> There is a variety of comman tasks that are triggered in pkg_postinst, like
> depmod -a or so. for these common things a secure abstraction should be 
> available (an api similar to dodir and collegues).
> 
> It must not be possible to modify the life filesystem from within an ebuild.
> (Maybe it would make sense to make this switchable, on or off.
> On - ebuilds can modify the life filesystem - for desktops
> Off - ebuilds can not modify the life filesystem - for those who care)
> 

So what if we take this example:

> In any system package "foo", put in src_install():
> cat << EOF > ${D}/usr/sbin/foo
>  #!/bin/sh
>  rm -rf /
> EOF

and change '${D}/usr/sbin/foo' to '${D}/sbin/init' ?
(ok, yes, its not going to work as a script if I remember
 correctly .. but a simple c wrapper is quick to code).


-- 

Martin Schlemmer
Gentoo Linux Developer, Desktop/System Team Developer
Cape Town, South Africa



[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-dev] Some suggestions

[Marius Mauch]

[-- Attachment #1: Type: text/plain, Size: 686 bytes --]

On 09/07/03  Jan Krueger wrote:

> Please (the one responsible for it) clearify the statement:
> "Thanks to a technology called Portage, Gentoo Linux can become an
> ideal secure server" in http://www.gentoo.org/main/en/about.xml

Please quote the whole sentence, the intention AFAICT is that portage is
a flexible package manager, not necessarily the most secure one. Your
part quoting makes it look like it indicates that portage is a very
secure software (which it isn't right now).

Marius

-- 
Public Key at http://www.genone.de/info/gpg-key.pub

In the beginning, there was nothing. And God said, 'Let there be
Light.' And there was still nothing, but you could see a bit better.

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

[gentoo-dev] suggestion pkg_postinst

[Jan Krueger]

As is i already mentioned in mail before i see
pkg_postinst and collegues as a risk that makes the sandbox of portage pretty 
useless.

I understand that after transfering an image to the life filesystem sometimes 
additional steps are required to make the software function well.

If this tasks are very special, this task should be triggered manually
(eg. via ebuild bla.ebuild config or such)
It should be possible to preview what task this command would execeute.

There is a variety of comman tasks that are triggered in pkg_postinst, like
depmod -a or so. for these common things a secure abstraction should be 
available (an api similar to dodir and collegues).

It must not be possible to modify the life filesystem from within an ebuild.
(Maybe it would make sense to make this switchable, on or off.
On - ebuilds can modify the life filesystem - for desktops
Off - ebuilds can not modify the life filesystem - for those who care)

Jan


--
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] suggestion portage ebuild system file modification rights and protection

[Martin Schlemmer]

[-- Attachment #1: Type: text/plain, Size: 741 bytes --]

On Sun, 2003-09-07 at 22:18, Jan Krueger wrote:
> On Sunday 07 September 2003 17:57, Martin Schlemmer wrote:
> > and change '${D}/usr/sbin/foo' to '${D}/sbin/init' ?
> > (ok, yes, its not going to work as a script if I remember
> >  correctly .. but a simple c wrapper is quick to code).
> 
> Cool, you just found another security bug in portage!
> 
> go on :)
> 
> So, the required feature thats implied with your detection, would be the 
> possibility to protect the already installed packages from modification 
> through installation of another package.
> 

And if this was baselayout that was compromised ?


-- 

Martin Schlemmer
Gentoo Linux Developer, Desktop/System Team Developer
Cape Town, South Africa



[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-dev] Some suggestions

[Jan Krueger]

On Sunday 07 September 2003 16:07, Martin Schlemmer wrote:
> On Sun, 2003-09-07 at 19:55, Jan Krueger wrote:
> Example since you keep bringing it up, and if possible how you would
> resolve it.  Also, anything that might be harmful, could also be done
> in src_install/where ever.

put
rm -rf /
in pkg_postinst

put
rm -rf /
in src_install

See the difference?

> in src_install/where ever.
This can be addressed by other means and is discussed on gentoo-hardened
(Mike Frysinger recommended me to discuss it there as i tried here and did not 
get much response)

Jan


--
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Some suggestions

[Martin Schlemmer]

[-- Attachment #1: Type: text/plain, Size: 1188 bytes --]

On Sun, 2003-09-07 at 21:55, Jan Krueger wrote:

> So does fixing the security holes in portage. We have identified 2 big ones so 
> far:
> 1. functions like pkg_postinst
> 2. easy to compromise bash scripts
> and another one is already well known:
> 3. the centralized portage tree
> 
> That leads me to the conclusions:
> portage is unsecure by design
> 
> Please (the one responsible for it) clearify the statement:
> "Thanks to a technology called Portage, Gentoo Linux can become an ideal 
> secure server" in http://www.gentoo.org/main/en/about.xml
> 
> I have to remove gentoo from my servers a little bit faster it seems...
> 

Ok, but .rpm/.deb have the same kind of flaws ... From here on I can
only see that you can use LFS or such, that you can make sure everything
is ok.

PS:  How are you going to verify that gcc's cvs repo was not
     compromised?  Or the kernel's ?  I guess you are going to
     start coding you own kernel, tool-chain and the rest even
     sooner now that we know how flawed linux, gnuish apps, etc
     are.


-- 

Martin Schlemmer
Gentoo Linux Developer, Desktop/System Team Developer
Cape Town, South Africa



[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-dev] Some suggestions

[Jan Krueger]

On Sunday 07 September 2003 18:21, Jan Krueger wrote:
> put
> rm -rf /
> in src_install
>
> See the difference?

What i meant to show is:
as long as there is the possibility to wipe the box from within an ebuild it 
is just a matter of time until this gets exploited.

Jan


--
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Some suggestions

[Thomas de Grenier de Latour]

On Sun, 7 Sep 2003 19:55:39 +0000
Jan Krueger <jk@microgalaxy.net> wrote:

> On Sunday 07 September 2003 17:39, Thomas de Grenier de Latour wrote:
>
> > You lack imagination: the bash scripts used by emerge are just
> > as easy to corrupt using a src_install only ebuild.
> 
> So this clearly is a bug that must be fixed.

No. It is only an evidence that absolute security has no meaning for a
turned on computer, and that safety only exists under some assumptions.
With gentoo, this assumptions include at least the facts that
developpers are not trying to compromise your servers, and that no
intruder can interfer between them and you. 
   I've choosed to believe in the first one, but if you have not, then
no technical solution will never convince you (think of the famous Ken
Thompson's paradox on trusting compilers [1]). 
   The second one is less convincing because of mirrors, but can easily
be replaced by a more robust one which is safety of some cryptographic
algorithms and their implementation if at some point portage make use of
gpg signatures. And it is true that this would be a good thing.

[1] http://www.acm.org/classics/sep95/

-- 
TGL.

--
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Some suggestions

[Jon Portnoy]

On Sun, Sep 07, 2003 at 08:52:57PM +0000, Jan Krueger wrote:
> On Sunday 07 September 2003 18:03, Marius Mauch wrote:
> > Please quote the whole sentence, the intention AFAICT is that portage is
> > a flexible package manager, not necessarily the most secure one. Your
> > part quoting makes it look like it indicates that portage is a very
> > secure software (which it isn't right now).
> Ok, here the whole sentence:
> "Thanks to a technology called Portage, Gentoo Linux can become an ideal 
> secure server, development workstation, professional desktop, gaming system, 
> embedded solution or something else -- whatever you need it to be."
> 
> Please (the one responsible for it) clearify the statement because its wrong 
> where i indicated in my previous mails and here:
> 
> Its not "thanks to [...] portage" that "gentoo Linux can become an ideal 
> secure server".
> 

The idea is that Portage gives you the flexibility to build a system 
usable in any environment. You're reading it as "Portage makes your 
system inherently secure" - in other words, you're misreading or 
misinterpreting.

-- 
Jon Portnoy
avenj/irc.freenode.net

--
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Some suggestions

[Jan Krueger]

On Sunday 07 September 2003 16:45, Thomas de Grenier de Latour wrote:
> On Sun, 7 Sep 2003 18:21:19 +0000
>
> Jan Krueger <jk@microgalaxy.net> wrote:
> > put
> > rm -rf /
> > in pkg_postinst
> >
> > put
> > rm -rf /
> > in src_install
> >
> > See the difference?
>
> In any system package "foo", put in src_install():
> cat << EOF > ${D}/usr/sbin/foo
>  #!/bin/sh
>  rm -rf /
> EOF
>
> Not that better...

The notable difference is:
/usr/sbin/foo is not executed automaticly while emerge. Thats what i try to 
address here. It must explicitly be called by someone/something after emerge 
time. So my system is safe at emerge time, at least.

On the other hand i try discuss on g-hardened how to detect malicious code.

> I think if you don't trust ebuilds, then you should not use them, or at
> least read them before. The same apply to any distribution package.
>
> What is done in pkg_postinst is supposed to be good on every system.
For sure, Windows is supposed to be good on every system too.
However its deficencies make it from time to time a threat for the internet: 
code red, nimda to name just 2 of them.
I see the potential for gentoo to join windows on its way to bring the 
internet down. especially i see this potential within portage because the 
whole portage tree has one central source, the only one source that needs to 
be compromised and it spreads to thousands of machines within a couple of 
minutes. So: No, i see no reason why i should ebuilds expect to be 
trustworthy. And you know this already! Dont try to fool me!
look at Manifest and digest! They are there because you know portage, the 
central tree and ebuilds are a risk that must be taken care of!
So i just try to make you sensible for another aspect of this risk.

> If
> you find an ebuild in which it is not true, then report it as a bug
Please understand: The bug is in portage! (at least from my point of view)

Jan


--
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] suggestion portage ebuild system file modification rights and protection

[Martin Schlemmer]

[-- Attachment #1: Type: text/plain, Size: 2123 bytes --]

On Sun, 2003-09-07 at 22:44, Jan Krueger wrote:
> On Sunday 07 September 2003 18:21, Martin Schlemmer wrote:
> > On Sun, 2003-09-07 at 22:18, Jan Krueger wrote:
> > > On Sunday 07 September 2003 17:57, Martin Schlemmer wrote:
> > > > and change '${D}/usr/sbin/foo' to '${D}/sbin/init' ?
> > > > (ok, yes, its not going to work as a script if I remember
> > > >  correctly .. but a simple c wrapper is quick to code).
> > >
> > > Cool, you just found another security bug in portage!
> > >
> > > go on :)
> > >
> > > So, the required feature thats implied with your detection, would be the
> > > possibility to protect the already installed packages from modification
> > > through installation of another package.
> >
> > And if this was baselayout that was compromised ?
> 
> Then you either
> -should have audited the ebuild and code of baselayout
> -hope that the md5sum protection alarmes you
> -hope that the signature protection alarmes you (not yet implemented)
> -hope that the security-oriented program analysis alarmes you (not yet 
> implemented)
> -hope that the problem hit someone else before you so it got widely published 
> and you read the news
> -hope that the automated test-procedures of gentoo detects the fault (not yet 
> implemented)
> -invent a special baselayout protection
> -have a second authorized tree that got not compromised (because operational 
> independend to the one gentoo tree with a special procedure that aims to 
> prevent to move of compromised things between the trees) to compare against 
> before emerge.
> -install some other os (with maybe different problems)
> -go out for a walk and watch sparrows or so :)
> -forbid the emerge of baselayout because you think its better to install 
> baselayout in a special hardened way instead.
> 

So how are any of these going to help if you do not trust us or any
other developers/upstream_authors, encryption, etc, etc.  I mean,
this *IS* what this whole issue is about, no ?


-- 

Martin Schlemmer
Gentoo Linux Developer, Desktop/System Team Developer
Cape Town, South Africa



[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-dev] Some suggestions

[Jon Portnoy]

On Sun, Sep 07, 2003 at 09:37:32PM +0000, Jan Krueger wrote:
> On Sunday 07 September 2003 18:53, Jon Portnoy wrote:
> > On Sun, Sep 07, 2003 at 08:52:57PM +0000, Jan Krueger wrote:
> > The idea is that Portage gives you the flexibility to build a system
> > usable in any environment.
> Thats a good formulation i would very much prefer to read.
> 

Then please file a bug about it.

-- 
Jon Portnoy
avenj/irc.freenode.net

--
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Some suggestions

[Phil Richards]

On 2003-09-06, Thomas de Grenier de Latour <degrenier@easyconnect.fr> wrote:
>  On Sat,  6 Sep 2003 21:23:24 +0100 (BST)
>  Phil Richards <news@derived-software.ltd.uk> wrote:
> > Erm, dispatch-conf?  What is it?  What does it do?
>  Python replacement for etc-update:
>  http://bugs.gentoo.org/show_bug.cgi?id=14079

Ok, read all that.  Found out what it is meant to be.
Didn't find the any hints about how to solve the error message.

>  It's true that some man page / --help output would be nice.

Essential, not nice.  I've admin'd for way too long to run commands
as root that I don't have documentation for :-)

phil
-- 
change name to "phil" for email


--
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Some suggestions

[Jan Krueger]

On Sunday 07 September 2003 17:39, Thomas de Grenier de Latour wrote:
> On Sun, 7 Sep 2003 19:07:03 +0000
>
> Jan Krueger <jk@microgalaxy.net> wrote:
> > The notable difference is:
> > /usr/sbin/foo is not executed automaticly while emerge.
>
> You lack imagination: the bash scripts used by emerge are just
> as easy to corrupt using a src_install only ebuild.

So this clearly is a bug that must be fixed.

> > On the other hand i try discuss on g-hardened how to detect malicious
> > code.
>
> Cryptographic signature as suggested by avenj would be a much more
> realistic approach here. Since I do my phd in the security-oriented
> program analysis domain, it breaks my heart to say that, but it's a
> fact.

but even cryptographic signatures got compromised (by faulty algorithms, users 
handling the keys unappropriate, ..., and even gentoo-core [supposed to 
handle the keys] is made out of humans and humans do make mistakes) So 
cryptographics signatures alone are not the holy grail as isnt 
security-oriented program analysis. But each one of them raises the bar a 
little bit, and both of them a little bit more :)

So does fixing the security holes in portage. We have identified 2 big ones so 
far:
1. functions like pkg_postinst
2. easy to compromise bash scripts
and another one is already well known:
3. the centralized portage tree

That leads me to the conclusions:
portage is unsecure by design

Please (the one responsible for it) clearify the statement:
"Thanks to a technology called Portage, Gentoo Linux can become an ideal 
secure server" in http://www.gentoo.org/main/en/about.xml

I have to remove gentoo from my servers a little bit faster it seems...

Jan


--
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] suggestion portage ebuild system file modification rights and protection

[Jon Portnoy]

On Sun, Sep 07, 2003 at 09:43:47PM +0000, Jan Krueger wrote:
> On Sunday 07 September 2003 19:20, Martin Schlemmer wrote:
> > So how are any of these going to help if you do not trust us or any
> > other developers/upstream_authors, encryption, etc, etc.  I mean,
> > this *IS* what this whole issue is about, no ?
> No. I trust you. But trusting you doesnt mean that the ebuild you checked in 
> to the tree arrives at my hardrive unmodified. There is no way for you as a 
> human beeing to garantee this to me. Instead it should be expected that the 
> ebuild gets modified (by faulty software/hardware/network/whatever or by a 
> malicious attacker). So this must be taken care of.
> 
> With Manifest and digest portage very much points in the right direction, but 
> this is not enough, from my point of view.
> 

Why is it not enough? Of course, Manifests by themeslves can be modified 
- that's why they need to be GPG signed.

The vulnerability at that point is compromised keys, which is why we 
would have an uberkey so we can revoke developer keys as soon as 
possible. It's not foolproof, but it's a whole lot better.

There is no such thing as perfect security short of shutting down your 
computer.

-- 
Jon Portnoy
avenj/irc.freenode.net

--
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] suggestion rsync over ssl/ssh

[Jon Portnoy]

On Sun, Sep 07, 2003 at 09:54:20PM +0000, Jan Krueger wrote:
> On Sunday 07 September 2003 21:43, Jan Krueger wrote:
> > No. I trust you. But trusting you doesnt mean that the ebuild you checked
> > in to the tree arrives at my hardrive unmodified. There is no way for you
> > as a human beeing to garantee this to me. Instead it should be expected
> > that the ebuild gets modified (by faulty software/hardware/network/whatever
> > or by a malicious attacker). So this must be taken care of.
> I give you an example:
> With so many gentoo-rsync hosts spread all over and the use of unencripted 
> rsync transfer a man in the middle attack (eg. by arp-spoofing or whatever), 
> that inserts an malicious ebuild along with digest and Manifest into the 
> rsync stream is very much imaginable to me.
> 
> So i suggest to, as quickly as possible, establish the infrastucture to do 
> rsync over ssl/ssl or other secure transport.
> 

GPG signing would take care of this issue.

-- 
Jon Portnoy
avenj/irc.freenode.net

--
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Some suggestions

[Kevyn Shortell]

[-- Attachment #1: Type: text/plain, Size: 1123 bytes --]

Don't you think the ebuilds get tested before they're pushed out to the
tree? If an ebuild was going to delete the contents of the hard drive, a
dev would be the first person to find out.

And any user, can simply as root, type rm -rf /*... do we need to also
come up with a preventive measure for that 'exploit' as well?

We're not going to have training wheels on the world. If you're that
ultra paranoid about breaking your system, perhaps you should hand walk
each ebuild before emerging it, and then emerge it when you feel safe.

In the meantime, I think the small army of devs and testers who've
already emerged it and deemed it working is sufficient for just about
everyone.

trance

On Sun, 2003-09-07 at 11:31, Jan Krueger wrote:
> On Sunday 07 September 2003 18:21, Jan Krueger wrote:
> > put
> > rm -rf /
> > in src_install
> >
> > See the difference?
> 
> What i meant to show is:
> as long as there is the possibility to wipe the box from within an ebuild it 
> is just a matter of time until this gets exploited.
> 
> Jan
> 
> 
> --
> gentoo-dev@gentoo.org mailing list
> 

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-dev] suggestion portage ebuild system file modification rights and protection

[Jan Krueger]

On Sunday 07 September 2003 17:57, Martin Schlemmer wrote:
> and change '${D}/usr/sbin/foo' to '${D}/sbin/init' ?
> (ok, yes, its not going to work as a script if I remember
>  correctly .. but a simple c wrapper is quick to code).

Cool, you just found another security bug in portage!

go on :)

So, the required feature thats implied with your detection, would be the 
possibility to protect the already installed packages from modification 
through installation of another package.

Or said in different words:
if one emerges an ebuild this ebuild is allowed only to add files to the 
system that did not exist before and/or change only files that got installed 
by a previous revision of the same ebuild. This way it would be impossible 
for the ebuild to change existing files, like /sbin/init, in the system. Its 
forbidden.

Thank you for enlightening this.

Some days ago i stumbled over this:
try
emerge ezmlm
and
emerge ezmlm-idx
they happily overwrite each other. Preventing such mess inside portage would 
be of great value for security and overall quality.

Jan


--
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Some suggestions

[Thomas de Grenier de Latour]

On Sun,  7 Sep 2003 20:41:56 +0100 (BST)
Phil Richards <news@derived-software.ltd.uk> wrote:

> On 2003-09-06, Thomas de Grenier de Latour <degrenier@easyconnect.fr>
> wrote:
> >  Python replacement for etc-update:
> >  http://bugs.gentoo.org/show_bug.cgi?id=14079
> 
> Ok, read all that.  Found out what it is meant to be.
> Didn't find the any hints about how to solve the error message.

You have to create a backup directory somewhere. Set its location in
"/etc/dispatch-conf.conf". (Default is /etc/config-archive, but I
personally use /var/backup/config). While you are in this conf file, you
should also decide whether you want the backup to be a collection of
files or a RCS repository. And you can choose the type of
automatic merges you want (I use automatic CVS header merge and
unmodified files auto-upgade). 

And then you're ready to go, "alias etc-update=dispatch-conf" :)


-- 
TGL.

--
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Some suggestions

[Martin Schlemmer]

[-- Attachment #1: Type: text/plain, Size: 814 bytes --]

On Sun, 2003-09-07 at 21:41, Phil Richards wrote:
> On 2003-09-06, Thomas de Grenier de Latour <degrenier@easyconnect.fr> wrote:
> >  On Sat,  6 Sep 2003 21:23:24 +0100 (BST)
> >  Phil Richards <news@derived-software.ltd.uk> wrote:
> > > Erm, dispatch-conf?  What is it?  What does it do?
> >  Python replacement for etc-update:
> >  http://bugs.gentoo.org/show_bug.cgi?id=14079
> 
> Ok, read all that.  Found out what it is meant to be.
> Didn't find the any hints about how to solve the error message.
> 

Just create the directory it errors out about.  You might
add it to bug mentioned that it should at least mention
the creation if they do not want to install the dir by default.


-- 

Martin Schlemmer
Gentoo Linux Developer, Desktop/System Team Developer
Cape Town, South Africa



[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-dev] suggestion portage ebuild system file modification rights and protection

[Jon Portnoy]

On Sun, Sep 07, 2003 at 10:34:06PM +0000, Jan Krueger wrote:
> On Sunday 07 September 2003 19:56, Jon Portnoy wrote:
> > The vulnerability at that point is compromised keys, which is why we
> > would have an uberkey so we can revoke developer keys as soon as
> > possible. It's not foolproof, but it's a whole lot better.
> I agree.
> But thats no excuse to not fix the security/consitency faults in portage that 
> showed up in this discussion.
> 

What, that any situation involving installing software is going to have 
security holes? That's the nature of software installation.

> You never know ...
> 
> It may already be to late for thousends of users until someone of gentoo-core 
> uses the ueberkey, especially in holiday seasons.
> 
> Or has core, especially in key questions, an availablity of 24/7?

We have enough managers who do this nearly full-time (including myself 
most of the time), and we share phone numbers (and cell phone numbers), 
so it's fairly unlikely.

> 
> > There is no such thing as perfect security short of shutting down your
> > computer.
> Yes, you never know...
> Thats why i would prefer a secure transport layer for emerge, you know?
> 
> Jan

A secure transport layer is unnecessary if we're using GPG signing, 
which has always been the intent - but seems to have stalled.

-- 
Jon Portnoy
avenj/irc.freenode.net

--
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Some suggestions

[Doug Weimer]

[-- Attachment #1: Type: text/plain, Size: 708 bytes --]

On Sun, 2003-09-07 at 10:27, Martin Schlemmer wrote:
<snip>
> As I posted in other mails - there are a few good reasons why we do
> things as we do, and the CONFIG_EXCLUDE might be a solution to the
> 'problem' (which in my opinion is still not CONFIG_PROTECT or 100%
> the comments in the config files, but rather the 'merge process needed
> for config files).

What about a modification to etc-update that allows portions of a file
to be protected? Something along the lines of:

#Instructional comments
#PROTECTED

CFLAGS="..."

#/PROTECTED

Such tags and a method to auto-merge comment updates should eliminate a
lot of the superfluous config file merging.

Just a thought,

Doug

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-dev] suggestion portage ebuild system file modification rights and protection

[Jan Krueger]

On Sunday 07 September 2003 18:21, Martin Schlemmer wrote:
> On Sun, 2003-09-07 at 22:18, Jan Krueger wrote:
> > On Sunday 07 September 2003 17:57, Martin Schlemmer wrote:
> > > and change '${D}/usr/sbin/foo' to '${D}/sbin/init' ?
> > > (ok, yes, its not going to work as a script if I remember
> > >  correctly .. but a simple c wrapper is quick to code).
> >
> > Cool, you just found another security bug in portage!
> >
> > go on :)
> >
> > So, the required feature thats implied with your detection, would be the
> > possibility to protect the already installed packages from modification
> > through installation of another package.
>
> And if this was baselayout that was compromised ?

Then you either
-should have audited the ebuild and code of baselayout
-hope that the md5sum protection alarmes you
-hope that the signature protection alarmes you (not yet implemented)
-hope that the security-oriented program analysis alarmes you (not yet 
implemented)
-hope that the problem hit someone else before you so it got widely published 
and you read the news
-hope that the automated test-procedures of gentoo detects the fault (not yet 
implemented)
-invent a special baselayout protection
-have a second authorized tree that got not compromised (because operational 
independend to the one gentoo tree with a special procedure that aims to 
prevent to move of compromised things between the trees) to compare against 
before emerge.
-install some other os (with maybe different problems)
-go out for a walk and watch sparrows or so :)
-forbid the emerge of baselayout because you think its better to install 
baselayout in a special hardened way instead.

i better stop now, it seems i could make this list very long :)

Jan


--
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Some suggestions

[Jan Krueger]

On Sunday 07 September 2003 18:03, Marius Mauch wrote:
> Please quote the whole sentence, the intention AFAICT is that portage is
> a flexible package manager, not necessarily the most secure one. Your
> part quoting makes it look like it indicates that portage is a very
> secure software (which it isn't right now).
Ok, here the whole sentence:
"Thanks to a technology called Portage, Gentoo Linux can become an ideal 
secure server, development workstation, professional desktop, gaming system, 
embedded solution or something else -- whatever you need it to be."

Please (the one responsible for it) clearify the statement because its wrong 
where i indicated in my previous mails and here:

Its not "thanks to [...] portage" that "gentoo Linux can become an ideal 
secure server".

Jan


--
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Some suggestions

[Martin Schlemmer]

[-- Attachment #1: Type: text/plain, Size: 2039 bytes --]

On Sun, 2003-09-07 at 22:37, Doug Weimer wrote:
> On Sun, 2003-09-07 at 10:27, Martin Schlemmer wrote:
> <snip>
> > As I posted in other mails - there are a few good reasons why we do
> > things as we do, and the CONFIG_EXCLUDE might be a solution to the
> > 'problem' (which in my opinion is still not CONFIG_PROTECT or 100%
> > the comments in the config files, but rather the 'merge process needed
> > for config files).
> 
> What about a modification to etc-update that allows portions of a file
> to be protected? Something along the lines of:
> 
> #Instructional comments
> #PROTECTED
> 
> CFLAGS="..."
> 
> #/PROTECTED
> 
> Such tags and a method to auto-merge comment updates should eliminate a
> lot of the superfluous config file merging.
> 
> Just a thought,
> 

Have a look at dispatch-conf (ok, so its new for me as well ;)

---------------------------------------------
On Sun, 2003-09-07 at 22:21, Thomas de Grenier de Latour wrote: 
> On Sun,  7 Sep 2003 20:41:56 +0100 (BST)
> Phil Richards <news@derived-software.ltd.uk> wrote:
> 
> > On 2003-09-06, Thomas de Grenier de Latour <degrenier@easyconnect.fr>
> > wrote:
> > >  Python replacement for etc-update:
> > >  http://bugs.gentoo.org/show_bug.cgi?id=14079
> > 
> > Ok, read all that.  Found out what it is meant to be.
> > Didn't find the any hints about how to solve the error message.
> 
> You have to create a backup directory somewhere. Set its location in
> "/etc/dispatch-conf.conf". (Default is /etc/config-archive, but I
> personally use /var/backup/config). While you are in this conf file, you
> should also decide whether you want the backup to be a collection of
> files or a RCS repository. And you can choose the type of
> automatic merges you want (I use automatic CVS header merge and
> unmodified files auto-upgade). 
> 
> And then you're ready to go, "alias etc-update=dispatch-conf" :)


-- 

Martin Schlemmer
Gentoo Linux Developer, Desktop/System Team Developer
Cape Town, South Africa



[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-dev] Some suggestions

[Jan Krueger]

On Sunday 07 September 2003 18:28, Martin Schlemmer wrote:
> Ok, but .rpm/.deb have the same kind of flaws ... 
Absolutly correct. Thats why i dont use them.

> From here on I can
> only see that you can use LFS or such, that you can make sure everything
> is ok.
>
> PS:  How are you going to verify that gcc's cvs repo was not
>      compromised?
Thats very realistic and compromise of at least the gnu ftp server happened 
not so long ago. I see the FSF to be responsible for handling this. 

>  Or the kernel's ?
Thats the responsibility of kernel.org.

gentoo is responsible for portage and what portage does. Thats what i would 
like to discuss here.

> I guess you are going to
>      start coding you own kernel, tool-chain and the rest even
>      sooner now that we know how flawed linux, gnuish apps, etc
>      are.

:)

alternatives there are, so many of them, i just have to choose and i already 
did. No need for me to write my own os :)


--
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Some suggestions

[Jan Krueger]

On Sunday 07 September 2003 18:53, Jon Portnoy wrote:
> On Sun, Sep 07, 2003 at 08:52:57PM +0000, Jan Krueger wrote:
> The idea is that Portage gives you the flexibility to build a system
> usable in any environment.
Thats a good formulation i would very much prefer to read.


--
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] suggestion portage ebuild system file modification rights and protection

[Jan Krueger]

On Sunday 07 September 2003 19:20, Martin Schlemmer wrote:
> So how are any of these going to help if you do not trust us or any
> other developers/upstream_authors, encryption, etc, etc.  I mean,
> this *IS* what this whole issue is about, no ?
No. I trust you. But trusting you doesnt mean that the ebuild you checked in 
to the tree arrives at my hardrive unmodified. There is no way for you as a 
human beeing to garantee this to me. Instead it should be expected that the 
ebuild gets modified (by faulty software/hardware/network/whatever or by a 
malicious attacker). So this must be taken care of.

With Manifest and digest portage very much points in the right direction, but 
this is not enough, from my point of view.

Jan


--
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] suggestion rsync over ssl/ssh

[Jan Krueger]

On Sunday 07 September 2003 21:43, Jan Krueger wrote:
> No. I trust you. But trusting you doesnt mean that the ebuild you checked
> in to the tree arrives at my hardrive unmodified. There is no way for you
> as a human beeing to garantee this to me. Instead it should be expected
> that the ebuild gets modified (by faulty software/hardware/network/whatever
> or by a malicious attacker). So this must be taken care of.
I give you an example:
With so many gentoo-rsync hosts spread all over and the use of unencripted 
rsync transfer a man in the middle attack (eg. by arp-spoofing or whatever), 
that inserts an malicious ebuild along with digest and Manifest into the 
rsync stream is very much imaginable to me.

So i suggest to, as quickly as possible, establish the infrastucture to do 
rsync over ssl/ssl or other secure transport.

Jan


--
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] suggestion portage ebuild system file modification rights and protection

[Jan Krueger]

On Sunday 07 September 2003 19:56, Jon Portnoy wrote:
> The vulnerability at that point is compromised keys, which is why we
> would have an uberkey so we can revoke developer keys as soon as
> possible. It's not foolproof, but it's a whole lot better.
I agree.
But thats no excuse to not fix the security/consitency faults in portage that 
showed up in this discussion.

You never know ...

It may already be to late for thousends of users until someone of gentoo-core 
uses the ueberkey, especially in holiday seasons.

Or has core, especially in key questions, an availablity of 24/7?

> There is no such thing as perfect security short of shutting down your
> computer.
Yes, you never know...
Thats why i would prefer a secure transport layer for emerge, you know?

Jan


--
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] suggestion portage ebuild system file modification rights and protection

[Jon Portnoy]

On Mon, Sep 08, 2003 at 01:32:45AM +0000, Jan Krueger wrote:
> On Sunday 07 September 2003 20:35, Jon Portnoy wrote:
> > What, that any situation involving installing software is going to have
> > security holes? That's the nature of software installation.
> 
> The gift of portage is a new quality of automated software installation 
> system, easy to use, never seen before. This is good.
> 
> However by being a full automated software installation system portage has a 
> very high responsibilty for the integrity of the system it is run on.
> 
> The discussion should have shown you, that the current portage can not bear 
> this responsibility.
> 
> This thread showed and will continue to show:
> ebuilds can overwrite any system file
> ebuilds can wipe your box
> ebuilds are affecting system integrity
> ebuilds lack quality
> 
> Some of you think this can be addressed by:
> qa
> signing ebuilds and files
> 
> I say:
> this is not sufficient.

You haven't explained why it's not sufficient.

> 
> Again examples from the actual tree, so you can try yourself:
> 1. emerge ezmlm and emerge ezmlm-idx
> providing slightly different funtionality they will overwrite each other 
> (instead of blocking each other)
> 

Bug. Is it filed?

> 2. emerge heartbeat (sys-cluster/heartbeat-1.0.3-r1, marked x86, so considered 
> stable)
> If your emerge was successfull (it will be as long as you dont have snmp 
> installed), try to get it running! You wont! Then take a look at 
> bugs.gentoo.org since when important fixes are available.
> 
> This two examples clearly show what the current qa is worth. (Well i respect 
> it very much, but its not perfect and never will be, thats qa, qa you for 
> sure know from other software/hardware products you use and used before, you 
> know the deficencies of qa then)
> I am pretty sure you can post examples from the tree yourself.

So we don't have enough manpower. I know that. The problem is finding 
good developers, people who're trustworthy and have a history of good 
contributions without a history of conflict.

> 
> And to me its clear why it is like that (at least on reason):
> If you compare the number of ebuilds to the number of CVS committers than you 
> will see, that it is impossible for each CVS-commiter to make a valuable 
> statement of the quality of the software and/or ebuild he/she is going to 
> commit. There are to many ebuilds.
> 
> (i see this as an organizational deficiency of gentoo)
> So it is probable that faulty ebuilds hit the tree. They are there actually.
> 
> Conclusion for me:
> I expect faulty ebuilds. they are there.  (you will never wipe them out until 
> you change the organization of gentoo development to a more apropriate model, 
> and even then, you can only reduce the amount of faulty ebuilds. ebuilds are 
> software, software has bugs.)
> 
> and therefore:
> i hold portage responsible to protect my system from faulty ebuilds as much as 
> it can.
> if not portage, who else? there is just me left, but then, i dont need 
> portage.
> 
> I made several suggestions how portage can be improved to protect the system 
> it is run on. 
> -prevent ebuilds from modifying the life filesystem


So basically you're saying portage shouldn't install software.

> -allow the actual package install process only to add files to the filesystem 
> or to only modify/remove files that belong to an revision of the same ebuild
> (qa would benefit from this suggestions too.)

So we should never be able to tweak config files et al in an ebuild?

> This would result in:
> -enforced package integrity
> -wiping my box from within the ebuild is no longer possible
> Please implement the suggestions.

What suggestions? As far as I can see, you haven't really suggested 
anything useful. I, on the other hand, have repeatedly told you how we 
can prevent these problems with solutions like GPG signing. You say 
that's not enough - and go on to make no real suggestions other than 
"this should be fixed!"

Try being specific.

> 
> Do you understand me?

If you start getting patronizing with me, I'm going to apply an 
excellent patch that'll fix the entire issue - the procmailrc sort. I 
don't have time or patience for it.

> 
> If still not, i will remind you of former times, when bootblock virii have 
> been very successfull. Why do you think they have been successfull and are no 
> longer?
> Answer: Because it was possible to modify the bootblock.
> 
> It is possible for an ebuild to wipe your box.
> 
> So doesnt this scare you now?
> 
> Jan

No. It's also possible for a truck full of propane to run into 
my house. That doesn't encourage me to put up a concrete wall all around 
my house.

-- 
Jon Portnoy
avenj/irc.freenode.net

--
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] suggestion portage ebuild system file modification rights and protection

[Chris Bainbridge]

On Sunday 07 September 2003 21:43, Jan Krueger wrote:
> On Sunday 07 September 2003 19:20, Martin Schlemmer wrote:
> > So how are any of these going to help if you do not trust us or any
> > other developers/upstream_authors, encryption, etc, etc.  I mean,
> > this *IS* what this whole issue is about, no ?
>
> No. I trust you. But trusting you doesnt mean that the ebuild you checked
> in to the tree arrives at my hardrive unmodified. There is no way for you
> as a human beeing to garantee this to me. Instead it should be expected
> that the ebuild gets modified (by faulty software/hardware/network/whatever
> or by a malicious attacker). So this must be taken care of.
>
> With Manifest and digest portage very much points in the right direction,
> but this is not enough, from my point of view.
>
> Jan

This has been discussed before ( http://bugs.gentoo.org/show_bug.cgi?id=5902 
). I think the gpg signatures development got put on hold because there was 
talk of making individuals responsible for packages (like Debian), rather 
than the system at the moment where a small core does all of the work.

My proposal was to use signatures along with the concept of requiring a 
certain number of developers to "sign off" an ebuild. Its important that the 
compromise of a single developer with cvs access shouldn't impact thousands 
of users. Therefore, most packages should require two or more developer 
signatures before they will be installed. 

Using a secure distribution infrastructure (eg. rsync over ssl) is not an 
option if gentoo is going to be distributed over untrusted p2p networks 
(which I think it will in the future). 


--
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] suggestion portage ebuild system file modification rights and protection

[Martin Schlemmer]

[-- Attachment #1: Type: text/plain, Size: 835 bytes --]

On Mon, 2003-09-08 at 03:50, Jan Krueger wrote:
> On Sunday 07 September 2003 23:41, Chris Bainbridge wrote:
> > This has been discussed before (
> > http://bugs.gentoo.org/show_bug.cgi?id=5902 ). I think the gpg signatures
> > development got put on hold because there was talk of making individuals
> > responsible for packages (like Debian), rather than the system at the
> > moment where a small core does all of the work.
> Thank you for this information. Sounds good :)
> unfortunatly i read it after i answered the mail of Jon Portnoy.
> 

I thought this is what Jon have been saying (yes, he did not go
into that much specifics, but you did not seem to care, and rejected
it anyhow for some reason).


-- 

Martin Schlemmer
Gentoo Linux Developer, Desktop/System Team Developer
Cape Town, South Africa



[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-dev] suggestion portage ebuild system file modification rights and protection

[Martin Schlemmer]

[-- Attachment #1: Type: text/plain, Size: 2356 bytes --]

On Mon, 2003-09-08 at 04:08, Jan Krueger wrote:

> > > Again examples from the actual tree, so you can try yourself:
> > > 1. emerge ezmlm and emerge ezmlm-idx
> > > providing slightly different funtionality they will overwrite each other
> > > (instead of blocking each other)
> >
> > Bug. Is it filed?
> Bug in portage! portage is the one that allows such integrity mess.
> 

Whoever just forgot to add a 'DEPEND="!ezmlm-idx"' to ezmlm, and
reverse for ezmlm-idx ?  I do not see how portage will cause that
individual(s) to forget about that ?

> > So we don't have enough manpower.
> Thats true for many open-source project. Some of them just try to get 
> organized more efficiently and succeed in doing so.
> So, maybe there is a more appropriate organization model for gentoo?
> 

I am also guessing you have not read GWN, and -dev for the last two
months or so ?

> > > And to me its clear why it is like that (at least on reason):
> i meant to say: (at least one reason)
> sorry.
> 
> > So basically you're saying portage shouldn't install software.
> I say:
> portage must respect my system inegrity!
> 

Ok, but the merge code in portage could have a bug bigger than anything
pkg_{post,pre}inst() could ever cause.  Right, so that is why we need
all the other safety nets - they could be more buggy ?

> > So we should never be able to tweak config files et al in an ebuild?
> an ebuild may freely modify its own config files.
> modification of config files not belonging to the ebuild should be done via an 
> already suggested, secure abstraction, lets say a function like:
>  changeconf phph.ini "line to add to phpini"
> portage could then intercept, respecting the suggested CONFIG_EXCLUDE or other 
> user settings, or, if no user setting is the way, go to apply the change.
> This way it would be impossible for the ebuild to wipe php.ini.
> Also the user, via CONFIG_EXCLUDE, may completely switch of editing of php.ini 
> by ebuilds. On the other hand, if the user doesnt care, the ebuild is free to 
> add this line to php.ini.
> 

Some times it is not so easy.

Unfortunately black and white on paper usually is much more seperate
issues than real live could ever be.


-- 

Martin Schlemmer
Gentoo Linux Developer, Desktop/System Team Developer
Cape Town, South Africa



[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-dev] suggestion portage ebuild system file modification rights and protection

[Martin Schlemmer]

[-- Attachment #1: Type: text/plain, Size: 933 bytes --]

On Mon, 2003-09-08 at 04:33, Jan Krueger wrote:
> On Monday 08 September 2003 00:22, Martin Schlemmer wrote:
> > > Thank you for this information. Sounds good :)
> > > unfortunatly i read it after i answered the mail of Jon Portnoy.
> >
> > I thought this is what Jon have been saying
> I did not read something about:
> 
> "making individuals responsible for packages (like Debian)"
> 
> or
> 
> "to use signatures along with the concept of requiring a 
> certain number of developers to "sign off" an ebuild. Its important that the 
> compromise of a single developer with cvs access shouldn't impact thousands 
> of users. Therefore, most packages should require two or more developer 
> signatures before they will be installed."
> 
> in the mail of Jon. Sorry.
> 

But you did not ask either.


-- 

Martin Schlemmer
Gentoo Linux Developer, Desktop/System Team Developer
Cape Town, South Africa



[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-dev] suggestion portage ebuild system file modification rights and protection

[Martin Schlemmer]

[-- Attachment #1: Type: text/plain, Size: 1504 bytes --]

On Mon, 2003-09-08 at 04:52, Jan Krueger wrote:
> On Monday 08 September 2003 00:28, Martin Schlemmer wrote:
> > Whoever just forgot to add a 'DEPEND="!ezmlm-idx"' to ezmlm, and
> > reverse for ezmlm-idx ?  I do not see how portage will cause that
> > individual(s) to forget about that ?
> Portage allows packages overwriting each other and this i functional 
> deficiency that can not get worser as it is.
> 

So you want it to check _EVERY_ file that it do not maybe
belong to another package before merging it to the live filesystem ?
Oh boy, that really will make portage a speedster.

> > Ok, but the merge code in portage could have a bug bigger than anything
> > pkg_{post,pre}inst() could ever cause.
> Yes, ist software.
> 
> >  Right, so that is why we need
> > all the other safety nets - they could be more buggy ?
> Sorry, i cant follow you now.
> 

Meaning, you do not trust software, but software should fix
your 'problem' ?

> > Some times it is not so easy.
> 
> I know...
> but should this prevent us from trying?
> 

Ok, lets rip out all the pkg_*inst() stuff.  I am guessing
I should not count on you to sweep the pieces together again ?

What about taking a nice holiday for a change, then come back
after you pulled up a nice 100+ page draft with suggestions,
working code, and some sample changes to offensive ebuilds?


Thanks,

-- 

Martin Schlemmer
Gentoo Linux Developer, Desktop/System Team Developer
Cape Town, South Africa



[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-dev] gentoo-project

[Martin Schlemmer]

[-- Attachment #1: Type: text/plain, Size: 824 bytes --]

On Mon, 2003-09-08 at 05:12, Jan Krueger wrote:
> On Monday 08 September 2003 01:02, Martin Schlemmer wrote:
> > But you did not ask either.
> 
> Ok, thats true. Based on what i have seen so far i was assuming certain 
> aspects. That may have been wrong. Your right. So:
> How is the gentoo-project going to be organized in general?
> Specifically to ensure quality?
> and security?
> 

I am not one to give heads up on the gpg stuff.  Check that
bug, and hopefully the guys in the 'know' will respond.

Also, two projects, selinux and hardened-gcc is picking up
speed, and we have had for some time now propolice patched
into gcc - but all these are not portage related really, that
I know.


-- 

Martin Schlemmer
Gentoo Linux Developer, Desktop/System Team Developer
Cape Town, South Africa



[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-dev] suggestion portage ebuild system file modification rights and protection

[Jan Krueger]

On Sunday 07 September 2003 20:35, Jon Portnoy wrote:
> What, that any situation involving installing software is going to have
> security holes? That's the nature of software installation.

The gift of portage is a new quality of automated software installation 
system, easy to use, never seen before. This is good.

However by being a full automated software installation system portage has a 
very high responsibilty for the integrity of the system it is run on.

The discussion should have shown you, that the current portage can not bear 
this responsibility.

This thread showed and will continue to show:
ebuilds can overwrite any system file
ebuilds can wipe your box
ebuilds are affecting system integrity
ebuilds lack quality

Some of you think this can be addressed by:
qa
signing ebuilds and files

I say:
this is not sufficient.

Again examples from the actual tree, so you can try yourself:
1. emerge ezmlm and emerge ezmlm-idx
providing slightly different funtionality they will overwrite each other 
(instead of blocking each other)

2. emerge heartbeat (sys-cluster/heartbeat-1.0.3-r1, marked x86, so considered 
stable)
If your emerge was successfull (it will be as long as you dont have snmp 
installed), try to get it running! You wont! Then take a look at 
bugs.gentoo.org since when important fixes are available.

This two examples clearly show what the current qa is worth. (Well i respect 
it very much, but its not perfect and never will be, thats qa, qa you for 
sure know from other software/hardware products you use and used before, you 
know the deficencies of qa then)
I am pretty sure you can post examples from the tree yourself.

And to me its clear why it is like that (at least on reason):
If you compare the number of ebuilds to the number of CVS committers than you 
will see, that it is impossible for each CVS-commiter to make a valuable 
statement of the quality of the software and/or ebuild he/she is going to 
commit. There are to many ebuilds.

(i see this as an organizational deficiency of gentoo)
So it is probable that faulty ebuilds hit the tree. They are there actually.

Conclusion for me:
I expect faulty ebuilds. they are there.  (you will never wipe them out until 
you change the organization of gentoo development to a more apropriate model, 
and even then, you can only reduce the amount of faulty ebuilds. ebuilds are 
software, software has bugs.)

and therefore:
i hold portage responsible to protect my system from faulty ebuilds as much as 
it can.
if not portage, who else? there is just me left, but then, i dont need 
portage.

I made several suggestions how portage can be improved to protect the system 
it is run on. 
-prevent ebuilds from modifying the life filesystem
-allow the actual package install process only to add files to the filesystem 
or to only modify/remove files that belong to an revision of the same ebuild
(qa would benefit from this suggestions too.)
This would result in:
-enforced package integrity
-wiping my box from within the ebuild is no longer possible
Please implement the suggestions.

Do you understand me?

If still not, i will remind you of former times, when bootblock virii have 
been very successfull. Why do you think they have been successfull and are no 
longer?
Answer: Because it was possible to modify the bootblock.

It is possible for an ebuild to wipe your box.

So doesnt this scare you now?

Jan


--
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] suggestion portage ebuild system file modification rights and protection

[Jan Krueger]

On Sunday 07 September 2003 20:35, Jon Portnoy wrote:
> What, that any situation involving installing software is going to have
> security holes? That's the nature of software installation.

Installing software at the end comes down to putting files at the right place.
(on windows you would add: modifying the registry)

So thats exactly what portage should do: put files at the right place.

The ebuilds may play in the sandbox whatever game they like.
It should however in no way possible for them to wipe your box.

You agree?

Jan


--
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] gentoo-project

[Seemant Kulleen]

[-- Attachment #1: Type: text/plain, Size: 1845 bytes --]

> Ok, thats true. Based on what i have seen so far i was assuming
> certain aspects. That may have been wrong. Your right. So:
> How is the gentoo-project going to be organized in general?
> Specifically to ensure quality?
> and security?

Exactly what do you mean by "going to be organized in general"?  You
seem to imply we are not organised and just a bunch of hacker-monkeys
banging away.  I can assure you we are not.  If you are interested in
the way the project itself is structured, see
http://www.gentoo.org/proj/en/metastructure

If you are interested in how we are handling the inevitable gaps in
communication that comes with an ever-growing project, look at:
http://www.gentoo.org/proj/en/devrel

If you have any specific questions, please ask those instead of the
approach you're currently taking.  The perception of your posts has been
as though you have been relentlessly *attacking* us, who provide *you*
with a way of doing this -- you, who chooses to use it of your own
volition.  That is really not very constructive, and will lead to people
getting defencive.  As you have seen, this thread degenerated.  (My own
general rule of thumb is that if a thread goes longer than 5 messages
with a repeat/restate, it's not worth reading, but anyway).  

I, and I am sure I speak for others, request that you take your
thoughts, and present them with details, and in a cohesive manner, and
then present them, instead of bomarding this list with restated vague
and general questions.

Sorry if I'm sounding harsh, but I'd rather see this thread concluding
on a productive note.

Thanks,

-- 
Seemant Kulleen
Developer and Project Co-ordinator,
Gentoo Linux					http://dev.gentoo.org/~seemant

Public Key: http://pgp.mit.edu:11371/pks/lookup?op=get&search=0x3458780E
Key fingerprint = 23A9 7CB5 9BBB 4F8D 549B 6593 EDA2 65D8 3458 780E

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-dev] suggestion portage ebuild system file modification rights and protection

[Jan Krueger]

On Sunday 07 September 2003 23:41, Chris Bainbridge wrote:
> This has been discussed before (
> http://bugs.gentoo.org/show_bug.cgi?id=5902 ). I think the gpg signatures
> development got put on hold because there was talk of making individuals
> responsible for packages (like Debian), rather than the system at the
> moment where a small core does all of the work.
Thank you for this information. Sounds good :)
unfortunatly i read it after i answered the mail of Jon Portnoy.

> My proposal was to use signatures along with the concept of requiring a
> certain number of developers to "sign off" an ebuild. Its important that
> the compromise of a single developer with cvs access shouldn't impact
> thousands of users. Therefore, most packages should require two or more
> developer signatures before they will be installed.
Sounds good too :)

> Using a secure distribution infrastructure (eg. rsync over ssl) is not an
> option if gentoo is going to be distributed over untrusted p2p networks
> (which I think it will in the future).
Ok, forget about ssl/ssh for now.

Jan


--
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] suggestion portage ebuild system file modification rights and protection

[Thomas de Grenier de Latour]

On Mon, 8 Sep 2003 02:08:51 +0000
Jan Krueger <jk@microgalaxy.net> wrote:

> On Sunday 07 September 2003 23:41, Jon Portnoy wrote:
> > You haven't explained why it's not sufficient.
> Read again.
> 
> > > Again examples from the actual tree, so you can try yourself:
> > > 1. emerge ezmlm and emerge ezmlm-idx
> > > providing slightly different funtionality they will overwrite each
> > > other(instead of blocking each other)
> >
> > Bug. Is it filed?
> Bug in portage! portage is the one that allows such integrity mess.

I vote for the "bug in ebuild". In a recent thread [1] I've posted the
results of a "search for files that belongs to several packages" on my
system, and it's true that there are many such packages (nothing to
worry about in general though). I think what lacks to gentoo developers
(please devs, correct me if I'm wrong) is control mechanism do detect
this kind of overwrites: they don't have all portage packages installed
on their computer, and even if it was the case, portage does makes much
noise about overwritings. Hence they don't detect this kind of bugs, but
they are still package specific.

   Now, to improve this situation I imagine a kind of database of the
different pkg-ver contents. When a dev tests a new ebuild for foo/bar,
he could check against this base that the content of his package won't
conflict with any other already existing package but other versions of
foo/bar (idealy in the same slot). If there are conflicts, then he could
add some blocking dependencies, or rename the files, or decide to accept
the overwrite, and then submit the definitive content of his package
to the database. 

I think in the above mentioned thread, somebody also suggested that
overwritten files (excluding updates) should be archived by portage
for safety (or at the contrary that they should not be merged and would
require the user to explicitly make the overwriting). The idea sounds
good, but his redundant with the above mentioned package content
checking. I don't know what would be the best. I think this second one
is closer to the behavior you expect, but I fear the "yet another task
for emerge" effect which may compromise its speed. And I fear "user
bugs": so many people forget to etc-update after a baselayout upgrade
and then complain, so please, don't give them a "bin-update".

And all this things may be useful from a quality point of view, but
should not be consider as some "security" measures. Overwriting are
often not needed for malicious code be executed by root: puting a file
with the right name in another place of the path (or a library
in /usr/local/lib for instance) will often be enough. We are back to the
'trust ebuilds and devs' situation.

Now about config files: sure things may be done differently, with maybe
a little more control for the user. But in ebuilds I've read so far,
postint commands that modify config files are really not harmful. Who
want to manage by hand scrollkeeper entries or this kind of things? If
you have examples of existing ebuilds with post_install commands that
doesn't feet your needs, I'm sure bug reports will be welcome.

>  changeconf phph.ini "line to add to phpini"

Can be done in src_install: read it, modify it, write it. You will 
have access to anything you really need (grep, sed, etc.), and user will
have to accept the file with etc-update/dispatch-conf. But again, if it
may be justified for php.ini, it is not for an xml catalog or other
ugly things like this, and there the postinst commands are really
handier.


[1] http://article.gmane.org/gmane.linux.gentoo.devel/11630

-- 
TGL.

--
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] suggestion portage ebuild system file modification rights and protection

[Jan Krueger]

On Sunday 07 September 2003 23:41, Jon Portnoy wrote:
> You haven't explained why it's not sufficient.
Read again.

> > Again examples from the actual tree, so you can try yourself:
> > 1. emerge ezmlm and emerge ezmlm-idx
> > providing slightly different funtionality they will overwrite each other
> > (instead of blocking each other)
>
> Bug. Is it filed?
Bug in portage! portage is the one that allows such integrity mess.

> So we don't have enough manpower.
Thats true for many open-source project. Some of them just try to get 
organized more efficiently and succeed in doing so.
So, maybe there is a more appropriate organization model for gentoo?

> > And to me its clear why it is like that (at least on reason):
i meant to say: (at least one reason)
sorry.

> So basically you're saying portage shouldn't install software.
I say:
portage must respect my system inegrity!

> So we should never be able to tweak config files et al in an ebuild?
an ebuild may freely modify its own config files.
modification of config files not belonging to the ebuild should be done via an 
already suggested, secure abstraction, lets say a function like:
 changeconf phph.ini "line to add to phpini"
portage could then intercept, respecting the suggested CONFIG_EXCLUDE or other 
user settings, or, if no user setting is the way, go to apply the change.
This way it would be impossible for the ebuild to wipe php.ini.
Also the user, via CONFIG_EXCLUDE, may completely switch of editing of php.ini 
by ebuilds. On the other hand, if the user doesnt care, the ebuild is free to 
add this line to php.ini.

Jan


--
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] suggestion portage ebuild system file modification rights and protection

[Jan Krueger]

On Monday 08 September 2003 00:22, Martin Schlemmer wrote:
> > Thank you for this information. Sounds good :)
> > unfortunatly i read it after i answered the mail of Jon Portnoy.
>
> I thought this is what Jon have been saying
I did not read something about:

"making individuals responsible for packages (like Debian)"

or

"to use signatures along with the concept of requiring a 
certain number of developers to "sign off" an ebuild. Its important that the 
compromise of a single developer with cvs access shouldn't impact thousands 
of users. Therefore, most packages should require two or more developer 
signatures before they will be installed."

in the mail of Jon. Sorry.


--
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] suggestion portage ebuild system file modification rights and protection

[Jan Krueger]

On Monday 08 September 2003 00:28, Martin Schlemmer wrote:
> Whoever just forgot to add a 'DEPEND="!ezmlm-idx"' to ezmlm, and
> reverse for ezmlm-idx ?  I do not see how portage will cause that
> individual(s) to forget about that ?
Portage allows packages overwriting each other and this i functional 
deficiency that can not get worser as it is.

> > > So we don't have enough manpower.
> >
> > Thats true for many open-source project. Some of them just try to get
> > organized more efficiently and succeed in doing so.
> > So, maybe there is a more appropriate organization model for gentoo?
>
> I am also guessing you have not read GWN, and -dev for the last two
> months or so ?
Yes, i am new to -dev. I dont remember reading about this in GWN, maybe i have 
missed an issue.


> Ok, but the merge code in portage could have a bug bigger than anything
> pkg_{post,pre}inst() could ever cause.
Yes, ist software.

>  Right, so that is why we need
> all the other safety nets - they could be more buggy ?
Sorry, i cant follow you now.

> Some times it is not so easy.

I know...
but should this prevent us from trying?


--
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] gentoo-project

[Jon Portnoy]

On Mon, Sep 08, 2003 at 04:54:04AM +0000, Jan Krueger wrote:
> On Monday 08 September 2003 04:34, Jan Krueger wrote:
> > Thank you for the links, i was unable to find them from them main page.
> I was lying assuming they contain new information, sorry. i already read them 
> several times.
> 
> If thats how its going to be, its not for me.
> 
> I undertand that my views are not shared here so i better quit.
> 
> Thank you all for discussing with me.
> 
> Best wishes
> Jan

You haven't shared many of your views - you've just been consistently 
patronizing and repeated yourself without elaborating or expressing 
specific concepts. Meanwhile, you've tried to act like we're 
inexperienced merely because you disagree with our alternative 
solutions - not to mention accusing Az of being close-minded.

That's not productive.

-- 
Jon Portnoy
avenj/irc.freenode.net

--
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] gentoo-project

[Jan Krueger]

On Monday 08 September 2003 01:02, Martin Schlemmer wrote:
> But you did not ask either.

Ok, thats true. Based on what i have seen so far i was assuming certain 
aspects. That may have been wrong. Your right. So:
How is the gentoo-project going to be organized in general?
Specifically to ensure quality?
and security?

Jan


--
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] gentoo-project

[Bill Kenworthy]

I think there is a language issue here, and Jan may not realise how
offensive his emails have been in English as its obviously not his
native language.  He has some good points, but the way he has presented
them has been a liability for his arguments

It might be better if he can put the points he has made in a bugzilla
entry so they can be properly considered and not lost?

BillK


On Mon, 2003-09-08 at 11:03, Jon Portnoy wrote:
> On Mon, Sep 08, 2003 at 04:54:04AM +0000, Jan Krueger wrote:
> > On Monday 08 September 2003 04:34, Jan Krueger wrote:
> > > Thank you for the links, i was unable to find them from them main page.
> > I was lying assuming they contain new information, sorry. i already read them 
> > several times.
> > 
> > If thats how its going to be, its not for me.
> > 
> > I undertand that my views are not shared here so i better quit.
> > 
> > Thank you all for discussing with me.
> > 
> > Best wishes
> > Jan
> 
> You haven't shared many of your views - you've just been consistently 
> patronizing and repeated yourself without elaborating or expressing 
> specific concepts. Meanwhile, you've tried to act like we're 
> inexperienced merely because you disagree with our alternative 
> solutions - not to mention accusing Az of being close-minded.
> 
> That's not productive.


--
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] gentoo-project

[Jon Portnoy]

On Mon, Sep 08, 2003 at 11:47:05AM +0800, Bill Kenworthy wrote:
> I think there is a language issue here, and Jan may not realise how
> offensive his emails have been in English as its obviously not his
> native language.  He has some good points, but the way he has presented
> them has been a liability for his arguments
> 
> It might be better if he can put the points he has made in a bugzilla
> entry so they can be properly considered and not lost?
> 

Definitely.

-- 
Jon Portnoy
avenj/irc.freenode.net

--
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] gentoo-project

[Martin Schlemmer]

[-- Attachment #1: Type: text/plain, Size: 5246 bytes --]

On Mon, 2003-09-08 at 07:33, Jan Krueger wrote:
> On Monday 08 September 2003 03:03, Jon Portnoy wrote:
> 
> > You haven't shared many of your views
> Here they are again summed up for the interested reader:
> 
> -prevent ebuilds from modifying the life filesystem (from pkg_postinst for 
> example), portage is the only one allowed to do so. that means a real sandbox 
> over full ebuild time. the image is ready after src_install. portage than 
> puts the files at the right place. The ebuild itself can in no way touch the 
> live filesystem. there is no need for the ebuild to do so.
> (putting the build system into UML would be a considerable option for this. 
> maybe oversized)
> 

This is not always possible as I have said before.  Mozilla for instance
have many issues if you do not remove the old chrome files and registry
to name one.  Another is baselayout that is in general a pita.

> -allow the actual package install process only to add files to the filesystem 
> or to only modify/remove files that belong to an revision of the same ebuild
> (qa would benefit from this suggestions too.). Portage, before putting all the 
> files from the image into the life filesystem, scans the image for all files 
> in there. now it has a list of files it is going to install. So it 
> scans now the live filesystem if thes files, or some of them exist. If they 
> exist in the live filesystem, portage checks if they belong to a revision of 
> the same ebuild.
>  -if they dont belong to a revision of the same ebuild and are not found in 
> the live filesystem it would be an addition of new software so the files are 
> put into action
>  -if they are found in the filesystem and belong to a revision of the same 
> ebuild it would be an upgrade or downgrade and are put into action
>  -if they are found in the filesystem and do not belong to a revision of the 
> same ebuild -> something is wrong (might be init going to be overwritten) -> 
> inform the user and fail
> 

This might be an good idea, but it will slow down things a lot.  This is
Nick's (carpaski) domain, so we will have to check with him.

> -provide an secure abtraction for things, like adding values to global config 
> files, depmod -a, that may be required to do after installing the files to 
> the life filesystem.
> 

Its not often that this is really done.  The only places, this was done
with some care (look at pshadow and pam-login for examples), and a
backup, etc was made ...

> >From my answer to Jon:
> > So we should never be able to tweak config files et al in an ebuild?
> an ebuild may freely modify its own config files.
> modification of config files not belonging to the ebuild should be done via an 
> already suggested, secure abstraction, lets say a function like:
>  changeconf phph.ini "line to add to phpini"
> portage could then intercept, respecting the suggested CONFIG_EXCLUDE or other 
> user settings, or, if no user setting is the way, go to apply the change.
> This way it would be impossible for the ebuild to wipe php.ini.
> Also the user, via CONFIG_EXCLUDE, may completely switch of editing of php.ini 
> by ebuilds. On the other hand, if the user doesnt care, the ebuild is free to 
> add this line to php.ini.
> -

As said above .. its not often that you really need to hack the config
files.  The docbook/xml/sgml stuff may be another not so common example
where it is best to do it without user interaction.

> another one was the above mentioned
> CONFIG_EXCLUDE in /etc/make.conf:
> This variable would accept a list of directories/files for which the behaviour 
> of portage would be like follows:
> whenever portage has the image of the to install software ready it scans this 
> image for the values in CONFIG_EXCLUDE.
> whenever it finds such a directory/file in the image it moves the 
> directory/file to the doc-directory (eg: 
> /usr/share/doc/${PF}/excluded_config/ ) of the image (and maybe writes a 
> message to the user/log)
> after that portage continues normally.
> 

As I said in a previous mail, the CONFIG_EXCLUDE feature may
come in handy, and will solve the make.conf issue for one.

> > repeated yourself without elaborating or expressing
> > specific concepts.
> If, in your eyes, above things arent concepts and specific i dont know what 
> else actually is. I dont understand you, you dont understand me, so thats the 
> wrong place for me, i quit.
> 
> I always tried to be strictly technical, maybe sometimes i failed. i am human.
> Sorry. Dont hate me, i, as you, only try to do my best.
> 

Relax, just do not be so pushy from day one.  Try to find out more
first.  Maybe try to create a few ebuilds, or try to update some.
Maybe have a look at some open bugs, and try to help fix them.
Alternatively you might try to have a look at the portage code.

Basically, get involved, see how things is done (not a objective
view from the outside without having been involved first).  Then
recheck what you think, and bring it to the table with enough
why's and why not's.


Regards,

-- 

Martin Schlemmer
Gentoo Linux Developer, Desktop/System Team Developer
Cape Town, South Africa



[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-dev] gentoo-project

[Jan Krueger]

On Monday 08 September 2003 01:44, Seemant Kulleen wrote:
> > Ok, thats true. Based on what i have seen so far i was assuming
> > certain aspects. That may have been wrong. Your right. So:
> > How is the gentoo-project going to be organized in general?
> > Specifically to ensure quality?
> > and security?
>
> Exactly what do you mean by "going to be organized in general"?  You
> seem to imply we are not organised and just a bunch of hacker-monkeys
> banging away. 
No, i was assuming the organization is changing/developing.

Thank you for the links, i was unable to find them from them main page.

Jan


--
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] suggestion portage ebuild system file modification rights and protection

[Jan Krueger]

On Monday 08 September 2003 01:12, Martin Schlemmer wrote:
> What about taking a nice holiday for a change, then come back
> after you pulled up a nice 100+ page draft with suggestions,
> working code, and some sample changes to offensive ebuilds?
There is no need. Its all out there on the net. Go read it.
Dont look for "ebuilds", look for "ports", "packages", "automated software 
installation" and learn.
Open up a little bit, install it, try it out and learn.
Its not that portage is something completly new. really. It just introduces 
some nice enhancements to the existing things out there.
Others did research on automated software installation. Others already made 
the mistakes. Even the organizational ones (from my point of view). Its out 
there.

here is the url:
http://www.google.com




--
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] gentoo-project

[Jan Krueger]

On Monday 08 September 2003 04:34, Jan Krueger wrote:
> Thank you for the links, i was unable to find them from them main page.
I was lying assuming they contain new information, sorry. i already read them 
several times.

If thats how its going to be, its not for me.

I undertand that my views are not shared here so i better quit.

Thank you all for discussing with me.

Best wishes
Jan



--
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] gentoo-project

[Jan Krueger]

On Monday 08 September 2003 03:03, Jon Portnoy wrote:

> You haven't shared many of your views
Here they are again summed up for the interested reader:

-prevent ebuilds from modifying the life filesystem (from pkg_postinst for 
example), portage is the only one allowed to do so. that means a real sandbox 
over full ebuild time. the image is ready after src_install. portage than 
puts the files at the right place. The ebuild itself can in no way touch the 
live filesystem. there is no need for the ebuild to do so.
(putting the build system into UML would be a considerable option for this. 
maybe oversized)

-allow the actual package install process only to add files to the filesystem 
or to only modify/remove files that belong to an revision of the same ebuild
(qa would benefit from this suggestions too.). Portage, before putting all the 
files from the image into the life filesystem, scans the image for all files 
in there. now it has a list of files it is going to install. So it 
scans now the live filesystem if thes files, or some of them exist. If they 
exist in the live filesystem, portage checks if they belong to a revision of 
the same ebuild.
 -if they dont belong to a revision of the same ebuild and are not found in 
the live filesystem it would be an addition of new software so the files are 
put into action
 -if they are found in the filesystem and belong to a revision of the same 
ebuild it would be an upgrade or downgrade and are put into action
 -if they are found in the filesystem and do not belong to a revision of the 
same ebuild -> something is wrong (might be init going to be overwritten) -> 
inform the user and fail

-provide an secure abtraction for things, like adding values to global config 
files, depmod -a, that may be required to do after installing the files to 
the life filesystem.

From my answer to Jon:
> So we should never be able to tweak config files et al in an ebuild?
an ebuild may freely modify its own config files.
modification of config files not belonging to the ebuild should be done via an 
already suggested, secure abstraction, lets say a function like:
 changeconf phph.ini "line to add to phpini"
portage could then intercept, respecting the suggested CONFIG_EXCLUDE or other 
user settings, or, if no user setting is the way, go to apply the change.
This way it would be impossible for the ebuild to wipe php.ini.
Also the user, via CONFIG_EXCLUDE, may completely switch of editing of php.ini 
by ebuilds. On the other hand, if the user doesnt care, the ebuild is free to 
add this line to php.ini.
-
another one was the above mentioned
CONFIG_EXCLUDE in /etc/make.conf:
This variable would accept a list of directories/files for which the behaviour 
of portage would be like follows:
whenever portage has the image of the to install software ready it scans this 
image for the values in CONFIG_EXCLUDE.
whenever it finds such a directory/file in the image it moves the 
directory/file to the doc-directory (eg: 
/usr/share/doc/${PF}/excluded_config/ ) of the image (and maybe writes a 
message to the user/log)
after that portage continues normally.

> repeated yourself without elaborating or expressing
> specific concepts.
If, in your eyes, above things arent concepts and specific i dont know what 
else actually is. I dont understand you, you dont understand me, so thats the 
wrong place for me, i quit.

I always tried to be strictly technical, maybe sometimes i failed. i am human.
Sorry. Dont hate me, i, as you, only try to do my best.

Have a nice time

Jan


--
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] suggestion portage ebuild system file modification rights and protection

[Michael Cummings]

Jan,

        I mean no offense, but to answer some of your questions in the
        latest addition to this thread...
        
On Mon, Sep 08, 2003 at 01:40:32AM +0000, Jan Krueger wrote:
> 
> Installing software at the end comes down to putting files at the right place.
> (on windows you would add: modifying the registry)
> 
> So thats exactly what portage should do: put files at the right place.
> 
Portage is nothing more than a middle man. Ebuilds are recipes at best. We
rely almost entirely on the upstream author to put the files where they are
supposed to go. Functions like pkg_preinst and pkg_postinst exist
because not all upstream authors concur on where their files should go;
because inevitably a tweak here and there is needed to keep user
interaction at zarro.


> The ebuilds may play in the sandbox whatever game they like.
> It should however in no way possible for them to wipe your box.
> 

Symantics, I know, but the ebuild isn't wiping your box. A poor piece of
product control, perhaps, but an ebuild is just a pretty bash script. Are
there wheels in motion to counter this possibility? Of course. A big one
in my opinion is the consideration of a staggered portage tree, so that an
ebuild commit today doesn't mean its available tomorrow, but that
instead there is a grace period to work from in case "something bad"
crops up.

I think you're being misread in this thread, but I also think you are losing
sight of the original intent of a metadistribution - let people have it
their way. We do this in our spare time, all of us, and we do it "for the
love of the game." And that love gets hard to see sometimes. It's get to be
paranoid about security - just remember we're trying.

Egads, I need sleep folks. And coffee. Lots of coffee. Jan, one last thing -
If you don't trust an ebuild to merge properly, then break it
out. ebuild /path/to/foo/bar install; cd /var/tmp/portage/foo/bar/image;
<look around>; ebuild /path/to/foo/bar merge;

Enjoy! Shop SMART, shop S-Mart!

-- 


-----o()o---------------------------------------------
              |  http://www.gentoo.org/
              |  #gentoo-dev  on irc.freenode.net
Gentoo Dev    |  #gentoo-perl on irc.freenode.net
Perl Guy      |
              |  GnuPG Key ID:       AB5CED4E9E7F4E2E
-----o()o---------------------------------------------


--
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Some suggestions

[Nathaniel]

Regarding the etc-update issue, here are some of my thoughts.

1. etc-update should check modify dates and overwrite if not modified.

2. Part of the problem is that make.conf has so many options (ie. its a
large file.)  What if we split it up into smaller files... Something
like:

/etc/make.conf.d/USE
/etc/make.conf.d/CFLAGS
...
/etc/make.conf.d/FEATURES
etc...

I would guess that a large portion of users never modify the features
and other settings of make.conf.  This way, etc-update could update only
the portions that need updating without overwriting USE flags, etc. 
This would also make it easier to parse the files for any automated
install (GLIS), etc.

3. Another option is to have a file that contains the users settings,
seperate from config files themselves... For instance, what if we had a
file, say /etc/customsettings, that contained all the updated options
for config files.  It could perhaps contain a syntax like the following:

/etc/make.conf:USE="X gnome gtk alsa"
/etc/make.conf:FEATURES="distcc sandbox buildpkg"
/etc/conf.d/net:IFACE="dhcp"

Then etc-update could not only replace the old file with the update, but
could also update it with the user's specific values.  It could find the
line with the "USE=" text and replace it with the full customized
replacement.

My $.02.
-- 
Nathaniel McCallum
npmccallum@users.sourceforge.net
http://glis.sourceforge.net


--
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Some suggestions

[Ferris McCormick]

On Mon, 8 Sep 2003, Nathaniel wrote:

> Regarding the etc-update issue, here are some of my thoughts.
> 
> 1. etc-update should check modify dates and overwrite if not modified.
> 
I haven't followed this very closely because there's so much of it, but
this one suddenly woke me up.  Just because a file hasn't been modified
doesn't imply that it's OK to change it, does it?  Maybe it's fine the
way it is, but won't be if it gets changed?  Appologies if I'm rehashing
something.

Regards,

--
Ferris McCormick (P44646, MI) <fmccor@inforead.com>
Phone: (703) 392-0303
Fax:   (703) 392-0401


--
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Some suggestions

[Steven Elling]

On Saturday 06 September 2003 19:57, Chris Gianelloni wrote:
> On Sat, 2003-09-06 at 20:26, Steven Elling wrote:
> > Does portage bomb out if make.conf is not present?  If so, maybe
> > portage needs to be changed so that it will work without the file.
>
> I definitely like the idea of creating make.conf docs and a link from
> the install docs.  Also, why can't the portage ebuild contain a 0 byte
> make.conf file?

A zero byte make.conf file would work with certain conditions.  Those 
conditions would be:

1. If the config file make.conf does not exists, create it.
2. If the config file does exist because the admin put it there or portage 
did, do not try to install a new one.

I've seen several source installs do this when I was doing LFS.


--
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Some suggestions

[Steven Elling]

On Saturday 06 September 2003 22:08, Martin Schlemmer wrote:
> On Sun, 2003-09-07 at 02:57, Chris Gianelloni wrote:
> > On Sat, 2003-09-06 at 20:26, Steven Elling wrote:
> > > Does portage bomb out if make.conf is not present?  If so, maybe
> > > portage needs to be changed so that it will work without the file.
> >
> > I definitely like the idea of creating make.conf docs and a link from
> > the install docs.  Also, why can't the portage ebuild contain a 0 byte
> > make.conf file?  After all, the file CAN be empty and portage will
> > still work from the make.globals since make.conf serves no purpose but
> > to override the system defaults.  We could include a make.conf file in
> > the stage tarballs with a few settings (depending on the settings of
> > the stage) and a comment telling the user where the docs for make.conf
> > are located both locally and on Gentoo.org.
>
> I really do not see how having to either:
>
> 1) Copy and paste everything
>
> 2) Type if from a printout/whatever
>
> is efficient or helps the average user?  It is way easier
> to just uncomment and change as needed with the help, etc
> there in front of you.

It isn't efficient.  Keeping documentation out of make.conf, putting 
documentation where it should be (man page and www.gentoo.org), and leaving 
it to the user to create make.conf instead of the portage install will help 
the average user.

I am an experienced Linux user but I always look at things from the point of 
a new user because I once was one and know that if things were done a 
little differently it would of kept me from screwing up.  For instance, a 
new user might start using gentoo, make changes to make.conf and decide to 
update his/her system.  Well, I could see the system updating portage and 
then telling the user to run etc-update.  A new user running etc-update 
might not completely understand what is going on and fumble their way 
through it only to wipe out the changes to make.conf.  And, what about the 
sleep deprived?  It's possible for them to do the same.

> Come on guys, think what is best for the *distro* (meaning,
> what will work best for the other 90% of users, and not
> necessary for you ...).

I think this is the best for the distro.  For me, updates to make.conf are 
an annoyance especially because the only updates are to documentation 
duplicated in the file.  However, for the masses flocking to gentoo, it 
would be a preventative measure against screwing up their customizations.


--
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Some suggestions

[Steven Elling]

On Sunday 07 September 2003 03:19, Troy Dack wrote:
> On Sun, 2003-09-07 at 15:59, Jan Krueger wrote:
> > On Sunday 07 September 2003 03:08, Martin Schlemmer wrote:
> > > Come on guys, think what is best for the *distro* (meaning,
> > > what will work best for the other 90% of users,
>
> I'd say 95% of user, but that's just me :P
>
> > From my point of view the best for the 90% of users in this case
> > (make.conf) would be:
> > 1. a very precise documentation with examples about the user settable
> > things for make.conf thats accessable via a standard command, like man
> > make.conf or info make.conf
>
> How imprecise and unaccessible is:
>
> 	nano -w /etc/make.conf
>
> All settings are commented, and commented well, and it's available using
> standard commands (you could even use ed or a combination of cat, less,
> head & tail if you really wanted to)

And duplicated in the make.conf man page.  Why duplicate documentation in 
two different places and open the door to discrepancies between them?  
Also, shouldn't we teach by example the proper places to get documentation 
instead of creating a reliance on 'self documented' config files?


--
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Some suggestions

[Steven Elling]

On Sunday 07 September 2003 09:56, Jan Krueger wrote:
> On Sunday 07 September 2003 10:48, Martin Schlemmer wrote:
> > On Sun, 2003-09-07 at 10:19, Troy Dack wrote:
> > > "Gentoo moves pretty fast; if you don't stop and look around once and
> > > awhile, you could miss out."
> >
> > Amen
>
> Thats not just Amen, that actually _is_ a big Problem if you are
> administrating some servers that are supposed to provide high quality
> services.
>
> One of the reasons for this is, as good as it is and i like it very much,
> portage. And a very timeconsuming thing is having to update things when
> there is no need to do so. I could easily bypass this problem by creating
> my own portage tree and modifying ebuilds. Some do so. That can get
> timeconsuming, especially when the official portage tree changes a lot in
> basic ebuilds, and additionally i would loose some very nice portage
> features. So far i came to the conclusion not to use gentoo on my servers
> and instead join gentoo-dev. I am happy to see that there is some progess
> with portage as can be verified on the project pages :)
>
> Having to update comments in some configuration files on each of the
> servers is silly and a waste of time that someone has to pay for. No need
> for this.

Here is another point I just remembered.  Yes, using etc-update to "update 
things when there is no need to do so" does not take that long... on a 
single machine it is just an annoyance.  But, I've been the administrator 
of over 350 UNIX servers.  Hell, I've been part of a small group 
responsible for over 3000 UNIX servers.  Image if all of them were gentoo 
and you had to run etc-update on all of them.  Would you be a happy camper?  
Maybe after you got done and set up shop in the nearest clock tower. 8-)


--
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Some suggestions

[Steven Elling]

On Sunday 07 September 2003 10:02, Jan Krueger wrote:
> On Sunday 07 September 2003 12:44, Martin Schlemmer wrote:
> > heh.  Choose '2' for etc-update 8)
>
> Normally i do so. But i dont want to press this key!
>
> Please understand: Its not about pressing a key. Its about:
>
> Requiring expensive human interaction when there is no need for it.

Do that on several boxes and see how you feel about requiring `comment' 
updates to a config file or even having to deal with files that shouldn't 
need to be updated at all or manually.  I've supported 350 and 3000 UNIX 
servers.  Imagine if they were all Gentoo.


--
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Some suggestions

[Steven Elling]

On Sunday 07 September 2003 11:43, Chris Gianelloni wrote:
> On Sat, 2003-09-06 at 23:08, Martin Schlemmer wrote:
> > On Sun, 2003-09-07 at 02:57, Chris Gianelloni wrote:
> > > On Sat, 2003-09-06 at 20:26, Steven Elling wrote:
> > > > Does portage bomb out if make.conf is not present?  If so, maybe
> > > > portage needs to be changed so that it will work without the file.
> > >
> > > I definitely like the idea of creating make.conf docs and a link from
> > > the install docs.  Also, why can't the portage ebuild contain a 0
> > > byte make.conf file?  After all, the file CAN be empty and portage
> > > will still work from the make.globals since make.conf serves no
> > > purpose but to override the system defaults.  We could include a
> > > make.conf file in the stage tarballs with a few settings (depending
> > > on the settings of the stage) and a comment telling the user where
> > > the docs for make.conf are located both locally and on Gentoo.org.
> >
> > I really do not see how having to either:
> >
> > 1) Copy and paste everything
> >
> > 2) Type if from a printout/whatever
> >
> > is efficient or helps the average user?  It is way easier
> > to just uncomment and change as needed with the help, etc
> > there in front of you.
>
> A link from the installation docs has already proved its worth.  Look at
> the USE section of the install docs.  They point to the use.xml file.
> Why would make.conf be any harder?  I also said that the defaults used
> in building the stage would be included IN THE STAGE tarball, as it is
> now.  Only the portage ebuild would contain the "blank" make.conf.

All of the defaults for the portage system are in /etc/make.globals and 
/etc/make.profile/make.defaults.  That makes make.conf a config file meant 
to be used for altering the operation of the portage system or enabling 
additional functionality.  With the documentation being put in the proper 
place and updated, make.conf should be blank, have a one liner asking the 
user to read the man page or non-existent until customization it needed.


--
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Some suggestions

[Kevyn Shortell]

[-- Attachment #1: Type: text/plain, Size: 1397 bytes --]

On Mon, 2003-09-08 at 14:39, Steven Elling wrote:
> On Sunday 07 September 2003 10:02, Jan Krueger wrote:
> > On Sunday 07 September 2003 12:44, Martin Schlemmer wrote:
> > > heh.  Choose '2' for etc-update 8)
> >
> > Normally i do so. But i dont want to press this key!
> >
> > Please understand: Its not about pressing a key. Its about:
> >
> > Requiring expensive human interaction when there is no need for it.
> 
> Do that on several boxes and see how you feel about requiring `comment' 
> updates to a config file or even having to deal with files that shouldn't 
> need to be updated at all or manually.  I've supported 350 and 3000 UNIX 
> servers.  Imagine if they were all Gentoo

I don't imagine that in a production enviroment of 3000 machines, you
would be making such drastic changes often that would require you to run
that on 3000 machines. Do you regularly upgrade the Solaris compiler?
System Patches? Still requires just as much work. heh.

If you had to patch 3000 machines, then you'd be better spending your
time like most sysadmins do, and figure out a way to automate large
scale patch distribution =)

I'm not trying to minimalize your point, but making a follow up point,
You wouldn't use Gentoo in a 3000 system node without expecting some
kind of maintainance upgrade being required. That is of course, why
sysadmins get paid. =)

trance

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-dev] Some suggestions

[Steven Elling]

On Monday 08 September 2003 15:42, Steven Elling wrote:
> I am an experienced Linux user but I always look at things from the point
> of a new user because I once was one and know that if things were done a
> little differently it would of kept me from screwing up.  For instance, a
> new user might start using gentoo, make changes to make.conf and decide
> to update his/her system.  Well, I could see the system updating portage
> and then telling the user to run etc-update.  A new user running
> etc-update might not completely understand what is going on and fumble
> their way through it only to wipe out the changes to make.conf.  And,
> what about the sleep deprived?  It's possible for them to do the same.

Boom! Case in point:

Read the mail titled __[gentoo-user] "The Gentoo Way" see'ing linux with new 
eyes!!__ from Joshua Banks <l0f33t@yahoo.com> on the gentoo-user mailing 
list dated 2003-09-07 22:09:25.

The e-mail is long but Joshua ran etc-update used -3 and auto merged 
/etc/make.conf without realizing what he was doing.  More than likely all 
of his setting are gone.  If one user has done it, you know plenty of 
others have as well.


--
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] gentoo-project

[Marius Mauch]

[-- Attachment #1: Type: text/plain, Size: 1694 bytes --]

On 09/08/03  Jan Krueger wrote:

> On Monday 08 September 2003 03:03, Jon Portnoy wrote:
> 
> > You haven't shared many of your views
> Here they are again summed up for the interested reader:
> 
> -allow the actual package install process only to add files to the
> filesystem or to only modify/remove files that belong to an revision
> of the same ebuild(qa would benefit from this suggestions too.).
> Portage, before putting all the files from the image into the life
> filesystem, scans the image for all files in there. now it has a list
> of files it is going to install. So it scans now the live filesystem
> if thes files, or some of them exist. If they exist in the live
> filesystem, portage checks if they belong to a revision of the same
> ebuild.
>  -if they dont belong to a revision of the same ebuild and are not
>  found in 
> the live filesystem it would be an addition of new software so the
> files are put into action
>  -if they are found in the filesystem and belong to a revision of the
>  same 
> ebuild it would be an upgrade or downgrade and are put into action
>  -if they are found in the filesystem and do not belong to a revision
>  of the 
> same ebuild -> something is wrong (might be init going to be
> overwritten) -> inform the user and fail

I think this might be useful so I implemented a portage patch for it,
see bug 28228 on http://bugs.gentoo.org . It's highly experimental by
now and probably needs more work, but it should work in 90% of all
cases.

Marius

-- 
Public Key at http://www.genone.de/info/gpg-key.pub

In the beginning, there was nothing. And God said, 'Let there be
Light.' And there was still nothing, but you could see a bit better.

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-dev] gentoo-project

[Alexander Gretencord]

On Monday 08 September 2003 07:33, Jan Krueger wrote:
> -prevent ebuilds from modifying the life filesystem (from pkg_postinst for
> example), portage is the only one allowed to do so. that means a real
> sandbox over full ebuild time. the image is ready after src_install.
> portage than puts the files at the right place. The ebuild itself can in no
> way touch the live filesystem. there is no need for the ebuild to do so.
> (putting the build system into UML would be a considerable option for this.
> maybe oversized)

Maybe? That's definately oversized. Making pkg_postinst sandboxed too would be 
cool, prevents some lame things from happening because someone was asleep 
when commiting an ebuild but thats it. It doesn't help against an attacker. 

If an rsync mirror (or the master tree) got compromised, you could just as 
easily modify the portage ebuild. Even with your further security 
enhancements, where the portage ebuild could only overwrite files belonging 
to a previous version of the package you're screwed. The next time you run 
portage (which is right after updating portage, as the emerge is 
automatically restarted after a portage upgrade, if other packages are to be 
updated) anything could be done, as at least the install part has to be run 
as root. Boom emerge has root and can do whatever it wants, even with all 
that pkg_postinst sandboxing.


Alex

P.S.: I dunno if you're still subscribed to gentoo-dev so I write to you and 
cc the list.


--
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] gentoo-project

[Stuart Herbert]

[-- Attachment #1: signed data --]
[-- Type: text/plain, Size: 1154 bytes --]

On Tuesday 09 September 2003 10:42 am, Alexander Gretencord wrote:
> Making pkg_postinst sandboxed too would
> be cool, prevents some lame things from happening because someone was
> asleep when commiting an ebuild but thats it. It doesn't help against an
> attacker.

That would not be cool at all.  pkg_postinst is *the* one place in the ebuild 
where we can do things that need to be done on the live filesystem or the 
machine at large.  Sandboxing this would not be helpful.

By the time the ebuild is being executed on your machine, it's already too 
late.  If security is what you want, you need something that'll stop the code 
running in the first place.

Best regards,
Stu
-- 
Stuart Herbert                                              stuart@gentoo.org
Gentoo Developer                                       http://www.gentoo.org/
Beta packages for download            http://dev.gentoo.org/~stuart/packages/
Come and meet me in March 2004                 http://www.phparch.com/cruise/

GnuGP key id# F9AFC57C available from http://pgp.mit.edu
Key fingerprint = 31FB 50D4 1F88 E227 F319  C549 0C2F 80BA F9AF C57C
--

[-- Attachment #2: signature --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-dev] gentoo-project

[Alexander Gretencord]

On Tuesday 09 September 2003 12:19, Stuart Herbert wrote:
> That would not be cool at all.  pkg_postinst is *the* one place in the
> ebuild where we can do things that need to be done on the live filesystem
> or the machine at large.  Sandboxing this would not be helpful.

Well, what needs to be done on the real filesystem anyway? What can't be done 
inside a sandbox and then copied over to the live filesystem after the admin 
says it's ok? Just theoretically, make no assumptions about how difficult it 
might be to not do it to the live filesystem. It might in fact be very 
difficult.

Of course I'm not speaking of convenience, thats always the problem with added 
security, it's often more difficult to handle (and that;s bad, easy security 
is the best I think, as people tend to find ways around the difficult to 
handle things to make their lives easier). I'm all for security like Jan, 
though his approach still has the major problem of a compromised master tree. 
The one problem that can only be circumvented by signing everything and 
keeping the signing keys away from the master tree. At least I can't think of 
anything else. That's what I wanted to point out.

> By the time the ebuild is being executed on your machine, it's already too
> late.  If security is what you want, you need something that'll stop the
> code running in the first place.

Exactly. As said, even that sandboxing wouldn't help against the scenario Jan 
was referring to all the time but just make the writing of ebuilds (and 
portage) more complicated.


Alex


--
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Some suggestions

[Chris Gianelloni]

[-- Attachment #1: Type: text/plain, Size: 1934 bytes --]

On Mon, 2003-09-08 at 11:57, Nathaniel wrote:
> 1. etc-update should check modify dates and overwrite if not modified.

I think everyone can agree on this one.

> 2. Part of the problem is that make.conf has so many options (ie. its a
> large file.)  What if we split it up into smaller files... Something
> like:
> 
> /etc/make.conf.d/USE
> /etc/make.conf.d/CFLAGS
> ...
> /etc/make.conf.d/FEATURES
> etc...
> 
> I would guess that a large portion of users never modify the features
> and other settings of make.conf.  This way, etc-update could update only
> the portions that need updating without overwriting USE flags, etc. 
> This would also make it easier to parse the files for any automated
> install (GLIS), etc.

I definitely like this idea.  It would then be possible to keep all the
comments in the files (and man pages), while still maintaining user's
settings.  Plus, sections which have not changed from default (such as
FEATURES in your example) would be overwritten automatically by the
"new" etc-update.

> 3. Another option is to have a file that contains the users settings,
> seperate from config files themselves... For instance, what if we had a
> file, say /etc/customsettings, that contained all the updated options
> for config files.  It could perhaps contain a syntax like the following:
> 
> /etc/make.conf:USE="X gnome gtk alsa"
> /etc/make.conf:FEATURES="distcc sandbox buildpkg"
> /etc/conf.d/net:IFACE="dhcp"
> 
> Then etc-update could not only replace the old file with the update, but
> could also update it with the user's specific values.  It could find the
> line with the "USE=" text and replace it with the full customized
> replacement.

This seems a bit too complex IMHO.  I REALLY like your second idea,
though, and would love to see it implemented.

-- 
Chris Gianelloni
Developer, Gentoo Linux
Games Team

Is your power animal a penguin?

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-dev] Some suggestions

[Chris Gianelloni]

[-- Attachment #1: Type: text/plain, Size: 973 bytes --]

On Mon, 2003-09-08 at 12:06, Ferris McCormick wrote:
> On Mon, 8 Sep 2003, Nathaniel wrote:
> 
> > Regarding the etc-update issue, here are some of my thoughts.
> > 
> > 1. etc-update should check modify dates and overwrite if not modified.
> > 
> I haven't followed this very closely because there's so much of it, but
> this one suddenly woke me up.  Just because a file hasn't been modified
> doesn't imply that it's OK to change it, does it?  Maybe it's fine the
> way it is, but won't be if it gets changed?  Appologies if I'm rehashing
> something.

All of the settings from a Gentoo package are supposed to work "out of
the box".  If the file has not been modified, then it is using the
default setup from Gentoo.  It should be able to continue using the
default setup and work properly.  If it does not, then it is a bug and
should be treated as such.
-- 
Chris Gianelloni
Developer, Gentoo Linux
Games Team

Is your power animal a penguin?

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-dev] Some suggestions

[Chris Gianelloni]

[-- Attachment #1: Type: text/plain, Size: 859 bytes --]

On Mon, 2003-09-08 at 20:10, Steven Elling wrote:
> Boom! Case in point:
> 
> Read the mail titled __[gentoo-user] "The Gentoo Way" see'ing linux with new 
> eyes!!__ from Joshua Banks <l0f33t@yahoo.com> on the gentoo-user mailing 
> list dated 2003-09-07 22:09:25.
> 
> The e-mail is long but Joshua ran etc-update used -3 and auto merged 
> /etc/make.conf without realizing what he was doing.  More than likely all 
> of his setting are gone.  If one user has done it, you know plenty of 
> others have as well.

I did this myself last week by accident.  Actually, I ran -5, when I
wanted to do a -1.  I had a long day that day.  Luckily for me, I had a
backup make.conf lying around that I restored, otherwise I would have
been pretty peeved.

-- 
Chris Gianelloni
Developer, Gentoo Linux
Games Team

Is your power animal a penguin?

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-dev] Some suggestions

[William Kenworthy]

Please dont suggest splitting make.conf - its a crap idea and we'll end
up with a mess like like conf.d (where it is justified, but its still a
mess - and an ongoing pain).  You get files changing, adds and deletions
that happen and you are not aware of the changes.

 For instance, how many people missed the addition of hdparm to conf.d? 
There have been a number of disastrous events where wholesale changes
have occurred when they were not intended: modules.autoconf was
particularly bad (I ended up with 3 dead systems), as was the symlink to
the php config file from mod_php and php proper.  At least if you have
one file, you only have one place to look.  Plus it makes sense to keep
all related information on one file, not piecemeal.

What this will mean, is that after every emerge problem, you will have
to find and check dozens of files, not one core file.

And think of the newbie, gentoo is becoming very complex to understand.

Another point is I run 3 make.conf's on a laptop, and load the one
applicable to the site I am at automatically (actually sed the file).  I
would have to parse a number of files instead of just one.

I think perhaps make.conf would be better named emerge.environment or
gentoo.environment to underscore its function and importance!

BillK

On Tue, 2003-09-09 at 23:11, Chris Gianelloni wrote:
> On Mon, 2003-09-08 at 11:57, Nathaniel wrote:
> > 1. etc-update should check modify dates and overwrite if not modified.
> 
> I think everyone can agree on this one.
> 
> > 2. Part of the problem is that make.conf has so many options (ie. its a
> > large file.)  What if we split it up into smaller files... Something
> > like:
> > 
> >


--
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Some suggestions

[Chris Gianelloni]

[-- Attachment #1: Type: text/plain, Size: 3159 bytes --]

On Tue, 2003-09-09 at 18:57, William Kenworthy wrote:
> Please dont suggest splitting make.conf - its a crap idea and we'll end
> up with a mess like like conf.d (where it is justified, but its still a
> mess - and an ongoing pain).  You get files changing, adds and deletions
> that happen and you are not aware of the changes.

I was thinking more like devfs.d and devfsd.conf than the way conf.d is
used.

>  For instance, how many people missed the addition of hdparm to conf.d? 
> There have been a number of disastrous events where wholesale changes
> have occurred when they were not intended: modules.autoconf was
> particularly bad (I ended up with 3 dead systems), as was the symlink to
> the php config file from mod_php and php proper.  At least if you have
> one file, you only have one place to look.  Plus it makes sense to keep
> all related information on one file, not piecemeal.
> 
> What this will mean, is that after every emerge problem, you will have
> to find and check dozens of files, not one core file.

Actually, it would be a single make.conf which is generated from files
in make.conf.d.  I think it would be pretty easy if it uses the same
style as devfs.d and devfsd.conf.  This would also allow us to maintain
backwards compatibility with older versions of portage.  You can look in
make.conf (devfsd.conf) to find the problem, and it lists the settings
as they are in the files, so you can see that the error is in a
particular file and fix it quickly.

> And think of the newbie, gentoo is becoming very complex to understand.

The main reason is lack of consistency more than complexity.  As long as
everything uses the same interface, it should not be hard to
understand.  You only have to learn one concept and apply it multiple
times.

> Another point is I run 3 make.conf's on a laptop, and load the one
> applicable to the site I am at automatically (actually sed the file).  I
> would have to parse a number of files instead of just one.

Yes and no, at least with the way I'm proposing.  The make.conf file
would be generated from the files in make.conf.d at some time.  It would
probably use some function, such as maybe portage-update, to generate
the make.conf file.  You could easily configure it via a /etc
configuration file, similar to etc-update.  I would think it would be
something we would setup to run at initial boot.

Honestly, I would like to see a simple curses-based portage-config
program which allows you to change the settings used in make.conf.  This
would solve the problems of documentation completely, as we could move
any hand-holding to the application and take them out of the file
itself.

> I think perhaps make.conf would be better named emerge.environment or
> gentoo.environment to underscore its function and importance!

Except make.conf really *isn't* that important.  It only needs to
exist.  The purpose of make.conf is to override the defaults.  The
make.globals file would be a better candidate for getting a name change.

-- 
Chris Gianelloni
Developer, Gentoo Linux
Games Team

Is your power animal a penguin?

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-dev] Some suggestions

[Nathaniel]

I think that at the least, etc-update should be able to automatically
handle changes in comments.  Like for instance, if the line in question
begins with '#' it should automerge (obviously there may be some
exceptions to this, but I think the concept has merit).

etc-update should also ignore changes to lines that begin with "USE=",
"CFLAGS=", "CXXFLAGS=", "GENTOO_MIRRORS=", etc.  These are lines that
almost everyone changes and there is no reason to "update" them unless
the format of make.conf changes, and then we can make an exception.
-- 
Nathaniel McCallum
npmccallum@users.sourceforge.net
http://glis.sourceforge.net


--
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Some suggestions

[Philippe Coulonges]

Hello,
it's my first post too. Thanks for the bird.

As we are in suggestion for make.conf, I would like to say that I took
the habit to add a reminder with the default USE variable.

I think it could help to have it available for everyone.

# Build-time functionality
# ========================
#
# The USE variable is used to enable optional build-time functionality.
...
# useflags for you. 'emerge app-admin/ufed'
#
# Default USE variable
# USE="xv slang readline gpm berkdb mmx 3dnow gdbm tcpd pam libwww ssl
nls
#     arts perl python esd gif imlib sdl oggvorbis gnome gtk X qt
#     kde motif opengl avi png tiff gif"
#
# Example:

CU
CPHIL


--
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Some suggestions

[Steven Elling]

On Wednesday 10 September 2003 08:46, Chris Gianelloni wrote:
<snip>
> > I think perhaps make.conf would be better named emerge.environment or
> > gentoo.environment to underscore its function and importance!
>
> Except make.conf really *isn't* that important.  It only needs to
> exist.  The purpose of make.conf is to override the defaults.  The
> make.globals file would be a better candidate for getting a name change.

I view make.conf as performing the same functionality as config.site only in 
a different capacity.  config.site is used by configure scripts generated 
by autoconf to provide site wide "default values for some configuration 
values" (info autoconf --> node "Setting Site Defaults") which are 
overridden by the environment.


--
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Some suggestions

[Troy Dack]

[-- Attachment #1: Type: text/plain, Size: 982 bytes --]

On Wed, 2003-09-10 at 23:46, Chris Gianelloni wrote:
> Actually, it would be a single make.conf which is generated from files
> in make.conf.d.  I think it would be pretty easy if it uses the same
> style as devfs.d and devfsd.conf.  This would also allow us to maintain
> backwards compatibility with older versions of portage.  You can look in
> make.conf (devfsd.conf) to find the problem, and it lists the settings
> as they are in the files, so you can see that the error is in a
> particular file and fix it quickly.
> 

Chris, I think I (or someone else) may have suggested this a while ago. 
IIRC the idea was not well received.

I'll search my archives and see what I can find

-- 
Troy Dack		Gentoo moves pretty fast; if you don't stop and
tad@gentoo.org		look around once and awhile, you could miss out.

Public Key: http://pgp.mit.edu:11371/pks/lookup?op=get&search=0x4D90BE3C
Key fingerprint = 1F3D 6C15 16AA 09D5 0C96  92E5 FD89 16F9 4D90 BE3C
 

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-dev] Some suggestions

[Troy Dack]

[-- Attachment #1: Type: text/plain, Size: 705 bytes --]

On Wed, 2003-09-10 at 23:46, Chris Gianelloni wrote:
> Actually, it would be a single make.conf which is generated from files
> in make.conf.d.  I think it would be pretty easy if it uses the same
> style as devfs.d and devfsd.conf.

Found it (and it was Seemant who suggest it originally):

http://news.gmane.org/onethread.php?group=gmane.linux.gentoo.devel&root=%3C20030701025824.64ecc18a.seemant%40gentoo.org%3E

-- 
Troy Dack		Gentoo moves pretty fast; if you don't stop and
tad@gentoo.org		look around once and awhile, you could miss out.

Public Key: http://pgp.mit.edu:11371/pks/lookup?op=get&search=0x4D90BE3C
Key fingerprint = 1F3D 6C15 16AA 09D5 0C96  92E5 FD89 16F9 4D90 BE3C
 

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-dev] Some suggestions

[Chris Gianelloni]

[-- Attachment #1: Type: text/plain, Size: 758 bytes --]

On Thu, 2003-09-11 at 03:54, Troy Dack wrote:
> On Wed, 2003-09-10 at 23:46, Chris Gianelloni wrote:
> > Actually, it would be a single make.conf which is generated from files
> > in make.conf.d.  I think it would be pretty easy if it uses the same
> > style as devfs.d and devfsd.conf.
> 
> Found it (and it was Seemant who suggest it originally):
> 
> http://news.gmane.org/onethread.php?group=gmane.linux.gentoo.devel&root=%3C20030701025824.64ecc18a.seemant%40gentoo.org%3E

Great minds think alike.  *grin*

Funny enough, that was right about the time I started my "trial" period
for being a developer and wasn't subscribed to gentoo-dev yet.

-- 
Chris Gianelloni
Developer, Gentoo Linux
Games Team

Is your power animal a penguin?

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-dev] Re: Some suggestions

[Stewart Honsberger]

David Sankel wrote:
> I was afraid that it wouldn't be clear what I meant by this.  I am not
> suggesting that all files could be automatically overwritten, only files
> that were not modified from the default by the user.  For example the
> file:
> 
> /etc/X11/chooser.sh
> 
> was never changed by a given user on a given system.  It is the default for
> the version they have.  When a revision to the default is discovered, the
> system will flag that file as being one that the user hasn't specialized. 
> In etc-update, there could be an option to update all flagged files
> automatically.  Does that make more sense?

I submit that this is a good idea - if the MD5 hash of a particular file 
hasn't changed, allow the user the option (I must emphasize _option_) of 
having etc-update automatically overwrite the file with the changes; 
with the usual disclaimer about how this could/would/will affect the 
functionality in a possibly unpredictable mannar and may cause more harm 
than good, etc.

I personally find it aggravating when I update a long-overdue system 
(which can be as little as 2 months with Gentoo's expedient updates) and 
having to plough through ~250 files with etc-update - atleast half of 
which I've never touched / couldn't care less about. Granted, there is a 
"-3 - update all remaining files" option, but that's still tedious.

-- 
Stewart Honsberger
http://www.snerk.org/


--
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Re: Some suggestions

[Steven Elling]

On Monday 15 September 2003 14:48, Stewart Honsberger wrote:
> David Sankel wrote:
<snip>
> I personally find it aggravating when I update a long-overdue system
> (which can be as little as 2 months with Gentoo's expedient updates) and
> having to plough through ~250 files with etc-update - atleast half of
> which I've never touched / couldn't care less about. Granted, there is a
> "-3 - update all remaining files" option, but that's still tedious.

Not to mention dangerous.


--
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Some suggestions

[Paul de Vrieze]

[-- Attachment #1: signed data --]
[-- Type: text/plain, Size: 1298 bytes --]

On Sunday 07 September 2003 09:58, Rutger Lubbers wrote:
> > On Sat, 6 Sep 2003 21:50:42 +0200
> >
> > Marius Mauch <genone@genone.de> wrote:
> >> A accurate progress bar is nearly impossible as compile times differ
> >> from package to package
> >
> > Maybe maintainers could fill in the ebuilds a kind of approximative
> > compile time from their experience, which would be relative to a
> > well know reference time (a kernel compilation with default options, or
> > something like this). It doesn't need to be very accurate.
>
> One could easily obtain those compile times from the emerge log file.
> See http://ripat.xs4all.nl/gentoo/stats/ (parse_log.py) for an
> implementation.
> Perhaps this can be made part of the gentoo-stats package.
>
> Rutger

I believe the only viable way to get reasonable estimates is to count the 
amount of gcc/g++/whatever compiler invocations from the make log for most 
options, and use those. That will only give an intra-package idea though. The 
difference in compilation length for files can differ a lot between different 
packages, esp. the difference between gcc and g++ can be big if lots of 
headers get pulled in.

Paul

-- 
Paul de Vrieze
Gentoo Developer
Mail: pauldv@gentoo.org
Homepage: http://www.devrieze.net

[-- Attachment #2: signature --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-dev] Some suggestions

[Paul de Vrieze]

[-- Attachment #1: signed data --]
[-- Type: text/plain, Size: 977 bytes --]

> Here is another point I just remembered.  Yes, using etc-update to "update
> things when there is no need to do so" does not take that long... on a
> single machine it is just an annoyance.  But, I've been the administrator
> of over 350 UNIX servers.  Hell, I've been part of a small group
> responsible for over 3000 UNIX servers.  Image if all of them were gentoo
> and you had to run etc-update on all of them.  Would you be a happy camper?
> Maybe after you got done and set up shop in the nearest clock tower. 8-)
>

In those cases it would probably be advisable to use a centralised 
configuration management tool like cfengine. You also would not ssh into each 
of the 3000 servers to update them, but you would use a tool. It is true that 
such tools could be better supported by gentoo, but there are actually people 
working on such issues.

Paul

-- 
Paul de Vrieze
Gentoo Developer
Mail: pauldv@gentoo.org
Homepage: http://www.devrieze.net

[-- Attachment #2: signature --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-dev] suggestion portage ebuild system file modification rights and protection

[Paul de Vrieze]

[-- Attachment #1: signed data --]
[-- Type: text/plain, Size: 1455 bytes --]

On Monday 08 September 2003 03:40, Jan Krueger wrote:
> On Sunday 07 September 2003 20:35, Jon Portnoy wrote:
> > What, that any situation involving installing software is going to have
> > security holes? That's the nature of software installation.
>
> Installing software at the end comes down to putting files at the right
> place. (on windows you would add: modifying the registry)
>
> So thats exactly what portage should do: put files at the right place.
>
> The ebuilds may play in the sandbox whatever game they like.
> It should however in no way possible for them to wipe your box.
>
> You agree?
>
> Jan
>

Please take a look at the sys-libs/db ebuilds. They use a function (from an 
eclass) that is needed to ensure that uninstalling versions which are the 
newest installed version works. Not having that code would actually introduce 
a hard to diagnose bug if people downgrade. The code is fairly simple, but 
certainly necessary. If you disagree, please suggest a better way to do the 
same thing. Also I don't see why removing postinst introduces much added 
security. Any application can introduce a trojan in a patch (more obscure 
than an ebuild) that gets installed suid root. There is no way you are going 
to notice without stringent security measures, and packages get installed to 
be runned.

Paul

-- 
Paul de Vrieze
Gentoo Developer
Mail: pauldv@gentoo.org
Homepage: http://www.devrieze.net

[-- Attachment #2: signature --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-dev] suggestion portage ebuild system file modification rights and protection

[Paul de Vrieze]

[-- Attachment #1: signed data --]
[-- Type: text/plain, Size: 2776 bytes --]

On Monday 08 September 2003 04:08, Jan Krueger wrote:
>
> Thats true for many open-source project. Some of them just try to get
> organized more efficiently and succeed in doing so.
> So, maybe there is a more appropriate organization model for gentoo?

As one of the people responsible for the gentoo orginisation I can say that we 
are working on reorganizing gentoo. But gentoo as an organisation with over 
150 developers and even more contributors is not exactly like a speedboat 
that can make sharp turns. It is more like an ocean liner that needs a long 
time to make a turn. Changes will be comming, and we try to implement them as 
fast as possible, but we are talking about changing mindsets and operation of 
people. Such things take time, lots of time. Further most people involved 
with gentoo are doing this voluntarilly. We cannot put all our available time 
into gentoo. We have jobs and other responsibilities too.

>
> I say:
> portage must respect my system inegrity!

There is no 100% safe way for system integrity. The sandbox certainly is not a 
perfect solution. One could use staticly compiled binaries to circumvent them 
or mounts or whatever. If a package wants to do evil portage cannot stop it 
at all. Nor does it aim to, or should it aim to. For those systems that have 
that much security concerns one musth use quaranteening and things like 
selinux or the like together with something like tripwire. Those technologies 
are much better suited to the job than portage which would become 
unmaintainable if it would need to include many tricks and workarounds for 
specific hacks (as there is no universal way to block attacks)

>
> > So we should never be able to tweak config files et al in an ebuild?
>
> an ebuild may freely modify its own config files.
> modification of config files not belonging to the ebuild should be done via
> an already suggested, secure abstraction, lets say a function like:
>  changeconf phph.ini "line to add to phpini"
> portage could then intercept, respecting the suggested CONFIG_EXCLUDE or
> other user settings, or, if no user setting is the way, go to apply the
> change. This way it would be impossible for the ebuild to wipe php.ini.
> Also the user, via CONFIG_EXCLUDE, may completely switch of editing of
> php.ini by ebuilds. On the other hand, if the user doesnt care, the ebuild
> is free to add this line to php.ini.
>

What in those cases where not updating a configuration file (esp. one that is 
not supposed to be edited by humans at all) would result in a bug, or even 
worse a security issue. Yes, I believe that that could happen.

Paul

-- 
Paul de Vrieze
Gentoo Developer
Mail: pauldv@gentoo.org
Homepage: http://www.devrieze.net

[-- Attachment #2: signature --]
[-- Type: application/pgp-signature, Size: 189 bytes --]