[gentoo-dev] system profiles, stackable profiles, virtuals, and dependencies

[gentoo-dev] system profiles, stackable profiles, virtuals, and dependencies

[Paul Varner]

(There are several things in here and what I'm looking for is discussion
and education)

I've been playing with my system configuration lately so that I can gain
a deeper understanding of the features of the portage system. 

I'm currently using portage-2.0.51_pre13, so I'm assuming this version
and higher for the purposes of thsi discussion.

One of my goals is to have a minimal world file that only contains the
applications that I want installed on a system and when combined with a
specific profile and /etc/portage/virtuals - (to set things where I
don't like the defaults i.e. postfix for virtual/mta) would allow me to
rebuild / maintain a fairly clean system.

During all of this I removed syslog-ng from my world file which led to
the following bug report <http://bugs.gentoo.org/show_bug.cgi?id=57787>
about vixie-cron not having a runtime dependency on virtual/logger. 

Anyhow, this led me to start thinking about the system profiles. 
Neither cron or a system logger are in the system profiles, yet having
many years of Unix/Linux administration experience, they are a fairly
vital part of a "normal" Unix/Linux system.

My conjecture for why they are not in the system profile is because they
are not required to operate a system and since Gentoo is about choice we
are not going to dictate a specific version of cron or a specific system
logger.

This is where stackable profiles and virtuals come in.  To me these
applications are part of a normal default Linux installation (be it a
server or workstation) and should be considered part of the system
software and I see a couple of ways to approach this.  

The first and simplest is that virtual/logger and virtual/cron be added
to the system profile. If this were to happen, the defaults (dcron &
sysklogd for default-x86 profiles) would be installed during the
installation process.

If I don't like those defaults, I could unmerge the defaults and emerge
the software I did prefer after the installation.  Kind of blech type of
solution to me. The other alternative is to setup /etc/portage/virtuals
to override the defaults prior to installation.

The second way based upon what I've read is to use a stackable profile. 
Unfortunately, I can't find any implementation documentation on them, so
I don't know if portage supports them and if so, how to use them. It
seems to me though that with a stackable profile, I could add those
pieces of software that I consider to be part of the system but are not
required to run the system to *my* system profile.  This allows Gentoo
to keep the default profiles small with only the required software, and
allows me the choice to expand upon what is provided by Gentoo.

Finally dependencies, since I have been writing and tinkering with
ebuilds at what point do you stop adding dependencies to the ebuild.  To
be anal about it would be to add everything back to glibc.  That is
obviously not the case and the assumption that I've been making is that
if it isn't in the system profile and it is required to build/run the
package then it should be listed as a dependency.  That was my reasoning
for filing the above bug. As a system logger isn't required by the
system profile and vixie-cron requires one to run, it should be listed
as a runtime dependency.  In this case, I was told politely that I was
nuts since "There's no reason for vixie-cron to rdepend on
virtual/logger".  I obviously disagree with that assessment, which is
why I'm asking the question.  (Side note: I don't have a lot of
heartburn with the bug report being rejected, although I do feel it is a
valid bug or I wouldn't have submitted it. My opinion is that if it is
not going to be addressed, the resolution should be WONTFIX instead of
INVALID)

If you've read this far, thanks for reading and I appreciate any
thoughts, ideas, etc on all of this.  

Regards,
Paul


--
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] system profiles, stackable profiles, virtuals, and dependencies

[Seemant Kulleen]

[-- Attachment #1: Type: text/plain, Size: 814 bytes --]

Hi Paul,

Others can give you a better response -- but as for the virtual/logger
and virtual/cron in system profiles, I think that would mess up the
livecd and install stages, because they would then get bloated up.

As far as cascading profiles are concerned -- we don't have any way to
add custom stuffs to the profiles (yet?).  It might be on the roadmap
for some future date, though.  The immediate goal for cascading profiles
was really to de-bloat /usr/portage/profiles.  I'd suggest filing a bug
to dev-portage@gentoo for the custom support though.
-- 
"Check out the hook while my DJ revolves it"

Seemant Kulleen
http://dev.gentoo.org/~seemant

Public Key: http://pgp.mit.edu:11371/pks/lookup?op=get&search=0x3458780E
Key fingerprint = 23A9 7CB5 9BBB 4F8D 549B 6593 EDA2 65D8 3458 780E

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-dev] system profiles, stackable profiles, virtuals, and dependencies

[Chris Gianelloni]

[-- Attachment #1: Type: text/plain, Size: 1535 bytes --]

On Wed, 2004-07-21 at 13:59, Paul Varner wrote:
> Finally dependencies, since I have been writing and tinkering with
> ebuilds at what point do you stop adding dependencies to the ebuild.  To
> be anal about it would be to add everything back to glibc.  That is
> obviously not the case and the assumption that I've been making is that
> if it isn't in the system profile and it is required to build/run the
> package then it should be listed as a dependency.  That was my reasoning
> for filing the above bug. As a system logger isn't required by the
> system profile and vixie-cron requires one to run, it should be listed
> as a runtime dependency.  In this case, I was told politely that I was
> nuts since "There's no reason for vixie-cron to rdepend on
> virtual/logger".  I obviously disagree with that assessment, which is
> why I'm asking the question.  (Side note: I don't have a lot of
> heartburn with the bug report being rejected, although I do feel it is a
> valid bug or I wouldn't have submitted it. My opinion is that if it is
> not going to be addressed, the resolution should be WONTFIX instead of
> INVALID)

vixie-cron can run fine without any logger at all.  I have actually
configured such a system, simply when playing around.  What happens is
any messages are dumped to the console, which isn't very good, but
proves that there is no hard requirement for a logger.

-- 
Chris Gianelloni
Release Engineering QA Manager/Games Developer
Gentoo Linux

Is your power animal a penguin?

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-dev] system profiles, stackable profiles, virtuals, and dependencies

[Paul Varner]

On Wed, 2004-07-21 at 16:52, Chris Gianelloni wrote:
> On Wed, 2004-07-21 at 13:59, Paul Varner wrote:
> > Finally dependencies, since I have been writing and tinkering with
> > ebuilds at what point do you stop adding dependencies to the ebuild.  To
> > be anal about it would be to add everything back to glibc.  That is
> > obviously not the case and the assumption that I've been making is that
> > if it isn't in the system profile and it is required to build/run the
> > package then it should be listed as a dependency.  That was my reasoning
> > for filing the above bug. As a system logger isn't required by the
> > system profile and vixie-cron requires one to run, it should be listed
> > as a runtime dependency.  In this case, I was told politely that I was
> > nuts since "There's no reason for vixie-cron to rdepend on
> > virtual/logger".  I obviously disagree with that assessment, which is
> > why I'm asking the question.  (Side note: I don't have a lot of
> > heartburn with the bug report being rejected, although I do feel it is a
> > valid bug or I wouldn't have submitted it. My opinion is that if it is
> > not going to be addressed, the resolution should be WONTFIX instead of
> > INVALID)
> 
> vixie-cron can run fine without any logger at all.  I have actually
> configured such a system, simply when playing around.  What happens is
> any messages are dumped to the console, which isn't very good, but
> proves that there is no hard requirement for a logger.

If that's the case then the ebuild shouldn't install a startup file into
/etc/init.d that has "need logger" in its depend section.  It should be
changed to a "use logger".

Currently as installed by the ebuild, vixie-cron requires a system
logger to be installed to operate correctly.

-- 
My Gentoo stuff: http://varnerfamily.org/pvarner/gentoo

--
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] system profiles, stackable profiles, virtuals, and dependencies

[Chris Gianelloni]

[-- Attachment #1: Type: text/plain, Size: 907 bytes --]

On Wed, 2004-07-21 at 20:36, Paul Varner wrote:
> > vixie-cron can run fine without any logger at all.  I have actually
> > configured such a system, simply when playing around.  What happens is
> > any messages are dumped to the console, which isn't very good, but
> > proves that there is no hard requirement for a logger.
> 
> If that's the case then the ebuild shouldn't install a startup file into
> /etc/init.d that has "need logger" in its depend section.  It should be
> changed to a "use logger".

I agree.  File a bug.  You might want to check dcron and the others,
too.

> Currently as installed by the ebuild, vixie-cron requires a system
> logger to be installed to operate correctly.

Then that is a bug in the ebuild and should be treated as such... =]

-- 
Chris Gianelloni
Release Engineering QA Manager/Games Developer
Gentoo Linux

Is your power animal a penguin?

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-dev] system profiles, stackable profiles, virtuals, and dependencies

[Ned Ludd]

[-- Attachment #1: Type: text/plain, Size: 1186 bytes --]

On Wed, 2004-07-21 at 16:02, Seemant Kulleen wrote:
> Hi Paul,
> 
> Others can give you a better response -- but as for the virtual/logger
> and virtual/cron in system profiles, I think that would mess up the
> livecd and install stages, because they would then get bloated up.
> 


> As far as cascading profiles are concerned -- we don't have any way to
> add custom stuffs to the profiles (yet?).  

Sure we do.. 
User symlinks his/her make.profile to the right cascading profile in
../../usr/portage/profiles/${profile_name} then fills his/her
/etc/portage/virtuals with all the fun files he/she wants to swapped out
for what the profiles/${profile_name}/arch/.. would provide.

This combined with a user supplied overlay can be quite powerful for
rapid deployment and allow for alot of custom build scenarios.

> It might be on the roadmap
> for some future date, though.  The immediate goal for cascading profiles
> was really to de-bloat /usr/portage/profiles.  I'd suggest filing a bug
> to dev-portage@gentoo for the custom support though.



-- 
Ned Ludd <solar@gentoo.org>
Gentoo (hardened,security,infrastructure,embedded,toolchain) Developer

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-dev] system profiles, stackable profiles, virtuals, and dependencies

[Marius Mauch]

[-- Attachment #1: Type: text/plain, Size: 1702 bytes --]

On 07/25/04  Ned Ludd wrote:

> On Wed, 2004-07-21 at 16:02, Seemant Kulleen wrote:
> > Hi Paul,
> > 
> > Others can give you a better response -- but as for the
> > virtual/logger and virtual/cron in system profiles, I think that
> > would mess up the livecd and install stages, because they would then
> > get bloated up.
> > 
> 
> 
> > As far as cascading profiles are concerned -- we don't have any way
> > to add custom stuffs to the profiles (yet?).  
> 
> Sure we do.. 
> User symlinks his/her make.profile to the right cascading profile in
> ../../usr/portage/profiles/${profile_name} then fills his/her
> /etc/portage/virtuals with all the fun files he/she wants to swapped
> out for what the profiles/${profile_name}/arch/.. would provide.
> 
> This combined with a user supplied overlay can be quite powerful for
> rapid deployment and allow for alot of custom build scenarios.
> 
> > It might be on the roadmap
> > for some future date, though.  The immediate goal for cascading
> > profiles was really to de-bloat /usr/portage/profiles.  I'd suggest
> > filing a bug to dev-portage@gentoo for the custom support though.

Hmm, I must have overseen that mail and ciaranm will probably kick me
for mentioning it, but you always have /etc/portage/profile as the
topmost level on the profile stack with the same rights and duties as
all other profiles, independent if you actually use    
    profiles/default-linux/x86/2004.0 
or 
    profiles/default-x86-2004.0 (adjust to your environment).

Marius

-- 
Public Key at http://www.genone.de/info/gpg-key.pub

In the beginning, there was nothing. And God said, 'Let there be
Light.' And there was still nothing, but you could see a bit better.

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-dev] system profiles, stackable profiles, virtuals, and dependencies

[Paul Varner]

On Sun, 2004-07-25 at 03:12, Marius Mauch wrote:
> On 07/25/04  Ned Ludd wrote:
> > On Wed, 2004-07-21 at 16:02, Seemant Kulleen wrote:
> > > As far as cascading profiles are concerned -- we don't have any way
> > > to add custom stuffs to the profiles (yet?).  
> > 
> > Sure we do.. 
> > User symlinks his/her make.profile to the right cascading profile in
> > ../../usr/portage/profiles/${profile_name} then fills his/her
> > /etc/portage/virtuals with all the fun files he/she wants to swapped
> > out for what the profiles/${profile_name}/arch/.. would provide.
> > 
> > This combined with a user supplied overlay can be quite powerful for
> > rapid deployment and allow for alot of custom build scenarios.
> > 
> Hmm, I must have overseen that mail and ciaranm will probably kick me
> for mentioning it, but you always have /etc/portage/profile as the
> topmost level on the profile stack with the same rights and duties as
> all other profiles, independent if you actually use    
>     profiles/default-linux/x86/2004.0 
> or 
>     profiles/default-x86-2004.0 (adjust to your environment).


This is the approach that I've been taking.  What I'm missing is how to
effectively use /etc/portage/profile to implement *my* changes to my
chosen profile.

Backing up a step. I'm using portage-2.0.51_pre13 and make.profile is
symlinked to profiles/default-x86-2004.  I have created
/etc/portage/virtuals with the packages that I want to satisfy a
virtual.

Now, this is where I hit the stumbling block.  My definition of what
should be part of system and the official Gentoo definition are
different.  In this specific case, I consider both a cron package and a
system logger to be system software (i.e. part of the system profile and
not part of the world file).  I know that cascading profiles exist from
hanging out here in gentoo-dev, but I have been unable to find any
documentation on using them from a user perspective.  So in this case,
how do I add virtual/cron and virtual/logger to *my* system profile?

As a side note, knowing how to do this also benefits "Enterprise
Gentoo".  If defined correctly, it becomes a lot easier to build/rebuild
systems containing the same packages on multiple machines.  Since I can
now define a desktop profile, a server profile, etc.  If it's
configurable enough I could even go so far as to defining the server
profile differently based upon if it is a mail server vs. a web server,
etc.

Regards,
Paul
-- 
My Gentoo stuff: http://varnerfamily.org/pvarner/gentoo

--
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] system profiles, stackable profiles, virtuals, and dependencies

[Ciaran McCreesh]

[-- Attachment #1: Type: text/plain, Size: 735 bytes --]

On Sun, 25 Jul 2004 11:01:15 -0500 Paul Varner
<gentoo-dev@varnerfamily.org> wrote:
| As a side note, knowing how to do this also benefits "Enterprise
| Gentoo".  If defined correctly, it becomes a lot easier to
| build/rebuild systems containing the same packages on multiple
| machines.  Since I can now define a desktop profile, a server profile,
| etc.  If it's configurable enough I could even go so far as to
| defining the server profile differently based upon if it is a mail
| server vs. a web server, etc.

Uhm, isn't that why we have /etc/portage/sets/ and GLEP 21?

-- 
Ciaran McCreesh : Gentoo Developer (Sparc, MIPS, Vim, Fluxbox)
Mail            : ciaranm at gentoo.org
Web             : http://dev.gentoo.org/~ciaranm


[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-dev] system profiles, stackable profiles, virtuals, and dependencies

[Paul Varner]

On Sun, 2004-07-25 at 11:03, Ciaran McCreesh wrote:
> On Sun, 25 Jul 2004 11:01:15 -0500 Paul Varner
> <gentoo-dev@varnerfamily.org> wrote:
> | As a side note, knowing how to do this also benefits "Enterprise
> | Gentoo".  If defined correctly, it becomes a lot easier to
> | build/rebuild systems containing the same packages on multiple
> | machines.  Since I can now define a desktop profile, a server profile,
> | etc.  If it's configurable enough I could even go so far as to
> | defining the server profile differently based upon if it is a mail
> | server vs. a web server, etc.
> 
> Uhm, isn't that why we have /etc/portage/sets/ and GLEP 21?

It's not completely what I'm looking for, although using sets would be
workable.

What I'm trying to accomplish is to explicitly seperate "system"
software from "user" software.  Where system software is the stuff that
I require to be installed on a system.  Typically, I have much stricter
controls on the system stuff versus the user stuff.

I'm still thinking through this stuff, but what I envision is something
similar to the following sets:

system - The set of software required for the machine to operate. This
may be defined differently depending on the function of the machine. 
For example, if virtual/mta is part of the system, I might define
mail-mta/ssmtp to be the MTA for a desktop and mail-mta/postfix to be
the MTA for a server.  The system software for a desktop would probably
include X, while X would not be part of the software for a server. An
emerge <whatever option> system would take care of all of this
software.  A web server would include apache in this list, where a file
server or mail server would not.  Depending upon policy, I would
probably also lock this profile down to specific versions, so that
upgrades to this set of software are not automatic and require me to
manually update the profile to allow the upgrades.

sets (as currently defined in GLEP 21) - The applications that I would
install on a machine for it to do its intended function.  A desktop set
would probably include Open Office, a web server might have the various
mod_* packages, and a mail server would have spamassassin and clamav. 
Depending upon policy, I may or may not care about upgrades.  It might
be locked down tight, allow automatic upgrades as long as it isn't the
next major version, etc.

world - All of the one-off stuff not installed with the system profile
or by the set.  For example: I've decided that I hate KDE/Gnome and I
want to see if Fluxbox meets my needs.  While I'm testing/playing with
it, it would be in the world file and I don't deeply care if it breaks
or is automatically upgraded.  I see the stuff in the world file as
being specific to the machine/users for that machine that wouldn't
neccessarily be applicable to another machine or user.

It is this kind of configurabilty that has drawn me to Gentoo in the
first place.  I've gotten fed up with the RPM hell from other distros
and the inability to define what/how I want a machine to have installed
in an easily repeatable and controllable fashion.  The configuration
capabilities of portage has accomplished much of my goals as a system
administrator and user.  Where I'm at now is what I consider to be the
grey areas of configuration (i.e. my definition of system probably
doesn't match your definition) thus my request for discussion and
education.

Regards,
Paul
-- 
My Gentoo stuff: http://varnerfamily.org/pvarner/gentoo

--
gentoo-dev@gentoo.org mailing list