Re: [gentoo-dev] [GLEP] Web Application Installation

Re: [gentoo-dev] [GLEP] Web Application Installation

[Max Kalika]

Quoting Stuart Herbert <stuart@gentoo.org>:

> Here's an example.  Imagine hosting (oh, I don't know) www.iammax.com and 
> www.maxisgreat.com on the same physical box.  Document roots for each are 
> /var/www/<host>/public_html/ for arguments sake.

Love those names.  I wonder if they're taken? ;-)

> a) A bit of mod_alias magic, and you make the /phpbb/ directories aliases
> for  /usr/share/webapps/phpBB-<version>/files/.  If you do this, though,
> how do  you get each copy of phpBB to use a separate set of configuration
> files?   What happens when the app needs write-access to the directories?
> The  directories are on /usr, and we're all agreed that /usr should be
> mountable  read-only.  And does every webserver have something like
> mod_alias in the  first place?

If not mod_alias, then symlink to the DocRoot, but then we're back to not
knowing where the user configured his/her/virtualhosts's DocRoot.

> If I've understood your eclass correctly, this is what it tries to do,
> yes?

Yes.

> b) A bit of .htaccess magic, and you have the directory structure of the 
> webapp on /var, but directives telling PHP where to find the .php files
> by  setting the include_path.  The config file problem is solved, because
> you can  drop in local config files, and there are real directories that
> can be made  writable.  Only problem with this approach is that not every
> type of web  server has the equivalent of .htaccess files - and the PHP
> SAPI for each type  of web server doesn't necessarily support
> configuration directives in config  files either.  And that's before we
> think about Perl, Python and other ways  that webapps can be implemented.

Local config files?  Where would these be stored?  How are these handled
for virtual hosts?

> c) A bit of find and cp -l, and you've got the directory structure of the 
> webapp on /var, plus links back to the original on /usr.  Again, the
> config  file problem is solved, because local copies can be parachuted
> in, and again  there are real directories that can be made writable.  No
> webserver-specific  tricks are required - as far as the web server is
> concerned, each domain has  its own installation of phpBB.

Ok, I'll pool some replies below...

>> Why base it on the flag?  If the webserver is installed and is supported,
>> configure for it.
> 
> Because that's what USE flags are there for.  If the user puts '-apache2'
> in  their USE flags, it's the job of the ebuild to respect that.
> Otherwise, the  ebuild is broken - and probably in breach of policy too.

Ok, this is debatable, but I'm willing to change the eclass to take
-apache2 into effect.

>> This whole thing needs a separate discussion.  There's no portage support
>> for dealing with more than one installation of the same version of the
>> same package.
> 
> Why a separate discussion?  If this GLEP isn't here to address this 
> fundamental issue, then I'd say that the GLEP has the wrong scope.
> 
> You're right - Portage currently can't do all of this on its own.
> Perhaps it  never will be able to.  As I understand it, that's why
> Robin's volunteered to  write and maintain additional tools to bridge the
> gap.

Keep reading, going to answer this below. :-)

>> Keep in mind it is not completely accurate if the per-package
>> USE flags goes into portage
>> (http://bugs.gentoo.org/show_bug.cgi?id=13616). Things in PUSE will also
>> have to be taken into account.  But this is a completely different
>> matter and will be easy to integrate.
> 
> It's a nice start though.  Ideally, we need a programatic interface to 
> portage, an API we can use to handle these types of queries.  karltk is 
> working on one ;-)

Cool!

>> Thats just more to call from the ebuild.  My goal was to have very small
>> ebuilds -- just declare a few variables and inherit the eclass.
> 
> Hmm ... there's a tradeoff here.  Smaller ebuilds and less flexible (and 
> re-usable) eclasses.  Or larger ebuilds, but more re-usable eclasses.  I 
> guess I prefer the latter.

We can still achieve flexibility along with smaller ebuilds.  If we break
down the eclass install function into smaller functions and optionally call
them from install, we've got ourself a nice tidy system.

>> If config
>> files aren't stored in /etc/webapps/${PN}, then we need to have a way of
>> generating an env.d/${PN} which contains a CONFIG_PROTECT line, otherwise
>> we're forcing sysadins to reconfigure the application at every upgrade.
> 
> Yeah, but if each instance of the installed app has its own config files,
> then  what's the relevance of /etc/webapps at all?
> 
>> And as I mentioned before, there's no way for portage to handle
>> multiple-copy installs, so I'm not sure the best way to go about
>> achieving this goal.
> 
> This is the problem that I think we should be solving - how to support
> the  installation, configuration, and upgrading of multiple-copy installs.

There's nothing I can think of that will make this work, but see below for
some arguments...

>> Why should this be handled any different then the rest of apps handled by
>> portage?  
> 
> Because most of the rest of the apps handled by portage don't run in a
> virtual  domain environment.  Webapps really are a different beast.

They certainly are. :-)

> Here's an example.  Imagine you're running your own hosting firm, and you
> have  a non-trivial number of customers using the same webapp (say, phpBB
> for  arguments sake).  A new version of phpBB comes out.  Some customers
> will ask  for the upgrade, and some explicitly will ask you not to
> upgrade.  So, in  this situation, you need to have two copies of phpBB
> installed on the same  box at the same time.
> 
> Now let's look at what happens when you run 'emerge -u phpBB', with the 
> appropriate ACCEPT_KEYWORDS of course.  Portage goes and installs the new 
> version of phpBB over the top of the old phpBB files.  The old version of 
> phpBB gets overwritten, yes?  I don't see how AUTOCLEAN will prevent that 
> from happening.
> 
> The whole point of SLOTing apps (as I understand it) is to allow you to
> have  multiple versions installed alongside each other.  This is the
> mechanism that  Portage offers us.  

Here's the problem.  First of all, SLOTing won't help that because it
assumes that the versions of the packages *and* the files you are
installing are _different_. I.e. db4 and db3 are SLOTted: they install
libdb-4.so and libdb-3.so respectively and thus can run simultaneously.
Now, wrt to the whole virtual server thing.  Personally if I was running a
server with, as you say, a non-trivial number of virtual domains who have
different needs, I probably wouldn't use a package management system to
begin with, or truly virtualize everything and use something like
user-mode-linux in which case we're back to single-domain installs.

Going along with your example, how can you do an 'emerge -u phpBB' _for a
particular domain_?  You're telling portage to maintain a package, and
upgrade parts of it randomly.  I simply don't think portage will ever have
any way of handling multiple installs of a package in this manner.  How
about we (Gentoo) just provide single domain/host installs and maybe
SLOTing different versions so they won't be overwritten on upgrades.
Beyhond that, we have to let the sysadmins know to cp -l the installed
files into their vhosts.

AFAIK, no other distro has support for virtual hosting because of this very
issue.

> That's up to you.  If you're maintaining webapp ebuilds currently in
> portage,  though, I'd urge you not to stop maintaining them just because
> you're waiting  for a design solution via this GLEP.

Not in charge of anything else in portage.  Though I'd like to help out
with Horde down the road (seeing as I'm on the horde dev team as well).

>> Right, but as we already agreed, not all apps need to have their files
>> owned by apache:apache.  This should be configurable in the eclass.
>> Correct?
> 
> Do we need a new user/group to own most of the files?  And then we just
> make  the files that need write-access owned by the webserver?

The most robust solution would be to create a new group (web?) and make all
the webserver users be a part of that group and make these directory
group-owned by said group.  (Similar to the "mail" group for many
mail-related services)

> In his email in that thread you pointed me at, Robin was explicitly
> talking  about /var being a read-only NFS mount at install time.

Ok, install-time aside, the installed, writeable directories cannot exist
on a readonly mount for day-to-day operations.  Which takes me back to
wanting to support a single instance install of an app that an admin has to
copy in place if the default location is unacceptable.

>> So in any case, we have to pull the
>> apache-specific things out into a separate framework.  Therefore things
>> like DocumentRoot can't even be considered.  So a central location for
>> webapps must be once again taken into account.  
> 
> Yep.  How does this sound as the design of the central layout?  Let's
> agree a  design, so that it can be added to the GLEP.
> 
> * /usr/webapps/<app-name> as the main directory.  
> * /usr/webapps/<app-name>/public_html/ for files served by the web server
> * /usr/webapps/<app-name>/cgi-bin/ for CGI-BIN files
> * /etc/webapps/<app-name>/ to hold the box-default config files
> * <app-name> is ${PN} for non-slotted packages
> * <app-name> is ${P} for slotted packages

I'd say <app-name> should just be ${PF} for both and not worry about what
is SLOTed and what is not -- it works itself out.  Otherwise, this seems ok
to me and is easy to implement in the eclass.  The only thing I would
change is the names public_html and cgi-bin.  There may be no html and not
binary files in them (respectively).  How's about 'public' and 'cgi'?  (I
know, I know, nitpicking).

> a) you need a standard way of detecting which one is installed, and 

Is the approach currently in the eclass unacceptable? (Check for the
existance of the directory/configfile)

> b) you need to honour the 'apache2' USE flag

Yup, see above...no higher...no, higher than that.  Ok there, stop!

--mk

--
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] [GLEP] Web Application Installation

[Stuart Herbert]

[-- Attachment #1: signed data --]
[-- Type: text/plain, Size: 8557 bytes --]

'lo Max,

On Tuesday 05 August 2003 12:16 am, Max Kalika wrote:
> > a) A bit of mod_alias magic, and you make the /phpbb/ directories aliases
> > for  /usr/share/webapps/phpBB-<version>/files/.  If you do this, though,
> > how do  you get each copy of phpBB to use a separate set of configuration
> > files?   What happens when the app needs write-access to the directories?
> > The  directories are on /usr, and we're all agreed that /usr should be
> > mountable  read-only.  And does every webserver have something like
> > mod_alias in the  first place?
>
> If not mod_alias, then symlink to the DocRoot, but then we're back to not
> knowing where the user configured his/her/virtualhosts's DocRoot.

mod_alias provides aliases for URLs.  Why would you need to know what DocRoot 
is?

> > If I've understood your eclass correctly, this is what it tries to do,
> > yes?
>
> Yes.

Ah good.

> > b) A bit of .htaccess magic, and you have the directory structure of the
> > webapp on /var, but directives telling PHP where to find the .php files
> > by  setting the include_path.  The config file problem is solved, because
> > you can  drop in local config files, and there are real directories that
> > can be made  writable.  Only problem with this approach is that not every
> > type of web  server has the equivalent of .htaccess files - and the PHP
> > SAPI for each type  of web server doesn't necessarily support
> > configuration directives in config  files either.  And that's before we
> > think about Perl, Python and other ways  that webapps can be implemented.
>
> Local config files?  Where would these be stored?  How are these handled
> for virtual hosts?

They would be stored under the virtual DocRoot, for example (using the paths 
for my example) in /var/www/www.iammax.com/public_html/phpbb/config.php.

Putting the config file into the virtual DocRoot is where Robin's tools would 
come in (Robin, are you following this thread at all?).  We could put a 
master copy - a skeleton if you like - in /etc/webapps/<app-name>/.  It's the 
skeleton that gets updated by portage.

> > Because that's what USE flags are there for.  If the user puts '-apache2'
> > in  their USE flags, it's the job of the ebuild to respect that.
> > Otherwise, the  ebuild is broken - and probably in breach of policy too.
>
> Ok, this is debatable, but I'm willing to change the eclass to take
> -apache2 into effect.

[OT, but interesting] How is this debatable?

> We can still achieve flexibility along with smaller ebuilds.  If we break
> down the eclass install function into smaller functions and optionally call
> them from install, we've got ourself a nice tidy system.

Good idea - I'll sign up to that.

> Here's the problem.  First of all, SLOTing won't help that because it
> assumes that the versions of the packages *and* the files you are
> installing are _different_. I.e. db4 and db3 are SLOTted: they install
> libdb-4.so and libdb-3.so respectively and thus can run simultaneously.

I must be missing something.  Why won't SLOTing help?  The behaviour you've 
described is *exactly* the behaviour that I want.  Why can't phpBB-2.0.4 and 
phpBB-2.0.5 run at the same time on the same box, but under different virtual 
domains?

> Now, wrt to the whole virtual server thing.  Personally if I was running a
> server with, as you say, a non-trivial number of virtual domains who have
> different needs, I probably wouldn't use a package management system to
> begin with, or truly virtualize everything and use something like
> user-mode-linux in which case we're back to single-domain installs.

UML is one solution, but the overhead is not insignificant.

If the package management system provided a viable solution, don't you think 
that people would use it?

> Going along with your example, how can you do an 'emerge -u phpBB' _for a
> particular domain_?  

Like this.

a) emerge -u 'webapp' installs the master copy of 'webapp' in /usr/webapps.  
If the webapp is SLOT'ed, none of the domains are upgraded at this point.
b) Then, run 'webapp-config --upgrade <webapp> /path/to/domain/directories' to 
upgrade the specific domain.

Two stage process.  Finally, once all domains are upgraded, you can simply do 
an 'emerge -C webapp-old-version' to remove the old SLOT'ed version that is 
no longer required.

What's wrong with this as a solution?

> You're telling portage to maintain a package, and
> upgrade parts of it randomly.  

Nope.  I'm telling Portage to maintain the master copy of a package, and to 
upgrade that master copy only.

> I simply don't think portage will ever have
> any way of handling multiple installs of a package in this manner.  

Why does Portage have to do *everything*?  We use portage to get the app + 
dependencies onto the box, and we use additional tools to manage the virtual 
domain upgrades.

> How
> about we (Gentoo) just provide single domain/host installs and maybe
> SLOTing different versions so they won't be overwritten on upgrades.
> Beyhond that, we have to let the sysadmins know to cp -l the installed
> files into their vhosts.

Because we can do better?

> AFAIK, no other distro has support for virtual hosting because of this very
> issue.

I have no idea whether other distros support virtual hosting, and if they 
don't, why they don't.  If you have any threads I can read on this, please 
send them my way.  Otherwise, your statement is an assumption, and 
assumptions are the mother of all screw-ups.

> The most robust solution would be to create a new group (web?) and make all
> the webserver users be a part of that group and make these directory
> group-owned by said group.  (Similar to the "mail" group for many
> mail-related services)

Why is that robust?

> Which takes me back to
> wanting to support a single instance install of an app that an admin has to
> copy in place if the default location is unacceptable.

It's one solution, sure.  I think we can do better, and I'm willing to write 
the code to do it if necessary.

> > * /usr/webapps/<app-name> as the main directory.
> > * /usr/webapps/<app-name>/public_html/ for files served by the web server
> > * /usr/webapps/<app-name>/cgi-bin/ for CGI-BIN files
> > * /etc/webapps/<app-name>/ to hold the box-default config files
> > * <app-name> is ${PN} for non-slotted packages
> > * <app-name> is ${P} for slotted packages
>
> I'd say <app-name> should just be ${PF} for both and not worry about what
> is SLOTed and what is not -- it works itself out.  

(thinks about this)  I'd want to test it to be sure, but I think ${PF} would 
work out okay.  Depends whether having the -rX part of the package name is 
really important or not.

> Otherwise, this seems ok to me and is easy to implement in the eclass.

Good-o.  So the two of us (ominously quiet in here ;-) are in agreement then?  
Oh, thought it was too good to be true ;-)

>  The only thing I would
> change is the names public_html and cgi-bin.  There may be no html and not
> binary files in them (respectively).  How's about 'public' and 'cgi'?  (I
> know, I know, nitpicking).

Shrugs.  I just picked 'public_html' because it's a recognised convention 
(although not the only one I'm sure).  'public' is too generic for my liking.  
'cgi-bin' *is* a recognised convention, and one we shouldn't break.

> Is the approach currently in the eclass unacceptable? (Check for the
> existance of the directory/configfile)

I think so.  See how I determine which version of apache to use in 
webapp-apache.eclass (as stolen/morphed from Robin's mod_php ebuild!)

> > b) you need to honour the 'apache2' USE flag
>
> Yup, see above...no higher...no, higher than that.  Ok there, stop!

Lol.  For now, just make your eclass rely on mine, seeing as mine's already in 
Portage ;-) and re-use what's already coded.  No offence, but because of the 
reasons in one of my earlier emails, I don't think your eclass should go into 
Portage until the outcome of this GLEP is determined.

Take care,
Stu
-- 
Stuart Herbert                                              stuart@gentoo.org
Gentoo Developer                                       http://www.gentoo.org/
Beta packages for download            http://dev.gentoo.org/~stuart/packages/

GnuGP key id# F9AFC57C available from http://pgp.mit.edu
Key fingerprint = 31FB 50D4 1F88 E227 F319  C549 0C2F 80BA F9AF C57C
--

[-- Attachment #2: signature --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-dev] [GLEP] Web Application Installation

[Donny Davies]

[-- Attachment #1: Type: text/plain, Size: 5743 bytes --]

On Tue, Aug 05, 2003 at 01:14:26AM +0100, Stuart Herbert wrote:
[...]
>On Tuesday 05 August 2003 12:16 am, Max Kalika wrote:
[...]
>> Ok, this is debatable, but I'm willing to change the eclass to take
>> -apache2 into effect.
>
>[OT, but interesting] How is this debatable?

I think it's clear distinct support for both Apache-1.x and Apache-2.x
is needed as the software is just so pervasively deployed on so many
different sites and configurations.  I think we can support both.

>> Here's the problem.  First of all, SLOTing won't help that because it
>> assumes that the versions of the packages *and* the files you are
>> installing are _different_. I.e. db4 and db3 are SLOTted: they install
>> libdb-4.so and libdb-3.so respectively and thus can run simultaneously.
>
>I must be missing something.  Why won't SLOTing help?  The behaviour you've 
>described is *exactly* the behaviour that I want.  Why can't phpBB-2.0.4 and 
>phpBB-2.0.5 run at the same time on the same box, but under different virtual 
>domains?

SLOT was certainly forseen to be destined primarily for use in the ebuilds
of shared libraries although it's now used in many different places
including the Apache ebuilds.  SLOTing webapps sounds great to me.

>> Now, wrt to the whole virtual server thing.  Personally if I was running a
>> server with, as you say, a non-trivial number of virtual domains who have
>> different needs, I probably wouldn't use a package management system to
>> begin with, or truly virtualize everything and use something like
>> user-mode-linux in which case we're back to single-domain installs.
>
>UML is one solution, but the overhead is not insignificant.
>
>If the package management system provided a viable solution, don't you think 
>that people would use it?

Sure they will.  The #1 reason it seems to me people would change --datadir
and/or DocumentRoot from their defaults isnt because you really want
them to be floating around, is it?  I'd say it's because /home/httpd
simply doesn't work for them.

This discussion has come a fair ways now, and proposes some very cool
or should I say "strong" ideas for solving a much richer variety of
problems and scenarios.  You'll recall I asked for what other highly
desirable features developers wanted to see supported were.  I'm pleased
to see the discussion of the *-config toolset approach, I foresaw
multiple instances and mass virtual hosting as eventually going to
be discussed.

>
>> Going along with your example, how can you do an 'emerge -u phpBB' _for a
>> particular domain_?  
>
>Like this.
>
>a) emerge -u 'webapp' installs the master copy of 'webapp' in /usr/webapps.  
>If the webapp is SLOT'ed, none of the domains are upgraded at this point.
>b) Then, run 'webapp-config --upgrade <webapp> /path/to/domain/directories' to 
>upgrade the specific domain.
>
>Two stage process.  Finally, once all domains are upgraded, you can simply do 
>an 'emerge -C webapp-old-version' to remove the old SLOT'ed version that is 
>no longer required.
>
>What's wrong with this as a solution?
>
>> You're telling portage to maintain a package, and
>> upgrade parts of it randomly.  
>
>Nope.  I'm telling Portage to maintain the master copy of a package, and to 
>upgrade that master copy only.

It looks to me as if he's exactly answered the question.  The *-config
toolset is if you ask me, a perfectly natural progression here.  Pretty
slick for webapps, which if this work continues to persist in the
direction of solving easily for everybody, would be maybe the
best out-of-the-box setup I've seen anyway.

[...]

>> The most robust solution would be to create a new group (web?) and make all
>> the webserver users be a part of that group and make these directory
>> group-owned by said group.  (Similar to the "mail" group for many
>> mail-related services)
>
>Why is that robust?
>
>> Which takes me back to
>> wanting to support a single instance install of an app that an admin has to
>> copy in place if the default location is unacceptable.
>
>It's one solution, sure.  I think we can do better, and I'm willing to write 
>the code to do it if necessary.
>
>> > * /usr/webapps/<app-name> as the main directory.
>> > * /usr/webapps/<app-name>/public_html/ for files served by the web server
>> > * /usr/webapps/<app-name>/cgi-bin/ for CGI-BIN files
>> > * /etc/webapps/<app-name>/ to hold the box-default config files
>> > * <app-name> is ${PN} for non-slotted packages
>> > * <app-name> is ${P} for slotted packages
>>
>> I'd say <app-name> should just be ${PF} for both and not worry about what
>> is SLOTed and what is not -- it works itself out.  
>
>(thinks about this)  I'd want to test it to be sure, but I think ${PF} would 
>work out okay.  Depends whether having the -rX part of the package name is 
>really important or not.
>
>> Otherwise, this seems ok to me and is easy to implement in the eclass.
>
>Good-o.  So the two of us (ominously quiet in here ;-) are in agreement then?  
>Oh, thought it was too good to be true ;-)
>
>>  The only thing I would
>> change is the names public_html and cgi-bin.  There may be no html and not
>> binary files in them (respectively).  How's about 'public' and 'cgi'?  (I
>> know, I know, nitpicking).
>
>Shrugs.  I just picked 'public_html' because it's a recognised convention 
>(although not the only one I'm sure).  'public' is too generic for my liking.  
>'cgi-bin' *is* a recognised convention, and one we shouldn't break.

Please, public_html and cgi-bin are very fine suggestions I think.

[...]

Cool developments here, I'm of course following and enjoying; Good Work.

Donny

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-dev] [GLEP] Web Application Installation

[Max Kalika]

Uh oh.  I think we are talking about the same thing.  It seems to me that
we're going towards the same ends but by a different means.  I think we're
getting really close.  I know you're dying of anticipation so ... read on!

Quoting Stuart Herbert <stuart@gentoo.org>:

>> If not mod_alias, then symlink to the DocRoot, but then we're back to not
>> knowing where the user configured his/her/virtualhosts's DocRoot.
> 
> mod_alias provides aliases for URLs.  Why would you need to know what
> DocRoot  is?

Lets skip this...see below.

>> Local config files?  Where would these be stored?  How are these handled
>> for virtual hosts?
> 
> They would be stored under the virtual DocRoot, for example (using the
> paths  for my example) in
> /var/www/www.iammax.com/public_html/phpbb/config.php.
> 
> Putting the config file into the virtual DocRoot is where Robin's tools
> would  come in (Robin, are you following this thread at all?).  We could
> put a  master copy - a skeleton if you like - in
> /etc/webapps/<app-name>/.  It's the  skeleton that gets updated by
> portage.

Ok, I'm following now.

>> Ok, this is debatable, but I'm willing to change the eclass to take
>> -apache2 into effect.
> 
> [OT, but interesting] How is this debatable?

Seems to me that if the server is installed, it should be taken into
account.  But that's entirely personal opinion.  USE flags are more correct
for the general public.

>> We can still achieve flexibility along with smaller ebuilds.  If we break
>> down the eclass install function into smaller functions and optionally
>> call them from install, we've got ourself a nice tidy system.
> 
> Good idea - I'll sign up to that.

Right-o. I'll make a note to do this in the eclass.

> I must be missing something.  Why won't SLOTing help?  The behaviour
> you've  described is *exactly* the behaviour that I want.  Why can't
> phpBB-2.0.4 and  phpBB-2.0.5 run at the same time on the same box, but
> under different virtual  domains?

I think I missed something.  I thought you wanted to use SLOTs for somehow
maintaining the same version of the same package.  I think we are on the
same page, now.  In the eclass we can go ahead and do something similar to
the kernel ebuilds -- make SLOT="${PV}" or even SLOT="${PV}-${PR}".

> If the package management system provided a viable solution, don't you
> think  that people would use it?

Certainly.

>> Going along with your example, how can you do an 'emerge -u phpBB' _for a
>> particular domain_?  
> 
> Like this.
> 
> a) emerge -u 'webapp' installs the master copy of 'webapp' in
> /usr/webapps.   If the webapp is SLOT'ed, none of the domains are
> upgraded at this point. b) Then, run 'webapp-config --upgrade <webapp>
> /path/to/domain/directories' to  upgrade the specific domain.

This is where I misunderstood.  I thought you wanted portage to take care
of everything -- i.e. emerge -u 'webapp' would somehow magically upgrade
all the copies.  This is actually a more elegant version of the cp -a thing
that I mentioned in the previous post.  I think we have our answer.  Next
step would be to write up the necessary functionality for the webapp-config
utility.

> Two stage process.  Finally, once all domains are upgraded, you can
> simply do  an 'emerge -C webapp-old-version' to remove the old SLOT'ed
> version that is  no longer required.
> 
> What's wrong with this as a solution?

Nothing.  It just took 16 posts to get it through my dense skull. :-)

> Nope.  I'm telling Portage to maintain the master copy of a package, and
> to  upgrade that master copy only.

Ding Ding Ding!

>> I simply don't think portage will ever have
>> any way of handling multiple installs of a package in this manner.  
> 
> Why does Portage have to do *everything*?  We use portage to get the app
> +  dependencies onto the box, and we use additional tools to manage the
> virtual  domain upgrades.

It doesn't and can't.  The one instance/configure is definitely the way to
go.

>> AFAIK, no other distro has support for virtual hosting because of this
>> very issue.
> 
> I have no idea whether other distros support virtual hosting, and if they 
> don't, why they don't.  If you have any threads I can read on this,
> please  send them my way.  Otherwise, your statement is an assumption,
> and  assumptions are the mother of all screw-ups.

Well then, lets the the first.  :-)

>> The most robust solution would be to create a new group (web?) and make
>> all the webserver users be a part of that group and make these directory
>> group-owned by said group.  (Similar to the "mail" group for many
>> mail-related services)
> 
> Why is that robust?

Because we have the one group that we know all webservers are going to run
as (yes yes, the sysadmin can change what group the server runs as, and if
he/she does, then he/she can also change the ownership of the directory).
If we assume that all webservers will run as the same group, then
configuring those apps which need a writeable directory becomes easy indeed.

>> Which takes me back to
>> wanting to support a single instance install of an app that an admin has
>> to copy in place if the default location is unacceptable.
> 
> It's one solution, sure.  I think we can do better, and I'm willing to
> write  the code to do it if necessary.

Yup.  We were talking about the same thing.  Just a different way of doing
the post-install config.  I'd like to help with the design/code of the
webapp-config utility.

>> I'd say <app-name> should just be ${PF} for both and not worry about what
>> is SLOTed and what is not -- it works itself out.  
> 
> (thinks about this)  I'd want to test it to be sure, but I think ${PF}
> would  work out okay.  Depends whether having the -rX part of the package
> name is  really important or not.

I'd say it is important because -rX releases may have added
functionality/features that some may not want, etc.

>> Otherwise, this seems ok to me and is easy to implement in the eclass.
> 
> Good-o.  So the two of us (ominously quiet in here ;-) are in agreement
> then?   Oh, thought it was too good to be true ;-)

It could happen!

> Shrugs.  I just picked 'public_html' because it's a recognised convention 
> (although not the only one I'm sure).  'public' is too generic for my
> liking.   'cgi-bin' *is* a recognised convention, and one we shouldn't
> break.

I thought those were apache conventions.  It really doesn't matter one way
or another. :-)

>> Is the approach currently in the eclass unacceptable? (Check for the
>> existence of the directory/configfile)
> 
> I think so.  See how I determine which version of apache to use in 
> webapp-apache.eclass (as stolen/morphed from Robin's mod_php ebuild!)

checking...still checking...done!  It could probably be simplified to
something like the following: (I'm just the nitpick mongrel, aren't I?)

if [ "`has_version '=net-www/apache-2*'`" -a "`use apache2`" ] ; then
	APACHEVER=2
elif [ "`has_version '=net-www/apache-1*'`" ] ; then
	APACHEVER=1
else
	# no apache version detected
	return 1
fi

> Lol.  For now, just make your eclass rely on mine, seeing as mine's
> already in  Portage ;-) and re-use what's already coded.  No offence, but
> because of the  reasons in one of my earlier emails, I don't think your
> eclass should go into  Portage until the outcome of this GLEP is
> determined.

No offense taken of course.  Lets just evolve yours overtime to do
everything that's needed because, as you say, it is already in portage. :-)

--mk

--
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] [GLEP] Web Application Installation

[Paul de Vrieze]

[-- Attachment #1: signed data --]
[-- Type: text/plain, Size: 1587 bytes --]


Ok what about the following.

The problem that is most existent in this case is the fact that you want the 
eclass to support almost every kind of use. That is nice indeed. But making 
things easy for single hostname smalltime servers, and for extensive virtual 
hosting supports with customized webservers etc. is complicated. Also in that 
case default apache configuration snippets don't make that much sense. I 
think a solution could be to introduce a "local" useflag that selects 
behaviour.

If you have a simple webserver for personal use, you can just leave things as 
they are and you can just run the config script or take some simple actions 
and everything works. For the other case the configuration tool can be used 
to create instances of a webapp for different virtual servers. That tool 
would probably need some configuration file etc. and it might even perform 
changes to the CONTENTS file to make the instances being included in the 
ebuild (when removing). It could then also automatically change the slot of 
the installed package so that it does not automatically get removed.

I think that should be able to keep both sides happy.

Paul

ps. one thing I want to add though is that for a simple installation I think 
the configuration files should be config-protected (how that is aranged I 
don't care), and for complex setups the instance configurations should also 
be protected in some way (but that might be handled by the tool)

-- 
Paul de Vrieze
Researcher
Mail: pauldv@cs.kun.nl
Homepage: http://www.cs.kun.nl/~pauldv

[-- Attachment #2: signature --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-dev] [GLEP] Web Application Installation

[Stuart Herbert]

[-- Attachment #1: signed data --]
[-- Type: text/plain, Size: 1026 bytes --]

Hi Donny, everyone,

Let me float a follow-on suggestion, after thinking about some of Donny's 
comments.

How do you feel about making /var/www the "official" Gentoo home for dynamic 
vhosts?  So we'd have something like

	/var/www/<host>/public_html/

as the docroot for the virtual host.

At the same time, how would you feel if, for single-domain installs, we made 
/var/www/localhost/public_html the default docroot, instead of /home/httpd. 

If the box ever needs to move to dynamic vhosts, the admin or config-toolset 
just has to drop in a symlink in /var/wwww to expose the site as a virtual 
domain.

Best regards,
Stu
-- 
Stuart Herbert                                              stuart@gentoo.org
Gentoo Developer                                       http://www.gentoo.org/
Beta packages for download            http://dev.gentoo.org/~stuart/packages/

GnuGP key id# F9AFC57C available from http://pgp.mit.edu
Key fingerprint = 31FB 50D4 1F88 E227 F319  C549 0C2F 80BA F9AF C57C
--

[-- Attachment #2: signature --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-dev] [GLEP] Web Application Installation

[Stuart Herbert]

[-- Attachment #1: signed data --]
[-- Type: text/plain, Size: 3692 bytes --]

Hi Max,

Looks like we're almost there, doesn't it? ;-)

On Tuesday 05 August 2003 4:04 am, Max Kalika wrote:
> >> The most robust solution would be to create a new group (web?) and make
> >> all the webserver users be a part of that group and make these directory
> >> group-owned by said group.  (Similar to the "mail" group for many
> >> mail-related services)
> >
> > Why is that robust?
>
> Because we have the one group that we know all webservers are going to run
> as (yes yes, the sysadmin can change what group the server runs as, and if
> he/she does, then he/she can also change the ownership of the directory).
> If we assume that all webservers will run as the same group, then
> configuring those apps which need a writeable directory becomes easy
> indeed.

We'd need to get Donnie on board to make this happen.

I'd just assumed that the webapp-config toolset would automagically take care 
of this for us (see my earlier post about what assumptions are ;-)

> > (thinks about this)  I'd want to test it to be sure, but I think ${PF}
> > would  work out okay.  Depends whether having the -rX part of the package
> > name is  really important or not.
>
> I'd say it is important because -rX releases may have added
> functionality/features that some may not want, etc.

Then let's ask Tad to put ${PF} into the GLEP.

> >> Otherwise, this seems ok to me and is easy to implement in the eclass.
> >
> > Good-o.  So the two of us (ominously quiet in here ;-) are in agreement
> > then?   Oh, thought it was too good to be true ;-)
>
> It could happen!

Lots of laughter.

> > Shrugs.  I just picked 'public_html' because it's a recognised convention
> > (although not the only one I'm sure).  'public' is too generic for my
> > liking.   'cgi-bin' *is* a recognised convention, and one we shouldn't
> > break.
>
> I thought those were apache conventions.  It really doesn't matter one way
> or another. :-)

Let's stick with 'public_html' and 'cgi-bin' then.

> checking...still checking...done!  It could probably be simplified to
> something like the following: (I'm just the nitpick mongrel, aren't I?)
>
> if [ "`has_version '=net-www/apache-2*'`" -a "`use apache2`" ] ; then
> 	APACHEVER=2
> elif [ "`has_version '=net-www/apache-1*'`" ] ; then
> 	APACHEVER=1
> else
> 	# no apache version detected
> 	return 1
> fi

According to the description in profiles/use.desc ...

	"apache2 - Chooses Apache2 support when a package supports both Apache1
	 and Apache2"

With your version, if the user has Apache2 support, but doesn't have 'apache2' 
in the USE flags, no apache version will be detected.  That doesn't seem to 
match the description in profiles/use.desc.

> No offense taken of course.  Lets just evolve yours overtime to do
> everything that's needed because, as you say, it is already in portage. :-)

Tbh, I wouldn't be surprised (or upset - I'm not the possessive type) if the 
eclass I've added to portage has to go when we implement this GLEP.

Last night on IRC, I offered to TaD to code up a new eclass to provide a 
demonstration implementation of that part of the GLEP.  Very earliest I could 
do this would be sometime next week, as I'm away for a long weekend from 
Thursday.

Take care,
Stu
-- 
Stuart Herbert                                              stuart@gentoo.org
Gentoo Developer                                       http://www.gentoo.org/
Beta packages for download            http://dev.gentoo.org/~stuart/packages/

GnuGP key id# F9AFC57C available from http://pgp.mit.edu
Key fingerprint = 31FB 50D4 1F88 E227 F319  C549 0C2F 80BA F9AF C57C
--

[-- Attachment #2: signature --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-dev] [GLEP] Web Application Installation

[Stuart Herbert]

[-- Attachment #1: signed data --]
[-- Type: text/plain, Size: 4706 bytes --]

Hi Paul,

Thanks for joining in.  I was getting worried that this was turning into the 
'Max & Stuart Show' ;-)

Here's my rambling thoughts about how the 'local' USE flag (can we call it 
'-vhosts' instead? 'local' feels a bit non-descript to me) could work, and 
what problems might occur.  I'm sure that my thoughts here aren't the only 
possible solution.

Under the '+vhosts' approach, webapps would be SLOT'd, and would go into

	/usr/share/webapps/${PF}/

and users would have to use the -config toolset to then make the app available 
inside one or more vhosts.

What would happen in the '-vhosts' world?

I think it all pivots around one crucial design decision.  In the '-vhosts' 
world, do we want 'emerge -u' to be all that the user has to do (i.e. no need 
for the -config toolset)?  If not, then I can't see what '-vhosts' would 
actually do, and I'd suggest dropping it.

Let's explore what changes the 'emerge -u' objective might have to bring into 
the design.

I think that SLOT'ing webapps would have to be switched off when '-vhosts' is 
in effect.  Can we dynamically change the SLOT value inside an ebuild without 
upsetting Portage?  Let's assume we can for the moment.

I think the webapp would have to be installed directly under the 
'DocumentRoot' (/home/httpd/htdocs currently for Apache), instead of 
/usr/share/webapps/.  Would this create support hell if a user then wanted to 
move to vhosting on the same box?  emerge -C <webapp> will clean up 
correctly, so I'm not worried about that.  I think we could live with this.

As for config-protecting a webapp's configuration files, I'd do it by making 
the eclass add the config files to the CONFIG_PROTECT mask somehow.  The 
webapp ebuild needs to be able to identify the package's config files as part 
of the '+vhosts' solution, so the information would be available.  But I 
wonder if we'd need a change to Portage to allow the eclass to dynamically 
add these files to the CONFIG_PROTECT mask.  It wouldn't need to be a 
permanent addition - in fact, I'd argue against it.

(Lots of laughter.  Just listening to the radio, and apparently the heatwave 
we're having right now is causing parts of one of our busiest motorways to 
melt).

Speaking of eclasses, one way to hide the differences of the +/-vhosts world 
from the ebuild is to handle the differences entirely in the eclasses.

	# webapp.ebuild
	inhert webapp-proxy

	# webapp-proxy.eclass
	use 'vhosts' && inherit webapp-vhosts || inherit webapp-simple

webapp-vhosts.eclass (or whatever it's called) would have to present the same 
interface (same functions w/ same parameters in) as webapp-simple.eclass.

Just some thoughts,
Stu
--
On Tuesday 05 August 2003 10:34 am, Paul de Vrieze wrote:
> Ok what about the following.
>
> The problem that is most existent in this case is the fact that you want
> the eclass to support almost every kind of use. That is nice indeed. But
> making things easy for single hostname smalltime servers, and for extensive
> virtual hosting supports with customized webservers etc. is complicated.
> Also in that case default apache configuration snippets don't make that
> much sense. I think a solution could be to introduce a "local" useflag that
> selects behaviour.
>
> If you have a simple webserver for personal use, you can just leave things
> as they are and you can just run the config script or take some simple
> actions and everything works. For the other case the configuration tool can
> be used to create instances of a webapp for different virtual servers. That
> tool would probably need some configuration file etc. and it might even
> perform changes to the CONTENTS file to make the instances being included
> in the ebuild (when removing). It could then also automatically change the
> slot of the installed package so that it does not automatically get
> removed.
>
> I think that should be able to keep both sides happy.
>
> Paul
>
> ps. one thing I want to add though is that for a simple installation I
> think the configuration files should be config-protected (how that is
> aranged I don't care), and for complex setups the instance configurations
> should also be protected in some way (but that might be handled by the
> tool)

-- 
Stuart Herbert                                              stuart@gentoo.org
Gentoo Developer                                       http://www.gentoo.org/
Beta packages for download            http://dev.gentoo.org/~stuart/packages/

GnuGP key id# F9AFC57C available from http://pgp.mit.edu
Key fingerprint = 31FB 50D4 1F88 E227 F319  C549 0C2F 80BA F9AF C57C
--

[-- Attachment #2: signature --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-dev] [GLEP] Web Application Installation

[Paul de Vrieze]

[-- Attachment #1: signed data --]
[-- Type: text/plain, Size: 730 bytes --]

On Tuesday 05 August 2003 13:19, Stuart Herbert wrote:
> Hi Paul,
>
> Thanks for joining in.  I was getting worried that this was turning into
> the 'Max & Stuart Show' ;-)
>
> Here's my rambling thoughts about how the 'local' USE flag (can we call it
> '-vhosts' instead? 'local' feels a bit non-descript to me) could work, and
> what problems might occur.  I'm sure that my thoughts here aren't the only
> possible solution.

'Local' was meant as local use flag in comparision with global use flag. Not 
the name. It probably needs to be global anyway, because you would want it 
for all packages using the eclass.

-- 
Paul de Vrieze
Researcher
Mail: pauldv@cs.kun.nl
Homepage: http://www.cs.kun.nl/~pauldv

[-- Attachment #2: signature --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-dev] [GLEP] Web Application Installation

[Max Kalika]

Quoting Stuart Herbert <stuart@gentoo.org>:

> Thanks for joining in.  I was getting worried that this was turning into
> the  'Max & Stuart Show' ;-)

Can we get a time slot on FOX?  I'm thinking 9:12AM. :-)

> Under the '+vhosts' approach, webapps would be SLOT'd, and would go into
> 
> 	/usr/share/webapps/${PF}/

Just the SLOTed? Or all the webapps?  Why not all?

> and users would have to use the -config toolset to then make the app
> available  inside one or more vhosts.
> 
> What would happen in the '-vhosts' world?
> 
> I think it all pivots around one crucial design decision.  In the
> '-vhosts'  world, do we want 'emerge -u' to be all that the user has to
> do (i.e. no need  for the -config toolset)?  If not, then I can't see
> what '-vhosts' would  actually do, and I'd suggest dropping it.

I don't think we want emerge -u to touch anything other than the central
install.

> I think the webapp would have to be installed directly under the 
> 'DocumentRoot' (/home/httpd/htdocs currently for Apache), instead of 
> /usr/share/webapps/.  Would this create support hell if a user then
> wanted to  move to vhosting on the same box?  emerge -C <webapp> will
> clean up  correctly, so I'm not worried about that.  I think we could
> live with this.
> 
> As for config-protecting a webapp's configuration files, I'd do it by
> making  the eclass add the config files to the CONFIG_PROTECT mask
> somehow.  The  webapp ebuild needs to be able to identify the package's
> config files as part  of the '+vhosts' solution, so the information would
> be available.  But I  wonder if we'd need a change to Portage to allow
> the eclass to dynamically  add these files to the CONFIG_PROTECT mask.
> It wouldn't need to be a  permanent addition - in fact, I'd argue against
> it.

Is it really necessary to separate the whole virtual/local?  Can we develop
the -config tool to take the following syntax?

  # webapp-config <application> <destination_dir>

The application can be specified by the currently supported portage syntax,
i.e. =web-dev/horde-2* or >=web-mail/imp-3 or simply web-db/phpmyadmin.
The tool then looks for a list of installed apps that match.  If more than
one is found it either bombs out or presents a choice menu.

Otherwise it goes ahead and configures the destination_dir to contain the
specified app.  First it checks for existence of the app already in the
destination.  If nothing exists, we create the directory, and copy the core
and conf files in.  If the app already exists, we switch to "upgrade" mode
where we overwrite all the core files and do dispatch-conf-type work to
protect the config files.

How do we know which files are config files and which are core files, I
hear you asking?  How about this:  Either use the Portage API or look at
the /var/db/pkg/$CATEGORY/$PF/CONTENTS file ourselves.  'obj' types stored
in '/usr/share/webapp' (or $WEBAPP_DEST or whatever) are "core" files;
'obj' types stored in '/etc/webapps' (or $WEBAPP_ETC) are "conf" files.
Everything else (docs, etc) doesn't need to go into the webserver space.

So if this is how things work, there's no need to worry about what is a
vhost and what is a single install.  Also we don't have to be concerned
where the DocRoot is -- the user just specifies the destination directory.

I guess this is a very rough first-draft proposal for the webapp-config.  I
am probably missing major chunks, but this is a braindump of what I thought
of so far.  Flog me for anything blatantly erroneous.


--mk

--
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] [GLEP] Web Application Installation

[Cal Evans]

Max & Stuart,

Personally, As someone who runs both types of servers, I would prefer a
separate behavior.

My server here at the house hosts 1 domain and I like the default way
gentoo handles that. It's easy for me to test things out etc.  But on my
production servers, I like the idea of being able to set +vhosts and have
it work as described.

Just chiming in on a very interesting thread.

My $0.02 worth, take it for what it's worth.
=C=
*
* Cal Evans
* http://www.christianperformer.com
* Stay plugged in to your audience!
*

Max Kalika said:
> Quoting Stuart Herbert <stuart@gentoo.org>:
> Is it really necessary to separate the whole virtual/local?

--
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] [GLEP] Web Application Installation

[Stuart Herbert]

[-- Attachment #1: signed data --]
[-- Type: text/plain, Size: 1984 bytes --]

Hi Cal!

On Wednesday 06 August 2003 12:43 am, Cal Evans wrote:
> Max & Stuart,
>
> Personally, As someone who runs both types of servers, I would prefer a
> separate behavior.
>
> My server here at the house hosts 1 domain and I like the default way
> gentoo handles that. It's easy for me to test things out etc.  But on my
> production servers, I like the idea of being able to set +vhosts and have
> it work as described.

Yeah.  I think we can make -vhosts be the equivalent of what we have today, 
and +vhosts do the whole virtual server thing.  I'm willing to code it up and 
trial it, to see how well it works in practice.

Which leads me onto another TonyTangent(tm).  Once we have something 
implemented, I don't think it should go into Portage straight away.  I think 
we should recruit a small team of knowledgable testers, and roll it out to 
them first for a whole cycle or two of release/test/scratch head/try again.  
It'll take a bit longer, sure, but it'll mean that what goes into Portage and 
into general use will already be bedded in.

Btw, talked to Woodchip on IRC tonight, and he seemed happy enough with us 
moving the default -vhosts DocRoot from /home/httpd.  So I'm gonna strongly 
advocate that the default -vhosts DocRoot moves to 
/var/www/localhost/public_html in the near future.

> Just chiming in on a very interesting thread.

Glad you have.  The more feedback we can get, the better the final solution 
will be.  I'd hate us to implement something that we devs found acceptable, 
but no-one else wanted to use ;-)

Best regards,
Stu
-- 
Stuart Herbert                                              stuart@gentoo.org
Gentoo Developer                                       http://www.gentoo.org/
Beta packages for download            http://dev.gentoo.org/~stuart/packages/

GnuGP key id# F9AFC57C available from http://pgp.mit.edu
Key fingerprint = 31FB 50D4 1F88 E227 F319  C549 0C2F 80BA F9AF C57C
--

[-- Attachment #2: signature --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-dev] [GLEP] Web Application Installation

[Robin H.Johnson]

[-- Attachment #1: Type: text/plain, Size: 1896 bytes --]

On Wed, Aug 06, 2003 at 02:54:19AM +0100, Stuart Herbert wrote:
Content-Description: signed data
> Yeah.  I think we can make -vhosts be the equivalent of what we have
> today, and +vhosts do the whole virtual server thing.  I'm willing to
> code it up and trial it, to see how well it works in practice.
Where do you see -vhosts fiting into having multiple instances of a
webapp on a single host ?

> Which leads me onto another TonyTangent(tm).  Once we have something
> implemented, I don't think it should go into Portage straight away.  I
> think we should recruit a small team of knowledgable testers, and roll
> it out to them first for a whole cycle or two of release/test/scratch
> head/try again.  It'll take a bit longer, sure, but it'll mean that
> what goes into Portage and into general use will already be bedded in.
On the experience I gained from working on the php eclass and ufed,
Portage is a very good distribution mechanism, even if it is only for
initial testing. Just keep it hardmasked while testing with a few big
warnings. This also enables us to get a very wide test base including
outside users if we so desire. The concept of an initial testing run is
definetly worthwhile, but also a strong warning that what you intend as
a testing round has a hard habit of sticking around a lot longer than
expected no matter what form it is in.

> Btw, talked to Woodchip on IRC tonight, and he seemed happy enough
> with us moving the default -vhosts DocRoot from /home/httpd.  So I'm
> gonna strongly advocate that the default -vhosts DocRoot moves to
> /var/www/localhost/public_html in the near future.
Sounds good.

-- 
Robin Hugh Johnson
E-Mail     : robbat2@orbis-terrarum.net
Home Page  : http://www.orbis-terrarum.net/?l=people.robbat2
ICQ#       : 30269588 or 41961639
GnuPG FP   : 11AC BA4F 4778 E3F6 E4ED  F38E B27B 944E 3488 4E85

[-- Attachment #2: Type: application/pgp-signature, Size: 232 bytes --]

Re: [gentoo-dev] [GLEP] Web Application Installation

[Stuart Herbert]

[-- Attachment #1: signed data --]
[-- Type: text/plain, Size: 1642 bytes --]

On Wednesday 06 August 2003 3:16 am, Robin H.Johnson wrote:
> Where do you see -vhosts fiting into having multiple instances of a
> webapp on a single host ?

Personally, I don't.  If you want multiple instances of a webapp on a single 
host, the webapps need to be installed with +vhosts.

-vhosts would be strictly single copy single domain scenario.

> On the experience I gained from working on the php eclass and ufed,
> Portage is a very good distribution mechanism, even if it is only for
> initial testing. Just keep it hardmasked while testing with a few big
> warnings. This also enables us to get a very wide test base including
> outside users if we so desire. The concept of an initial testing run is
> definetly worthwhile, but also a strong warning that what you intend as
> a testing round has a hard habit of sticking around a lot longer than
> expected no matter what form it is in.

Okay, I'll go with that.

> > Btw, talked to Woodchip on IRC tonight, and he seemed happy enough
> > with us moving the default -vhosts DocRoot from /home/httpd.  So I'm
> > gonna strongly advocate that the default -vhosts DocRoot moves to
> > /var/www/localhost/public_html in the near future.
>
> Sounds good.

;-)

Take care,
Stu
-- 
Stuart Herbert                                              stuart@gentoo.org
Gentoo Developer                                       http://www.gentoo.org/
Beta packages for download            http://dev.gentoo.org/~stuart/packages/

GnuGP key id# F9AFC57C available from http://pgp.mit.edu
Key fingerprint = 31FB 50D4 1F88 E227 F319  C549 0C2F 80BA F9AF C57C
--

[-- Attachment #2: signature --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-dev] [GLEP] Web Application Installation

[Donny Davies]

[-- Attachment #1: Type: text/plain, Size: 953 bytes --]

Hi Stuart

On Tue, Aug 05, 2003 at 11:12:05AM +0100, Stuart Herbert wrote:
>How do you feel about making /var/www the "official" Gentoo home for dynamic 
>vhosts?  So we'd have something like
>
>	/var/www/<host>/public_html/
>
>as the docroot for the virtual host.
>
>At the same time, how would you feel if, for single-domain installs, we made 
>/var/www/localhost/public_html the default docroot, instead of /home/httpd. 
>
>If the box ever needs to move to dynamic vhosts, the admin or config-toolset 
>just has to drop in a symlink in /var/wwww to expose the site as a virtual 
>domain.

Very reasonable.  /home/httpd was the result of moving from the very old
/usr/local/httpd, which is where things were installed before I gave the
Apache ebuild a massive overhaul.  We need not stay with /home/httpd
forever, in fact it really does need to move somewhere else.  Looks
like /var/www has become the defacto standard.

Donny

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-dev] [GLEP] Web Application Installation. Plotting a VHOST config tool.

[Robin H.Johnson]

[-- Attachment #1: Type: text/plain, Size: 5071 bytes --]

On Wed, Aug 06, 2003 at 03:44:44AM +0100, Stuart Herbert wrote:
Content-Description: signed data
> On Wednesday 06 August 2003 3:16 am, Robin H.Johnson wrote:
> > Where do you see -vhosts fiting into having multiple instances of a
> > webapp on a single host ?
> 
> Personally, I don't.  If you want multiple instances of a webapp on a single 
> host, the webapps need to be installed with +vhosts.
> 
> -vhosts would be strictly single copy single domain scenario.
Ok, so if -vhosts, we install to /usr/share/webapp/(whatever) and
automatically run the webapp-config tool ?

if +vhosts, we install to /usr/share/webapp/(whatever) and the
administrator is left to run the rest himself.

> > > Btw, talked to Woodchip on IRC tonight, and he seemed happy enough
> > > with us moving the default -vhosts DocRoot from /home/httpd.  So I'm
> > > gonna strongly advocate that the default -vhosts DocRoot moves to
> > > /var/www/localhost/public_html in the near future.
> > Sounds good.
As part of the planning, and a good first step
I'm plotting out a config tool for adding vhosts. All comments welcome
as this is intended to be a good outline of what the tool must do etc
before I write it (borrowing heavily from my exist custom scripts for
the purpose).

Stuart: once before you asked why I didn't use dynamic vhosts.
Two reasons behind it. Firstly, I want seperate log files for each
vhost, and secondly I have a lot of apache configuration directives that
vary per server (funky mod_rewrite stuff mainly to make variable
handling in php interesting). Based on this, I strongly believe that
'simple' vhosts are the best way to go in general.

Looking at the current /home/httpd:
We have:
htdocs cgi-bin error icon

All of these can go into the the /var/www/localhost/ quite easily.

/var/www/<vhost>/{error,icon} symlink back to the default host usually
(anybody wanting to use them changes them as needed).

I feel the name 'htdocs' belies the meaning of the directory more
accurately, but I'm looking for input on that item.

In my personal setup, I have:
/etc/apache2/conf/vhosts/${FQDN#*.}/${FQDN}
/var/www/${FQDN#*.}/${FQDN}/{htdocs,cgi-bin}
/var/log/apache2/${FQDN#*.}/${FQDN}/
Then I do:
"include /etc/apache2/conf/vhosts/" in my main apache config :-).

But I don't think we really need a hashed setup do we? make it a config
option because I don't know who will want it.

Rough outline of tool:

vhost-config must do three things:
- creates directories (copies a skeleton directory for the most part).
- create apache vhost config files.
- HUP apache so it reads in the new config without stopping.

Proposed rough usage:
vhost-config [opts] fully.qualified-domain.name

opts:
-s server
which server to use this with, defaults to apache, mostly a moot point
until more virtual/httpd exists.
-u owner 
user owner of all files, defaults root for security.
-g group
group owner of all files, defaults to root for security.
-p perms
initial permissions, defaults to 644 (OR'd with 111 for directories).
-G
shortcut for having group writable permissions (effective does OR 020).
-i 
symlink /var/www/localhost/icons to provide icons. (default)
-I [dir]
copies /var/www/localhost/icons to a new directory. Optionally provides
a different directory to take them from.
-e 
symlink /var/www/localhost/error to provide error pages. (default)
-E [dir]
copies /var/www/localhost/error to a new directory. Optionally provides
a different directory to take them from.
-k directory
specify a different skeleton directory (defaults to /etc/vhost-skel/)
-H
HUP apache on completion if everything is good to go.
-h
don't HUP apache at all.
-f file
path to alternative config file (default is /etc/vhost-config.conf).

/etc/vhost-skel/ might contain:
htdocs/ = maybe a simple index.html ?
cgi-bin/ = empty 
icons/ = symlink to /var/www/localhost/icons
error/ = symlink to /var/www/localhost/error
conf/ = [treated specially] ${server}.conf that is used for creating the initial config,
templates of some sort (I'm leaning to M4 as all my existing stuff is in
it, and it's good for templating like this).

After vhost-config has generated all of the nessicary stuff, it should
HUP the apache server if everything is in order.

There is one check that is outside the scope of the tool, namely the
existance of DNS for the specified hostname. If the admin creates a
vhost that does not yet have DNS or name resolution relative to the
system, we should display a warning.

I'd repeat again for good measure, this email is strongly based off the
present capabilites of my private vhost setup scripts, plus the features
I as a developer, sysadmin and user want in a new revision. However I
strongly feel that more input is needed as well.

-- 
Robin Hugh Johnson
E-Mail     : robbat2@orbis-terrarum.net
Home Page  : http://www.orbis-terrarum.net/?l=people.robbat2
ICQ#       : 30269588 or 41961639
GnuPG FP   : 11AC BA4F 4778 E3F6 E4ED  F38E B27B 944E 3488 4E85

[-- Attachment #2: Type: application/pgp-signature, Size: 232 bytes --]

Re: [gentoo-dev] [GLEP] Web Application Installation. Plotting a VHOST config tool.

[Stuart Herbert]

[-- Attachment #1: signed data --]
[-- Type: text/plain, Size: 5070 bytes --]

On Wednesday 06 August 2003 5:37 am, Robin H.Johnson wrote:
> Ok, so if -vhosts, we install to /usr/share/webapp/(whatever) and
> automatically run the webapp-config tool ?

Yeah, I think that'd be good, especially if we bring in the move to 
/var/www/localhost as the new default location for -vhosts systems.

> if +vhosts, we install to /usr/share/webapp/(whatever) and the
> administrator is left to run the rest himself.

Definitely.

> As part of the planning, and a good first step
> I'm plotting out a config tool for adding vhosts. All comments welcome
> as this is intended to be a good outline of what the tool must do etc
> before I write it (borrowing heavily from my exist custom scripts for
> the purpose).

Off the top of my head, here's a partial wish-list:

* add new vhost
* destroy existing vhost
* suspend vhost, but do not destroy
* add <webapp> to <location> inside <vhost>
* upgrade <webapp> at <location> inside <vhost>
* remove <webapp> at <location> inside <vhost>

The config tools are going to need to emulate CONFIG_PROTECT behaviour; either 
that, or we'll need some improvements to be made to Portage.

For now, I suggest we treat database creation/upgrades/etc as out-of-scope, 
unless someone wishes to step forth from the shadows with a plan for this?

> Stuart: once before you asked why I didn't use dynamic vhosts.
> Two reasons behind it. Firstly, I want seperate log files for each
> vhost, 

Excellent point.

> and secondly I have a lot of apache configuration directives that
> vary per server (funky mod_rewrite stuff mainly to make variable
> handling in php interesting). 

Grin.

> Based on this, I strongly believe that
> 'simple' vhosts are the best way to go in general.

Then let's do it.

> I feel the name 'htdocs' belies the meaning of the directory more
> accurately, but I'm looking for input on that item.

+1 for htdocs.

> Rough outline of tool:
>
> vhost-config must do three things:
> - creates directories (copies a skeleton directory for the most part).
> - create apache vhost config files.
> - HUP apache so it reads in the new config without stopping.
>
> Proposed rough usage:
> vhost-config [opts] fully.qualified-domain.name

[snip]

The options look good.  The only additional behaviour I'd suggest is to create 
symlinks from ~<user>/vhosts/<FQDM> to /var/www/<FQDM>, to make it easier for 
people with shell accounts on the box to find their own domains.

How about making /var/www/<FQDM> chmod'd 750 by default, and owned by the 
apache group by default?  On shared boxes with shell accounts, this'd mean 
that apache can serve the files, but other shell accounts can't take a peak.

> /etc/vhost-skel/ might contain:
> htdocs/ = maybe a simple index.html ?

Yes.  I'd go further and make it a symlink to a default index.html page, so 
that changes to a master copy are automatically picked up for domains that 
have been created but left unused.

> conf/ = [treated specially] ${server}.conf that is used for creating the
> initial config, templates of some sort (I'm leaning to M4 as all my
> existing stuff is in it, and it's good for templating like this).

/me hates m4 ;-)

What are you going to write the tools in? Python, bash, PHP, or something 
else?

> After vhost-config has generated all of the nessicary stuff, it should
> HUP the apache server if everything is in order.

Yup.

> There is one check that is outside the scope of the tool, namely the
> existance of DNS for the specified hostname. If the admin creates a
> vhost that does not yet have DNS or name resolution relative to the
> system, we should display a warning.

I like this.  Maybe the default behaviour could be to throw an error, and 
supply a '--force' option to downgrade to a warning?

> I'd repeat again for good measure, this email is strongly based off the
> present capabilites of my private vhost setup scripts, plus the features
> I as a developer, sysadmin and user want in a new revision. However I
> strongly feel that more input is needed as well.

Well, you've got mine now ;-)

Webapps aren't the only type of application that can take advantage of a 
vhosts solution.  Games servers such as the Half-Life Dedicated Server (HLDS) 
can also benefit from being deployed into a vhosts-type scenario.

Obviously, there are important differences, which means that the exact same 
tool can't be used to manage both.  But, if we can identify and extract out 
common elements so that they can be re-used by different scripts, that might 
help validate both the design and the implementation.

Take care,
Stu
-- 
Stuart Herbert                                              stuart@gentoo.org
Gentoo Developer                                       http://www.gentoo.org/
Beta packages for download            http://dev.gentoo.org/~stuart/packages/

GnuGP key id# F9AFC57C available from http://pgp.mit.edu
Key fingerprint = 31FB 50D4 1F88 E227 F319  C549 0C2F 80BA F9AF C57C
--

[-- Attachment #2: signature --]
[-- Type: application/pgp-signature, Size: 189 bytes --]