Re: [gentoo-dev] [GLEP] Web Application Installation

[Max Kalika <max@gentoo.org>]

Good morning!

Quoting Stuart Herbert <stuart@gentoo.org>:

> Apache supports virtual hosting already.  Tools such as phpBB and
> tikiwiki and  phpMyAdmin (to give but three examples) don't need explicit
> virtual host  support, because Apache handles that.  To run a copy of
> phpBB under two  different domains hosted on the same box, you install
> two copies of phpBB -  or at least simulate that situation using symlinks
> or whatever.

See, I don't think that running two (or more) copies of an application is
supporting virtual hosts.  Upgrading all the files is a pain.  Why not have
one installation of the core files and multiple installations of the config
files?  This is, of course, very application specific -- most of them will
still need database upgrades, but that has to be done by the sysadmin.

> Scary as I find it, one reason why many of my clients run Debian rather
> than  Gentoo on their production hardware is because they do that from a
> cron  script, and trust the results.  Personally, I've always introduced
> staging  platforms in environments where I've been wearing the Release
> Manager hat.

How do they handle db updates?  How do they handle config file updates?

> Possibly.  But I really wanted to clear out some outstanding bugs from 
> Bugzilla, and adding the eclass made it easy to do.

Fair enough.

> 1) Your eclass doesn't correctly (as I understand correct usage to be!) 
> support the apache2 USE flag.  Easy enough to do - see my webapp-apache 
> eclass for an example.

Why base it on the flag?  If the webserver is installed and is supported,
configure for it.

> 2) Your eclass doesn't specify the permissions that the source files
> should be  installed under.  Again, easy enough to fix.

This is very straightforward to add.

> 3) Your eclass doesn't provide support for running multiple copies of the
> same  app on the same machine.  This is a showstopper.

This whole thing needs a separate discussion.  There's no portage support
for dealing with more than one installation of the same version of the same
package.

> 4) Your eclass requires the admin to stop and start Apache as part of the 
> install.  This is a showstopper.  Not every site will want to stop and
> start  their web server just because they've installed a new app.
> Imagine a site  hosting hundreds of domains, and having to take them
> *all* offline at the  same time just because phpBB's been upgraded (for
> example!).  Robin's idea of  creating .htaccess files under the document
> root deals with this much more  managably - although I think we're gonna
> end up using symlinks, as that'll  make it easy to support multiple web
> servers.

..htaccess files only work in already web-accessable directories (in your
case, DocumentRoot).  If we're putting applications in /usr/share/webapps,
the webserver has no idea to look there. Which is what the Alias directive
does in all the .conf files that are generated.  However, Alias is not
allowed in .htaccess AFAIK.

http://httpd.apache.org/docs-2.0/mod/mod_alias.html#alias

Besides that, the server doesn't have to be restarted -- just HUPed.

> 5) I'm against users having to run 'ebuild /var/.../something.ebuild'
> after  installation to then complete the install.  If you're emerging
> enough  packages, it's easy to miss the notification that you need to do
> this.  It's  personal preference, I guess, but I've been caught out by
> this in the past.

Ok, if we can achieve the same thing without the post-install intervention,
all for the better.

> 6) I *like* the idea that check_php() is in this
> eclass, because that check is  specific to mod_php under Apache.  I'm
> gonna steal that and add it to my  webapp-apache eclass ;-)

Steal away!  Keep in mind it is not completely accurate if the per-package
USE flags goes into portage (http://bugs.gentoo.org/show_bug.cgi?id=13616).
Things in PUSE will also have to be taken into account.  But this is a
completely different matter and will be easy to integrate.

> 7) Your variable names are not generic enough for my liking ;-)
> AWEB_CFG, for  example, might be better off being WEBAPP_CFG.

Hey!  I originally had them as WEBAPP_* but was told to change the eclass
to apache-webapp to distinguish from other servers so I wanted the
variables to reflect the name of the eclass.  I'm as flexible on this as
playdoh. :-)

> 8) Instead of trying to supply an all-encompasing
> apache-webapp_src_install(),  relying as it does on defining global
> variables, I'd have supplied a number  of individual functions to do each
> bit.  Say, a webapp-install-appconfig,  webapp-install-serverconfig each
> taking parameters (this is off the top of my  head here ;-)  This is a
> personal preference thing.

Thats just more to call from the ebuild.  My goal was to have very small
ebuilds -- just declare a few variables and inherit the eclass.

> 9) If I'm not mistaken, your eclass does nothing to ensure that the
> webapp can  find the configuration files you've moved into
> /etc/webapps/$PN/.   Personally, I'm coming to the conclusion that
> /etc/webapps/$PN/ isn't a good  idea, because again it doesn't support
> the idea of running multiple copies of  the same app on the same machine.

I'd say we should be shooting for an easy upgrade path first.  If config
files aren't stored in /etc/webapps/${PN}, then we need to have a way of
generating an env.d/${PN} which contains a CONFIG_PROTECT line, otherwise
we're forcing sysadins to reconfigure the application at every upgrade.
And as I mentioned before, there's no way for portage to handle
multiple-copy installs, so I'm not sure the best way to go about achieving
this goal.

> 10) I'm coming to the conclusion that 'emerge -u <webapp>' shouldn't
> overwrite  the older version, but should always install alongside, in a
> different slot,  so that sites can easily run different versions of apps
> as required.  Perhaps  this should be configurable somehow?  Your eclass
> doesn't make this possible.

Why should this be handled any different then the rest of apps handled by
portage?  If a sysadmin doesn't want older versions removed, just add
AUTOCLEAN=no to make.conf.  Granted, the current eclass doesn't use the
full ${PF} in the target directory, but that is easily changed.

> I'm not saying that my eclass is better.  My eclass isn't trying to
> address  this GLEP - it's just trying to make maintenance easier for now.
> The show  must go on.  Especially as this GLEP isn't even showing up on
> the GLEP list  yet.

Ok.  Hopefully we can get this thing hashed out and out the door so things
can start getting into shape soon.

>> Double Ugh!  
> 
> Lots of laughter.  I'm gonna copyright that phrase, I think ;-)

I have my moments. ;-)

> 1) Apache1/Apache2 conundrum
> 
> My eclass uses the detection technique adopted for mod_php, and no-one
> has  complained about that.  If this eclass is invalid, then so's the
> ebuild for  mod_php.  And I don't think it is.

Ok, this one is a keeper.

> 2) Support for multiple DocumentRoot configurations, and also
> 3) Binary packages installing on machines with different DocumentRoot
> values
> 
> Until the GLEP is firmed up and approved, we don't have an agreed
> solution to  implement.  

Ah!  Ok, so before we have this issues ironed out, the current status-quo
will just be maintained.  Fair enough.

> Please excuse me, but I don't want to put support for existing ebuilds on
> hold  while we debate the GLEP.  I believe that we *have* to continue
> support until  we're ready and able to switch.  Stopping maintenance
> activities is *not* an  option.

Since I'm the laziest person I know, I didn't want to do the work twice.
I'll just hold off on introducing more ebuilds which will have to be
converted later. :-)

> 4) Which user/group to use
> 
> My class uses Robin's suggestion, and assumes that Apache is running with
> the  default settings of apache.apache.

Right, but as we already agreed, not all apps need to have their files
owned by apache:apache.  This should be configurable in the eclass.
Correct?

> 5) DocumentRoot pointing to a read-only mount
> 
> As far as I'm concerned, that's like trying to do 'make bzlilo' with
> /boot not  mounted, or run an 'emerge' with /usr mounted read-only.  It's
> the sysadmin's  job to make sure that any necessary filesystems are
> mounted read/write before  an installation is attempted.  This is not a
> problem unique to web  applications.

The issue is not whether DocumentRoot is a read-only mount during the
install, but during the day-to-day operations.  Running 'emerge' with /usr
mounted read-only is not the same as having apache running a webapp that
needs to write to /usr -- one is done seldomly, the other is all the time. 

> 6) "it's weak"
> 
> That's not an argument, it's an opinion ;-)  Anyway, I've taken 5-10
> lines of  broken and incorrectly duplicated code from a number of
> ebuilds, and moved  them all into one place where they can be re-used and
> maintained for now.   Reduced defects is a strong argument, not a weak
> one.

I never made the "weak" argument -- I don't believe in it myself.  Things
need to be justified a bit better than that, so I'm completely with you on
this one.

> Then I'd argue that the GLEP is incomplete in its scope, and needs
> expanding.   Unless you think that we can create the 'web-xxx' categories
> without a GLEP?   If so, then fair enough ;-)
> 
> Actually, I'd like to see a redrafting of the GLEP, with a clearly
> defined  list of requirements for supporting webapps.  We're debating
> implementation  issues without a clear list of requirements to address.
> This is not the  right way to go about it ;-)

Ok, agreed.  Troy, is there any chance for another draft with some of these
things incorporated?  (Thanks a bezillion, btw, for putting up with me) :-)

> It *was* fix-a-bug weekend ;-)

And a successful one at that from what I saw in my emerge --sync this
morning. :-)  Thanks!

> I am going to be adding a webapp or two that use the new eclass, but the 
> situation's no different to before the eclass existed.  The code in the 
> eclass would have had to have been duplicated in the new ebuilds.

All you! :-)

> If we *don't* do this, I reckon that the bugs about webapps running under 
> Apache'll get addressed, and many of the other bugs won't - because it's
> too  much effort.
> 
> I agree that most people will just the webapps under Apache, and never
> care  about other web servers.  But either we just support webapps under
> Apache, or  we do it properly, I'd argue.  And I reckon that doing it
> properly is  actually very easy to do.

If you say it is very easy to do, I'm on board.  I can't personally speak
for other webservers, so I'll leave the decision of whether/how to support
others to those with the experience.  So in any case, we have to pull the
apache-specific things out into a separate framework.  Therefore things
like DocumentRoot can't even be considered.  So a central location for
webapps must be once again taken into account.  And from here stems the
problem of how to tell the webserver where to look for a specific
applications.  The .htaccess approach isn't viable.  For apache the .conf
system works, but yes, it requires the server to be HUPed after an upgrade
to an install of an application (not an upgrade -- once the app is
installed and running, upgrading doesn't require the server to be touched).

> See above.  My eclass isn't attempting to solve the GLEP.  It's there to
> help  with existing packages for now, until the GLEP moves forward.

ok, I didn't realize that it wasn't a permanent solution.  All is ok.

>> How is mod_php related to the way applications are installed?
> 
> Erm, how about the whole 'do I use Apache 1 or Apache 2' conumdrum?  See
> the  mod_php ebuild for details.

Whichever is installed gets configured.  Either or both get touched,
depending on what is detected.  It is up to the sysadmin to start one or
the either automatically with rc-update.  I don't see a problem if both are
configured (if detected), but only one is running.

--mk

--
gentoo-dev@gentoo.org mailing list