public inbox for gentoo-dev@lists.gentoo.org
 help / color / mirror / Atom feed
* Re: [gentoo-dev] [GLEP] Web Application Installation
@ 2003-08-04 23:16 Max Kalika
  2003-08-05  0:14 ` Stuart Herbert
  0 siblings, 1 reply; 35+ messages in thread
From: Max Kalika @ 2003-08-04 23:16 UTC (permalink / raw
  To: stuart, Max Kalika, Troy Dack, gentoo-dev

Quoting Stuart Herbert <stuart@gentoo.org>:

> Here's an example.  Imagine hosting (oh, I don't know) www.iammax.com and 
> www.maxisgreat.com on the same physical box.  Document roots for each are 
> /var/www/<host>/public_html/ for arguments sake.

Love those names.  I wonder if they're taken? ;-)

> a) A bit of mod_alias magic, and you make the /phpbb/ directories aliases
> for  /usr/share/webapps/phpBB-<version>/files/.  If you do this, though,
> how do  you get each copy of phpBB to use a separate set of configuration
> files?   What happens when the app needs write-access to the directories?
> The  directories are on /usr, and we're all agreed that /usr should be
> mountable  read-only.  And does every webserver have something like
> mod_alias in the  first place?

If not mod_alias, then symlink to the DocRoot, but then we're back to not
knowing where the user configured his/her/virtualhosts's DocRoot.

> If I've understood your eclass correctly, this is what it tries to do,
> yes?

Yes.

> b) A bit of .htaccess magic, and you have the directory structure of the 
> webapp on /var, but directives telling PHP where to find the .php files
> by  setting the include_path.  The config file problem is solved, because
> you can  drop in local config files, and there are real directories that
> can be made  writable.  Only problem with this approach is that not every
> type of web  server has the equivalent of .htaccess files - and the PHP
> SAPI for each type  of web server doesn't necessarily support
> configuration directives in config  files either.  And that's before we
> think about Perl, Python and other ways  that webapps can be implemented.

Local config files?  Where would these be stored?  How are these handled
for virtual hosts?

> c) A bit of find and cp -l, and you've got the directory structure of the 
> webapp on /var, plus links back to the original on /usr.  Again, the
> config  file problem is solved, because local copies can be parachuted
> in, and again  there are real directories that can be made writable.  No
> webserver-specific  tricks are required - as far as the web server is
> concerned, each domain has  its own installation of phpBB.

Ok, I'll pool some replies below...

>> Why base it on the flag?  If the webserver is installed and is supported,
>> configure for it.
> 
> Because that's what USE flags are there for.  If the user puts '-apache2'
> in  their USE flags, it's the job of the ebuild to respect that.
> Otherwise, the  ebuild is broken - and probably in breach of policy too.

Ok, this is debatable, but I'm willing to change the eclass to take
-apache2 into effect.

>> This whole thing needs a separate discussion.  There's no portage support
>> for dealing with more than one installation of the same version of the
>> same package.
> 
> Why a separate discussion?  If this GLEP isn't here to address this 
> fundamental issue, then I'd say that the GLEP has the wrong scope.
> 
> You're right - Portage currently can't do all of this on its own.
> Perhaps it  never will be able to.  As I understand it, that's why
> Robin's volunteered to  write and maintain additional tools to bridge the
> gap.

Keep reading, going to answer this below. :-)

>> Keep in mind it is not completely accurate if the per-package
>> USE flags goes into portage
>> (http://bugs.gentoo.org/show_bug.cgi?id=13616). Things in PUSE will also
>> have to be taken into account.  But this is a completely different
>> matter and will be easy to integrate.
> 
> It's a nice start though.  Ideally, we need a programatic interface to 
> portage, an API we can use to handle these types of queries.  karltk is 
> working on one ;-)

Cool!

>> Thats just more to call from the ebuild.  My goal was to have very small
>> ebuilds -- just declare a few variables and inherit the eclass.
> 
> Hmm ... there's a tradeoff here.  Smaller ebuilds and less flexible (and 
> re-usable) eclasses.  Or larger ebuilds, but more re-usable eclasses.  I 
> guess I prefer the latter.

We can still achieve flexibility along with smaller ebuilds.  If we break
down the eclass install function into smaller functions and optionally call
them from install, we've got ourself a nice tidy system.

>> If config
>> files aren't stored in /etc/webapps/${PN}, then we need to have a way of
>> generating an env.d/${PN} which contains a CONFIG_PROTECT line, otherwise
>> we're forcing sysadins to reconfigure the application at every upgrade.
> 
> Yeah, but if each instance of the installed app has its own config files,
> then  what's the relevance of /etc/webapps at all?
> 
>> And as I mentioned before, there's no way for portage to handle
>> multiple-copy installs, so I'm not sure the best way to go about
>> achieving this goal.
> 
> This is the problem that I think we should be solving - how to support
> the  installation, configuration, and upgrading of multiple-copy installs.

There's nothing I can think of that will make this work, but see below for
some arguments...

>> Why should this be handled any different then the rest of apps handled by
>> portage?  
> 
> Because most of the rest of the apps handled by portage don't run in a
> virtual  domain environment.  Webapps really are a different beast.

They certainly are. :-)

> Here's an example.  Imagine you're running your own hosting firm, and you
> have  a non-trivial number of customers using the same webapp (say, phpBB
> for  arguments sake).  A new version of phpBB comes out.  Some customers
> will ask  for the upgrade, and some explicitly will ask you not to
> upgrade.  So, in  this situation, you need to have two copies of phpBB
> installed on the same  box at the same time.
> 
> Now let's look at what happens when you run 'emerge -u phpBB', with the 
> appropriate ACCEPT_KEYWORDS of course.  Portage goes and installs the new 
> version of phpBB over the top of the old phpBB files.  The old version of 
> phpBB gets overwritten, yes?  I don't see how AUTOCLEAN will prevent that 
> from happening.
> 
> The whole point of SLOTing apps (as I understand it) is to allow you to
> have  multiple versions installed alongside each other.  This is the
> mechanism that  Portage offers us.  

Here's the problem.  First of all, SLOTing won't help that because it
assumes that the versions of the packages *and* the files you are
installing are _different_. I.e. db4 and db3 are SLOTted: they install
libdb-4.so and libdb-3.so respectively and thus can run simultaneously.
Now, wrt to the whole virtual server thing.  Personally if I was running a
server with, as you say, a non-trivial number of virtual domains who have
different needs, I probably wouldn't use a package management system to
begin with, or truly virtualize everything and use something like
user-mode-linux in which case we're back to single-domain installs.

Going along with your example, how can you do an 'emerge -u phpBB' _for a
particular domain_?  You're telling portage to maintain a package, and
upgrade parts of it randomly.  I simply don't think portage will ever have
any way of handling multiple installs of a package in this manner.  How
about we (Gentoo) just provide single domain/host installs and maybe
SLOTing different versions so they won't be overwritten on upgrades.
Beyhond that, we have to let the sysadmins know to cp -l the installed
files into their vhosts.

AFAIK, no other distro has support for virtual hosting because of this very
issue.

> That's up to you.  If you're maintaining webapp ebuilds currently in
> portage,  though, I'd urge you not to stop maintaining them just because
> you're waiting  for a design solution via this GLEP.

Not in charge of anything else in portage.  Though I'd like to help out
with Horde down the road (seeing as I'm on the horde dev team as well).

>> Right, but as we already agreed, not all apps need to have their files
>> owned by apache:apache.  This should be configurable in the eclass.
>> Correct?
> 
> Do we need a new user/group to own most of the files?  And then we just
> make  the files that need write-access owned by the webserver?

The most robust solution would be to create a new group (web?) and make all
the webserver users be a part of that group and make these directory
group-owned by said group.  (Similar to the "mail" group for many
mail-related services)

> In his email in that thread you pointed me at, Robin was explicitly
> talking  about /var being a read-only NFS mount at install time.

Ok, install-time aside, the installed, writeable directories cannot exist
on a readonly mount for day-to-day operations.  Which takes me back to
wanting to support a single instance install of an app that an admin has to
copy in place if the default location is unacceptable.

>> So in any case, we have to pull the
>> apache-specific things out into a separate framework.  Therefore things
>> like DocumentRoot can't even be considered.  So a central location for
>> webapps must be once again taken into account.  
> 
> Yep.  How does this sound as the design of the central layout?  Let's
> agree a  design, so that it can be added to the GLEP.
> 
> * /usr/webapps/<app-name> as the main directory.  
> * /usr/webapps/<app-name>/public_html/ for files served by the web server
> * /usr/webapps/<app-name>/cgi-bin/ for CGI-BIN files
> * /etc/webapps/<app-name>/ to hold the box-default config files
> * <app-name> is ${PN} for non-slotted packages
> * <app-name> is ${P} for slotted packages

I'd say <app-name> should just be ${PF} for both and not worry about what
is SLOTed and what is not -- it works itself out.  Otherwise, this seems ok
to me and is easy to implement in the eclass.  The only thing I would
change is the names public_html and cgi-bin.  There may be no html and not
binary files in them (respectively).  How's about 'public' and 'cgi'?  (I
know, I know, nitpicking).

> a) you need a standard way of detecting which one is installed, and 

Is the approach currently in the eclass unacceptable? (Check for the
existance of the directory/configfile)

> b) you need to honour the 'apache2' USE flag

Yup, see above...no higher...no, higher than that.  Ok there, stop!

--mk

--
gentoo-dev@gentoo.org mailing list


^ permalink raw reply	[flat|nested] 35+ messages in thread
* [gentoo-dev] [GLEP] Web Application Installation
@ 2003-08-07  1:08 Troy Dack
  0 siblings, 0 replies; 35+ messages in thread
From: Troy Dack @ 2003-08-07  1:08 UTC (permalink / raw
  To: glep; +Cc: gentoo-dev@gentoo.org


[-- Attachment #1.1: Type: text/plain, Size: 861 bytes --]

Please find attached a GLEP (#11?) detailing standards for Gentoo Web
Application Installations.

Some areas (notably vhost related) require more detail.

The initial draft of this GLEP resulted in a very productive discussion
on the gentoo-dev mailing list, with a number of developers
contributing.

Acceptance of this GLEP will enable the developers of the required tools
to complete the tools and the details of the proposal.

-- 
Troy Dack        "Yes, yes, I know that, Sydney ... Everybody knows that!
tad@gentoo.org    ... But look: Four wrongs squared, minus two wrongs to 
                  the fourth power, divided by this formula, do make a
                  right." -- Gary Larson

Public Key: http://pgp.mit.edu:11371/pks/lookup?op=get&search=0x4D90BE3C
Key fingerprint = 1F3D 6C15 16AA 09D5 0C96  92E5 FD89 16F9 4D90 BE3C
 

[-- Attachment #1.2: glep-webapps-1.1.txt --]
[-- Type: text/plain, Size: 9809 bytes --]

GLEP: 11
Title: Web Application Installation
Version: $Revision: 1.1 $
Last-Modified: $Date: 2003/08/07 10:00 $
Author:	Troy Dack <tad@gentoo.org>
Discussions-To: gentoo-dev@gentoo.org
Status: Draft
Type: Standards Track
Content-Type: text/x-rst
Created: 02 August 2003
Post-History: 03 Aug 2003

Credits
=======

Based on comments posted to gentoo-dev mailing list [#WebAppPost1]_
[#WebAppPost2]_ [#WebAppPost3]_ by:

	Stuart Herbert <stuart@gentoo.org>, Max Kalika <max@gentoo.org>,
	Robin H.Johnson <robbat2@gentoo.org> and others

Definitions
===========

	*Web Application*
		an application that requires a web server to function and interacts with
		the user via a browser

	*Web Application Instance*
		An apparent install of the Web Application that is served up via the
		webserver. There may be any number of instances per Web Application.
		This is a major use for web applications. Our Gentoo Zope setup
		already provides instances and can be used for some concepts on this
		matter.

	*Web Application Setup Program*
		A script similar in function to zope-config that sets up instances.

	*Document Root*
		a location in the file system that forms the main document tree visible from
		the web

Conventions
===========

	When describing the location of a directory in the file system it
	wil be shown *with* a trailing, eg::

		/foo/bar/

	When describing the location of a specific file (irrespective of any
	file extention) it will shown *with out* a trailing slash, eg::

		/foo/blah

Abstract
========

To define where and how web based applications should be installed by Gentoo.

Motivation
==========

Currently there is no standard defined regarding the installation of web
based applicaitons in Gentoo.  This leads to ebuild authors creating a
variety of methods to determine:

	* where the application should be installed
	* what user and permissions the application should be given
	* where any configuration files related to the application should be
	  installed.

Due to a lack of standard install method configuration files are at
risk of being overwritten during upgrade, potentially causing system
administrators down tine as they have to reconfigure web applications
after an upgrade.

Rationale
=========

A discussion on the gentoo-dev mailing list [#WebAppPost1]_ raised the
following points regarding how Gentoo handles the installation of web based
applications:

	1. Gentoo installed web applications (eg: horde, phpbb, cacti,
	   phpmysql) should not be installed in the Document Root of a web server.
	2. Web applications should not have their configuration files installed
	   under the Document Root of a web server.

		i. Web Application must be slotted by their major version numbers to
		   further avoid downtime when true configuration changes are required.

	3. Web applications should not be owned by the same user as the web server.
	4. It should be easily possible to have multiple instances of a web
	   application without any duplication of source files.
	5. It should be immediately apparent how to control instances of a web
	   application.

Implementation
==============

Max Kalika <max@gentoo.org> stated that he has a preliminary eclass that
implements a good deal of this GLEP.

Stuart Herbert <stuart@gentoo.org> has committed::

	webapp-apache.eclass

to CVS, this is a stop-gap measure whilst this GLEP is being finalised.


1. Web Server
---------------------

A common default web server will have to be selected and ebuild authors should
ensure that their applications contain configuration directives suitable for
that server.  Given the popularity of the Apache web server it is suggested
that Apache be selected as the Gentoo default web server.

Whilst it is acknowledged that other web servers do exist and are used, there
has to be an assumption made somewhere that people who choose to use something
other than the default have enough knowledge to adapt configurations
accordingly.

1.1 Default Document Root
'''''''''''''''''''''''''

To ensure the greatest flexibility when installing applications the following
*Document Root* locations are to be used:

	* For single host installations::

		/var/www/localhost/htdocs/

	* For multiple virtual host installastions::

		/var/www/<fully qualified domain name>/htdocs/
		eg:
			/var/www/www.gentoo.org/htdocs/

1.2 Apache 2
''''''''''''''''''''''''

All web application .ebuild will honour any USE flags that are intended to
add support for Apache 2 as well as supporting Apache 1 installations.


2. Virtual Host Flexibility
---------------------------

In a similar vein to Gentoo's Zope scripts, namely zope-config, we
should be able to have multiple instances of a single web application
without duplicating all of the files.

This also allows system administrators to control where web applications
will appear on their system, as well as to customize a file in a single
instance of a web application without effecting the original material.

This is easily acheived thru use of Apache configuration directivies and
symlinks. For PHP instances, see http://tavi.sourceforge.net/VirtualHosts
for some details.

The primary idea here is that to the web-application, it appears that
all of it's configuration and files are in the instance directory, but
the files are physicalled located elsewhere.

2.1 New "vhost" USE Flag
''''''''''''''''''''''''

To enable support for multiple virtual host installations a new USE flag is
to be added to Portage. The use flag will be::

	vhost

When *vhost* is _set_ the installation location and configuration for the web
application will be effected, see below for more details.

2.2 VHost Configuration Tool
''''''''''''''''''''''''''''

To assist administration of multiple virtual hosts a "VHost Configuration Tool"
needs to be developed and implemented.  Initial discussion and regarding the VHost
Config tool can be found at http://article.gmane.org/gmane.linux.gentoo.devel/10874.

The VHost Configuration Utility will need to be a seperate package, maintained by Gentoo.
Apache .ebuilds will require the VHost Config tool as a dependency (DEPEND).

<< TO BE EXPANDED UPON >>

3. Application Installation Location
------------------------------------

The current accepted standard Document Root in Gentoo is /home/httpd.  The
discussion suggest that this is not the best location to install web based
applications.

Web applications should be installed outside of the Document Root using the following
defaults:

	* for files to be served to clients::

		/usr/share/webapps/${PF}/

		/usr/share/webapps/${PF}/public_html/ for files served by the web server

		/usr/share/webapps/${PF}/cgi-bin/ for CGI-BIN files

	* install configuration files in::

		/etc/webapps/${PF}/

	* for documentation files (not served to clients)::

		/usr/share/doc/${PF}/

3.1 Single Host Installation
''''''''''''''''''''''''''''

For single host installations the .ebuild will make the required
configurations changes and symlinks using the VHost Config tool to ensure
that the web application is available to be served from::

	/var/www/localhost/htdocs/${PN}

3.2 Virtual Host Installation
'''''''''''''''''''''''''''''

For installations that support multiple virtual hosts the .ebuild will
install the web application into the default location and then leave configuration
to the user through the VHost Config tool.

<< TO BE EXPANDED UPON >>

4. Application Configuration
----------------------------

Having application configuration files in the Document Root of a web
server is a potential security risk.  Additionally given the way that many
ebuilds currently install web applications it can also lead to the
overwriting of important configuration files.

As stated above web application configuration files are to be installed into::

	/etc/webapps/${PF}/

By installing application configuration files in /etc Portage CONFIG_PROTECT
features can be used to ensure that configuration files are not overwritten.

4.1 Virtual Host Support
''''''''''''''''''''''''

<< TO BE EXPANDED UPON >>

5. Application Permissions
--------------------------

Installing web applications and giving the web server ownership of the files
is a security risk.  This can possibly lead to application configuration
files being accessed by unwanted third parties.

All web applications should be owned by *root* unless the application
absolutely requires write access to its installation directories at execution
time.

Backwards Compatibility
=======================

There may be some issues regarding compatibility with existing installs of
web applications.  This is particularly true if the default Document Root is
moved from what is accepted as the current standard (/home/httpd).

The main issues are:
	* transition of existing configuration files to the
	  /etc/webapps/${PF}/ directory.
	* modification/reconfiguration of applications so that they
	  are aware of the location of configuration files.
	* creating approriate Apache configuration snippets for inclusion
	  in the Apache configuration files.


References
==========

.. [#WebAppPost1] http://article.gmane.org/gmane.linux.gentoo.devel/10411
.. [#WebAppPost2] http://news.gmane.org/onethread.php?group=gmane.linux.gentoo.devel&root=%3C1059843010.5023.80.camel%40carbon.internal.lan%3E
.. [#WebAppPost3] http://news.gmane.org/onethread.php?group=gmane.linux.gentoo.devel&root=%3C86960000.1060038977%40valkyrie.lsit.ucsb.edu%3E

Copyright
=========

This document has been placed in the public domain.


[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

^ permalink raw reply	[flat|nested] 35+ messages in thread
* Re: [gentoo-dev] [GLEP] Web Application Installation
@ 2003-08-04 17:11 Max Kalika
  2003-08-04 22:16 ` Stuart Herbert
  0 siblings, 1 reply; 35+ messages in thread
From: Max Kalika @ 2003-08-04 17:11 UTC (permalink / raw
  To: stuart, Max Kalika, Troy Dack, gentoo-dev

Good morning!

Quoting Stuart Herbert <stuart@gentoo.org>:

> Apache supports virtual hosting already.  Tools such as phpBB and
> tikiwiki and  phpMyAdmin (to give but three examples) don't need explicit
> virtual host  support, because Apache handles that.  To run a copy of
> phpBB under two  different domains hosted on the same box, you install
> two copies of phpBB -  or at least simulate that situation using symlinks
> or whatever.

See, I don't think that running two (or more) copies of an application is
supporting virtual hosts.  Upgrading all the files is a pain.  Why not have
one installation of the core files and multiple installations of the config
files?  This is, of course, very application specific -- most of them will
still need database upgrades, but that has to be done by the sysadmin.

> Scary as I find it, one reason why many of my clients run Debian rather
> than  Gentoo on their production hardware is because they do that from a
> cron  script, and trust the results.  Personally, I've always introduced
> staging  platforms in environments where I've been wearing the Release
> Manager hat.

How do they handle db updates?  How do they handle config file updates?

> Possibly.  But I really wanted to clear out some outstanding bugs from 
> Bugzilla, and adding the eclass made it easy to do.

Fair enough.

> 1) Your eclass doesn't correctly (as I understand correct usage to be!) 
> support the apache2 USE flag.  Easy enough to do - see my webapp-apache 
> eclass for an example.

Why base it on the flag?  If the webserver is installed and is supported,
configure for it.

> 2) Your eclass doesn't specify the permissions that the source files
> should be  installed under.  Again, easy enough to fix.

This is very straightforward to add.

> 3) Your eclass doesn't provide support for running multiple copies of the
> same  app on the same machine.  This is a showstopper.

This whole thing needs a separate discussion.  There's no portage support
for dealing with more than one installation of the same version of the same
package.

> 4) Your eclass requires the admin to stop and start Apache as part of the 
> install.  This is a showstopper.  Not every site will want to stop and
> start  their web server just because they've installed a new app.
> Imagine a site  hosting hundreds of domains, and having to take them
> *all* offline at the  same time just because phpBB's been upgraded (for
> example!).  Robin's idea of  creating .htaccess files under the document
> root deals with this much more  managably - although I think we're gonna
> end up using symlinks, as that'll  make it easy to support multiple web
> servers.

..htaccess files only work in already web-accessable directories (in your
case, DocumentRoot).  If we're putting applications in /usr/share/webapps,
the webserver has no idea to look there. Which is what the Alias directive
does in all the .conf files that are generated.  However, Alias is not
allowed in .htaccess AFAIK.

http://httpd.apache.org/docs-2.0/mod/mod_alias.html#alias

Besides that, the server doesn't have to be restarted -- just HUPed.

> 5) I'm against users having to run 'ebuild /var/.../something.ebuild'
> after  installation to then complete the install.  If you're emerging
> enough  packages, it's easy to miss the notification that you need to do
> this.  It's  personal preference, I guess, but I've been caught out by
> this in the past.

Ok, if we can achieve the same thing without the post-install intervention,
all for the better.

> 6) I *like* the idea that check_php() is in this
> eclass, because that check is  specific to mod_php under Apache.  I'm
> gonna steal that and add it to my  webapp-apache eclass ;-)

Steal away!  Keep in mind it is not completely accurate if the per-package
USE flags goes into portage (http://bugs.gentoo.org/show_bug.cgi?id=13616).
Things in PUSE will also have to be taken into account.  But this is a
completely different matter and will be easy to integrate.

> 7) Your variable names are not generic enough for my liking ;-)
> AWEB_CFG, for  example, might be better off being WEBAPP_CFG.

Hey!  I originally had them as WEBAPP_* but was told to change the eclass
to apache-webapp to distinguish from other servers so I wanted the
variables to reflect the name of the eclass.  I'm as flexible on this as
playdoh. :-)

> 8) Instead of trying to supply an all-encompasing
> apache-webapp_src_install(),  relying as it does on defining global
> variables, I'd have supplied a number  of individual functions to do each
> bit.  Say, a webapp-install-appconfig,  webapp-install-serverconfig each
> taking parameters (this is off the top of my  head here ;-)  This is a
> personal preference thing.

Thats just more to call from the ebuild.  My goal was to have very small
ebuilds -- just declare a few variables and inherit the eclass.

> 9) If I'm not mistaken, your eclass does nothing to ensure that the
> webapp can  find the configuration files you've moved into
> /etc/webapps/$PN/.   Personally, I'm coming to the conclusion that
> /etc/webapps/$PN/ isn't a good  idea, because again it doesn't support
> the idea of running multiple copies of  the same app on the same machine.

I'd say we should be shooting for an easy upgrade path first.  If config
files aren't stored in /etc/webapps/${PN}, then we need to have a way of
generating an env.d/${PN} which contains a CONFIG_PROTECT line, otherwise
we're forcing sysadins to reconfigure the application at every upgrade.
And as I mentioned before, there's no way for portage to handle
multiple-copy installs, so I'm not sure the best way to go about achieving
this goal.

> 10) I'm coming to the conclusion that 'emerge -u <webapp>' shouldn't
> overwrite  the older version, but should always install alongside, in a
> different slot,  so that sites can easily run different versions of apps
> as required.  Perhaps  this should be configurable somehow?  Your eclass
> doesn't make this possible.

Why should this be handled any different then the rest of apps handled by
portage?  If a sysadmin doesn't want older versions removed, just add
AUTOCLEAN=no to make.conf.  Granted, the current eclass doesn't use the
full ${PF} in the target directory, but that is easily changed.

> I'm not saying that my eclass is better.  My eclass isn't trying to
> address  this GLEP - it's just trying to make maintenance easier for now.
> The show  must go on.  Especially as this GLEP isn't even showing up on
> the GLEP list  yet.

Ok.  Hopefully we can get this thing hashed out and out the door so things
can start getting into shape soon.

>> Double Ugh!  
> 
> Lots of laughter.  I'm gonna copyright that phrase, I think ;-)

I have my moments. ;-)

> 1) Apache1/Apache2 conundrum
> 
> My eclass uses the detection technique adopted for mod_php, and no-one
> has  complained about that.  If this eclass is invalid, then so's the
> ebuild for  mod_php.  And I don't think it is.

Ok, this one is a keeper.

> 2) Support for multiple DocumentRoot configurations, and also
> 3) Binary packages installing on machines with different DocumentRoot
> values
> 
> Until the GLEP is firmed up and approved, we don't have an agreed
> solution to  implement.  

Ah!  Ok, so before we have this issues ironed out, the current status-quo
will just be maintained.  Fair enough.

> Please excuse me, but I don't want to put support for existing ebuilds on
> hold  while we debate the GLEP.  I believe that we *have* to continue
> support until  we're ready and able to switch.  Stopping maintenance
> activities is *not* an  option.

Since I'm the laziest person I know, I didn't want to do the work twice.
I'll just hold off on introducing more ebuilds which will have to be
converted later. :-)

> 4) Which user/group to use
> 
> My class uses Robin's suggestion, and assumes that Apache is running with
> the  default settings of apache.apache.

Right, but as we already agreed, not all apps need to have their files
owned by apache:apache.  This should be configurable in the eclass.
Correct?

> 5) DocumentRoot pointing to a read-only mount
> 
> As far as I'm concerned, that's like trying to do 'make bzlilo' with
> /boot not  mounted, or run an 'emerge' with /usr mounted read-only.  It's
> the sysadmin's  job to make sure that any necessary filesystems are
> mounted read/write before  an installation is attempted.  This is not a
> problem unique to web  applications.

The issue is not whether DocumentRoot is a read-only mount during the
install, but during the day-to-day operations.  Running 'emerge' with /usr
mounted read-only is not the same as having apache running a webapp that
needs to write to /usr -- one is done seldomly, the other is all the time. 

> 6) "it's weak"
> 
> That's not an argument, it's an opinion ;-)  Anyway, I've taken 5-10
> lines of  broken and incorrectly duplicated code from a number of
> ebuilds, and moved  them all into one place where they can be re-used and
> maintained for now.   Reduced defects is a strong argument, not a weak
> one.

I never made the "weak" argument -- I don't believe in it myself.  Things
need to be justified a bit better than that, so I'm completely with you on
this one.

> Then I'd argue that the GLEP is incomplete in its scope, and needs
> expanding.   Unless you think that we can create the 'web-xxx' categories
> without a GLEP?   If so, then fair enough ;-)
> 
> Actually, I'd like to see a redrafting of the GLEP, with a clearly
> defined  list of requirements for supporting webapps.  We're debating
> implementation  issues without a clear list of requirements to address.
> This is not the  right way to go about it ;-)

Ok, agreed.  Troy, is there any chance for another draft with some of these
things incorporated?  (Thanks a bezillion, btw, for putting up with me) :-)

> It *was* fix-a-bug weekend ;-)

And a successful one at that from what I saw in my emerge --sync this
morning. :-)  Thanks!

> I am going to be adding a webapp or two that use the new eclass, but the 
> situation's no different to before the eclass existed.  The code in the 
> eclass would have had to have been duplicated in the new ebuilds.

All you! :-)

> If we *don't* do this, I reckon that the bugs about webapps running under 
> Apache'll get addressed, and many of the other bugs won't - because it's
> too  much effort.
> 
> I agree that most people will just the webapps under Apache, and never
> care  about other web servers.  But either we just support webapps under
> Apache, or  we do it properly, I'd argue.  And I reckon that doing it
> properly is  actually very easy to do.

If you say it is very easy to do, I'm on board.  I can't personally speak
for other webservers, so I'll leave the decision of whether/how to support
others to those with the experience.  So in any case, we have to pull the
apache-specific things out into a separate framework.  Therefore things
like DocumentRoot can't even be considered.  So a central location for
webapps must be once again taken into account.  And from here stems the
problem of how to tell the webserver where to look for a specific
applications.  The .htaccess approach isn't viable.  For apache the .conf
system works, but yes, it requires the server to be HUPed after an upgrade
to an install of an application (not an upgrade -- once the app is
installed and running, upgrading doesn't require the server to be touched).

> See above.  My eclass isn't attempting to solve the GLEP.  It's there to
> help  with existing packages for now, until the GLEP moves forward.

ok, I didn't realize that it wasn't a permanent solution.  All is ok.

>> How is mod_php related to the way applications are installed?
> 
> Erm, how about the whole 'do I use Apache 1 or Apache 2' conumdrum?  See
> the  mod_php ebuild for details.

Whichever is installed gets configured.  Either or both get touched,
depending on what is detected.  It is up to the sysadmin to start one or
the either automatically with rc-update.  I don't see a problem if both are
configured (if detected), but only one is running.

--mk

--
gentoo-dev@gentoo.org mailing list


^ permalink raw reply	[flat|nested] 35+ messages in thread
* [gentoo-dev] [GLEP] Web Application Installation
@ 2003-08-02 16:50 Troy Dack
  2003-08-02 20:39 ` Robin H.Johnson
                   ` (3 more replies)
  0 siblings, 4 replies; 35+ messages in thread
From: Troy Dack @ 2003-08-02 16:50 UTC (permalink / raw
  To: gentoo-dev; +Cc: Gentoo Core


[-- Attachment #1.1: Type: text/plain, Size: 979 bytes --]

Recently there was a thread on where web apps should be placed
(inside/outside of the htdocs tree), where their config files should be
placed and what category they should be in.

Please review the attached GLEP and comment on my attempt to formalise
and summarise this discussion.

I'll submit this GLEP on 08 AUG 03 @ 0200 UTC (1200 .au EDT) with any
additions or suggestions that I receive.

This message has been CC'd to -core for those that may not keep up with
-dev.

Please post all replies to the gentoo-dev mailing list.

Thankyou

-- 
Troy Dack           "Yes, yes, I know that, Sydney ... Everybody knows that!
tad@gentoo.org    ... But look: Four wrongs squared, minus two wrongs to 
                  the fourth power, divided by this formula, do make a
                  right." -- Gary Larson

Public Key: http://pgp.mit.edu:11371/pks/lookup?op=get&search=0x4D90BE3C
Key fingerprint = 1F3D 6C15 16AA 09D5 0C96  92E5 FD89 16F9 4D90 BE3C
 

[-- Attachment #1.2: glep-webapps.txt --]
[-- Type: text/plain, Size: 5987 bytes --]

GLEP: xx
Title: Web Application Installation
Version: $Revision: 1.0 $
Last-Modified: $Date: 2003/08/03 00:30:0 $
Author:	Troy Dack <tad@gentoo.org>
Discussions-To: gentoo-dev@gentoo.org
Status: Draft
Type: Standards Track
Content-Type: text/x-rst
Created: 02 August 2003
Post-History:


Credits
=======

Based on comments posted to gentoo-dev mailing list [#WebAppPost]_ by Stuart
Herbert <stuart@gentoo.org> and Max Kalika <max@gentoo.org>.

Definitions
===========

	*Web Application*
		an application that requires a web server to function and interacts with
		the user via a browser

	*Document Root*
		a location in the file system that forms the main document tree visible from
		the web

Conventions
===========

	When describing the location of a directory in the file system it
	wil be shown *with* a trailing, eg::

		/foo/bar/

	When describing the location of a specific file (irrespective of any
	file extention) it will shown *with out* a trailing slash, eg::

		/foo/blah

Abstract
========

To define where and how web based applications should be installed by Gentoo.

Motivation
==========

Currently there is no standard defined regarding the installation of web
based applicaitons in Gentoo.  This leads to ebuild authors creating a
variety of methods to determine:

	* where the application should be installed
	* what user and permissions the application should be given
	* where any configuration files related to the application should be
	  installed.

Due to a lack of standard install method configuration files are at
risk of being overwritten during upgrade, potentially causing system
administrators down tine as they have to reconfigure web applications
after an upgrade.

Rationale
=========

A discussion on the gentoo-dev mailing list [#WebAppPost]_ raised the
following points regarding how Gentoo handles the installation of web based
applications:

	1. Gentoo installed web applications (eg: horde, phpbb, cacti,
	   phpmysql) should not be installed in the Document Root of a web server.
	2. Web applications should not have their configuration files installed
	   under the Document Root of a web server.
	3. Web applications should not be owned by the same user as the web server.

1. Default Web Server
---------------------

A common default web server will have to be selected and ebuild authors should
ensure that their applications contain configuration directives suitable for
that server.  Given the popularity of the Apache web server it is suggested that
Apache be selected as the Gentoo default web server.

Whilst it is acknowledged that other web servers do exist and are used, there
has to be an assumption made somewhere that people who choose to use something
other than a default have enough knowledge to adapt configurations accordingly.

2. Application Location
-----------------------

The current accepted standard Document Root in Gentoo is /home/httpd.  The
discussion suggest that this is not the best location to install web based
applications.

A suggested alternative installation location for web applications, outside of 
the Document:

	* for files to be served to clients::

		/usr/share/webapps/<application>/files/

	* for documentation files (not served to clients)::

		/usr/share/doc/webapps/<application>/

3. Application Configuration
----------------------------

Having application configuration files in the Document Root of a web
server is a potential security risk.  Additionally given the way that many
ebuilds currently install web applications it can also lead to the
overwriting of important configuration files.

A suggested alternative is:

	* install configuration files in::

		/etc/webapps/<application>/
		
	* Apache configuration directives:
	
		- install a sample file in::

			/usr/share/doc/webapps/<application>/conf/

		  the ebuild should inform the user how to include this 
		  information in their Apache configuration.

	  	- alternatively the configuration directives could be placed in::

			/etc/apache{1,2}/conf/webapps/<application>

		  this directory and the files in it should be included by the
		  main Apache configuration file.

By installing application configuration files in /etc Portage CONFIG_PROTECT
features can be used to ensure that configuration files are not overwritten.

4. Application Permissions
--------------------------

Installing web applications and giving the web server ownership of the files
is a security risk.  This can possibly lead to application configuration files 
being accessed by unwanted third parties.

All web applications should be owned by *root* unless the application absolutely
requires write access to its installation directories at execution time.

Implementation
==============

Max Kalika <max@gentoo.org> stated that he has a preliminary eclass that
implements a good deal of this GLEP.

Acceptance of the eclass into the portage tree will enable ebuild authors and
maintainers to transition the web applications to the new installation method
and locations.

Backwards Compatibility
=======================

There may be some issues regarding compatibility with existing installs of
web applications.  This is particularly true if the default Document Root is
moved from what is accepted as the current standard (/home/httpd).

The main issues are:
	* transition of existing configuration files to the 
	  /etc/webapps/<application>/ directory.
	* modification/reconfiguration of applications so that they
	  are aware of the location of configuration files.
	* creating approriate Apache configuration snippets for inclusion
	  in the Apache configuration files.

References
==========

.. [#WebAppPost] http://article.gmane.org/gmane.linux.gentoo.devel/10411

Copyright
=========

This document has been placed in the public domain.


[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

^ permalink raw reply	[flat|nested] 35+ messages in thread
end of thread, other threads:[~2003-08-07  1:08 UTC | newest]

Thread overview: 35+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2003-08-04 23:16 [gentoo-dev] [GLEP] Web Application Installation Max Kalika
2003-08-05  0:14 ` Stuart Herbert
2003-08-05  2:30   ` Donny Davies
2003-08-05 10:12     ` Stuart Herbert
2003-08-06  4:01       ` Donny Davies
2003-08-05  3:04   ` Max Kalika
2003-08-05 10:39     ` Stuart Herbert
2003-08-05  9:34   ` Paul de Vrieze
2003-08-05 11:19     ` Stuart Herbert
2003-08-05 11:37       ` Paul de Vrieze
2003-08-05 21:00       ` Max Kalika
2003-08-05 23:43         ` Cal Evans
2003-08-06  1:54           ` Stuart Herbert
2003-08-06  2:16             ` Robin H.Johnson
2003-08-06  2:44               ` Stuart Herbert
2003-08-06  4:37                 ` [gentoo-dev] [GLEP] Web Application Installation. Plotting a VHOST config tool Robin H.Johnson
2003-08-06 12:48                   ` Stuart Herbert
  -- strict thread matches above, loose matches on Subject: below --
2003-08-07  1:08 [gentoo-dev] [GLEP] Web Application Installation Troy Dack
2003-08-04 17:11 Max Kalika
2003-08-04 22:16 ` Stuart Herbert
2003-08-05  9:49   ` Michael Cummings
2003-08-02 16:50 Troy Dack
2003-08-02 20:39 ` Robin H.Johnson
2003-08-02 23:11 ` [gentoo-dev] Re: [gentoo-core] " Max Kalika
2003-08-02 23:51   ` Stuart Herbert
2003-08-03  2:26     ` [gentoo-dev] " Max Kalika
2003-08-03 14:46       ` Stuart Herbert
2003-08-03 15:20         ` Max Kalika
2003-08-03 17:43           ` Stuart Herbert
2003-08-03 19:03             ` Max Kalika
2003-08-03 19:43               ` Stuart Herbert
2003-08-04  4:29                 ` Max Kalika
2003-08-04 10:43                   ` Stuart Herbert
2003-08-03  0:30 ` Austin Frank
2003-08-03  7:50   ` Tal Peer
2003-08-03 14:45   ` Don Seiler
2003-08-05  8:12 ` Troy Dack

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox