[gentoo-user] Wrong dependencies to postgresql

[gentoo-user] Wrong dependencies to postgresql

[Enrico Weigelt]

Hi folks,

since jakub (as always) closes all my bugs, I'll report the issue 
to this list before completely giving up and never ever waste a 
single second on reporting bugs ...

Lots of packages have an wrong/unnecessary dependency to 
postgresql. Three cases:

a) probably traditionally depended on the whole postgresql, maybe 
   since before libpq was an own package. ie. qt, dovecot, ...
   
b) many apps (ie. webapps like bugzilla) have postgresql as dep.,
   although they do not need it to be installed. (ie. bugzilla does 
   not have to do anything directly w/ postgresql, since it uses 
   perl-DBD for database access). Of course they maybe want to 
   have access to some postgres database, but this obviously does
   not need an local server.
   

cu
-- 
---------------------------------------------------------------------
 Enrico Weigelt    ==   metux IT service - http://www.metux.de/
---------------------------------------------------------------------
 Please visit the OpenSource QM Taskforce:
 	http://wiki.metux.de/public/OpenSource_QM_Taskforce
 Patches / Fixes for a lot dozens of packages in dozens of versions:
	http://patches.metux.de/
---------------------------------------------------------------------
-- 
gentoo-user@gentoo.org mailing list

Re: [gentoo-user] Wrong dependencies to postgresql

[Boyd Stephen Smith Jr.]

[-- Attachment #1: Type: text/plain, Size: 2330 bytes --]

On Saturday 23 December 2006 16:54, Enrico Weigelt <weigelt@metux.de> wrote 
about '[gentoo-user] Wrong dependencies to postgresql':
> since jakub (as always) closes all my bugs,

Jakub is pretty bugzilla savvy, are you sure you bugs weren't closed for 
valid reasons?  Sometime he does jump the gun though, so I don't doubt 
that some of your bugs may be valid -- you can always add more information 
to the bug and ask for it to be reopened.

> I'll report the issue 
> to this list before completely giving up and never ever waste a
> single second on reporting bugs ...

Bugzilla is *the only* official channel to get bugs reported so that the 
developers will fix them.  If you don't report your bugs, you'll just have 
to wait for someone else to report or the developers to "accidentally" fix 
the bug while working on something else.

> Lots of packages have an wrong/unnecessary dependency to
> postgresql.

I don't doubt it.  It's generally better to depend on too much rather than 
too little, and once things are "working" it's hard to get someone 
to "fix" it and run the risk of breaking it further.

> Three cases: 
>
> a) probably traditionally depended on the whole postgresql, maybe
>    since before libpq was an own package. ie. qt, dovecot, ...

Have you confirmed these actually compile just against libpg?

> b) many apps (ie. webapps like bugzilla) have postgresql as dep.,
>    although they do not need it to be installed. (ie. bugzilla does
>    not have to do anything directly w/ postgresql, since it uses
>    perl-DBD for database access). Of course they maybe want to
>    have access to some postgres database, but this obviously does
>    not need an local server.

Are you sure this isn't bringing in postgresql to satisfy some virtual?

In all cases, you've confirmed that the dependency is direct and not USE 
flag controlled?  Also, have you patched the ebuild with the 
changed/removed dependency and then sucessfully installed and run the 
package on a system *without postgresql* installed?

What's the third case?

-- 
"If there's one thing we've established over the years,
it's that the vast majority of our users don't have the slightest
clue what's best for them in terms of package stability."
-- Gentoo Developer Ciaran McCreesh

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-user] Wrong dependencies to postgresql

[Enrico Weigelt]

[-- Attachment #1: Type: text/plain, Size: 3964 bytes --]

* Boyd Stephen Smith Jr. <bss03@volumehost.net> wrote:

Hi,

> Jakub is pretty bugzilla savvy, are you sure you bugs weren't 
> closed for valid reasons? 

well, after some discussion @ gentoo-dev, it's now a little bit
clearer:

The bugs are not completely valid yet (but soon will be), since 
libpq (<=8.0.8) is still buggy/incomplete (missing pg_config), so
some packages may still need postgresql for building (not runtime).
I supplied fixed ebuild to the -dev, but they told me they don't 
see what I'm fixing ;-o 

Well, this issue is actually fixed in 8.0.9 (IMHO not completely
stabelized yet, but soon), while introducing another, even worse at 
the same time: libpq-8.0.9 doesn't wanna play w/ postgresql<=8.0.8,
but postgresql-8.0.9 wants exactly libpq-8.0.9. So there's no clean 
way to solve this. Someone @ b.g.o suggested completely removing 
postgres+libpq and installing afresh, but that's an absolutely 
no-go for production systems (IMHO). Disabling deps for upgrade
should work, but is unclean.

My fixed ebuild should fit into the gap by adding pg_config to 
libpq (overwriting the one from postgresql) and changing the
postgresql ebuild to be satisfied w/ this libpq version. 

> Sometime he does jump the gun though, 

Not only at me ? Some bit salving ;-)

<snip>

> Bugzilla is *the only* official channel to get bugs reported 
> so that the developers will fix them. If you don't report your 
> bugs, you'll just have to wait for someone else to report or 
> the developers to "accidentally" fix the bug while working on 
> something else.

I'll simply do it by myself. I have to maintain my own overlay
nevertheless. Maybe I could drop a note to -user if perhaps
anyone's interested in my stuff. But I had to learn filing bugs
and talking @ -dev are just a waste my (rare+expensive) time.

> > Lots of packages have an wrong/unnecessary dependency to
> > postgresql.
> 
> I don't doubt it.  It's generally better to depend on too much 
> rather than too little, and once things are "working" it's hard 
> to get someone to "fix" it and run the risk of breaking it further.

Wouldn't be such an problem w/ really cleanroom builds + tests
from the first place. Seems, Gentoo isn't made for that.

<snip>

> > Three cases: 
> >
> > a) probably traditionally depended on the whole postgresql, maybe
> >    since before libpq was an own package. ie. qt, dovecot, ...
> 
> Have you confirmed these actually compile just against libpg?

I'm still in the process. The lack of automatic cleanroom builds 
requires me to install an minimal jail before each build.
(most of my gentoo systems actually have postgresql server running)

> > b) many apps (ie. webapps like bugzilla) have postgresql as dep.,
> >    although they do not need it to be installed. (ie. bugzilla does
> >    not have to do anything directly w/ postgresql, since it uses
> >    perl-DBD for database access). Of course they maybe want to
> >    have access to some postgres database, but this obviously does
> >    not need an local server.
> 
> Are you sure this isn't bringing in postgresql to satisfy some virtual?

Which virtual should it be ? And why ?

> In all cases, you've confirmed that the dependency is direct and not 
> USE flag controlled? 

Of course, it is controlled by the postgresql useflag. But that doesn't
make it better. Bugzilla does not need an local postgres server.
And I didn't find any sign that it needs pg_config, so this current
libpq bug also shouldn't be an issue.


cu
-- 
---------------------------------------------------------------------
 Enrico Weigelt    ==   metux IT service - http://www.metux.de/
---------------------------------------------------------------------
 Please visit the OpenSource QM Taskforce:
 	http://wiki.metux.de/public/OpenSource_QM_Taskforce
 Patches / Fixes for a lot dozens of packages in dozens of versions:
	http://patches.metux.de/
---------------------------------------------------------------------

[-- Attachment #2: libpq-8.0.8-r1.ebuild --]
[-- Type: text/plain, Size: 3338 bytes --]

# Copyright 1999-2006 Gentoo Foundation
# Distributed under the terms of the GNU General Public License v2
# $Header: /var/cvsroot/gentoo-x86/dev-db/libpq/libpq-8.0.8-r2.ebuild,v 1.12 2006/07/30 22:32:28 chtekk Exp $

inherit eutils gnuconfig flag-o-matic toolchain-funcs

DESCRIPTION="Libraries of postgresql"
HOMEPAGE="http://www.postgresql.org/"
MY_P="postgresql-${PV}"
SRC_URI="mirror://postgresql/source/v${PV}/postgresql-base-${PV}.tar.bz2"

LICENSE="POSTGRESQL"
SLOT="4"
KEYWORDS="alpha amd64 arm hppa ia64 mips ppc ppc64 s390 sh sparc x86"
IUSE="ssl nls pam readline zlib kerberos"
#pg-hier"

S=${WORKDIR}/${MY_P}
DEPEND="virtual/libc
	sys-devel/autoconf
	>=sys-libs/ncurses-5.2
	>=sys-devel/bison-1.875
	zlib? ( >=sys-libs/zlib-1.1.3 )
	pam? ( virtual/pam )
	readline? ( >=sys-libs/readline-4.1 )
	ssl? ( >=dev-libs/openssl-0.9.6-r1 )
	nls? ( sys-devel/gettext )
	kerberos? ( virtual/krb5 )"
RDEPEND="virtual/libc
	zlib? ( >=sys-libs/zlib-1.1.3 )
	ssl? ( >=dev-libs/openssl-0.9.6-r1 )
	kerberos? ( virtual/krb5 )"

MAKEOPTS="${MAKEOPTS} -j1"

pkg_preinst() {
	# removing wrong symlink which is created by previous ebuild.
	if [ -L /usr/include/libpq ]; then
		rm /usr/include/libpq
	fi
}

src_unpack() {
	unpack ${A}
	cd "${S}"
	epatch "${FILESDIR}/${P}-gentoo.patch"
}

src_compile() {
	filter-flags -ffast-math -feliminate-dwarf2-dups

	local myconf
	use ssl && myconf="$myconf --with-openssl"
	use nls && myconf="$myconf --enable-nls"
	use pam && myconf="$myconf --with-pam"
	use readline || myconf="$myconf --without-readline"
	use zlib || myconf="$myconf --without-zlib"
	use kerberos && myconf="$myconf --with-krb5"

	# these are the only working CFLAGS I could get on ppc, so locking them
	# down, anything more aggressive fails (i.e. -mcpu or -Ox)
	# Gerk - Nov 26, 2002
	use ppc && CFLAGS="-pipe -fsigned-char"

	# Detect mips systems properly
	gnuconfig_update

	./configure --prefix=/usr \
		--include=/usr/include/postgresql/libpq-${SLOT} \
		--mandir=/usr/share/man \
		--host=${CHOST} \
		--with-docdir=/usr/share/doc/${PF} \
		--libdir=/usr/$(get_libdir) \
		--enable-depend \
		$myconf || die

	cd ${S}/src/interfaces/libpq
	emake LD="$(tc-getLD) $(get_abi_LDFLAGS)" || die

	cd "${S}/src/bin/pg_config" || die "could not cd to pg_config"
	emake -j1 LD="$(tc-getLD) $(get_abi_LDFLAGS)" || die "emake pg_config failed"
}

src_install() {
	cd ${S}/src/interfaces/libpq
	make DESTDIR=${D} LIBDIR=${D}/usr/$(get_libdir) install || die

	cd ${S}/src/include
	make DESTDIR=${D} install || die

	cd ${S}
	dodoc README HISTORY COPYRIGHT INSTALL

	dosym libpq-${SLOT}.a /usr/$(get_libdir)/libpq.a

	for f in ${D}/usr/include/postgresql/libpq-${SLOT}/*.h
	do
		dosym postgresql/libpq-${SLOT}/$(basename $f) /usr/include/
	done

	dodir /usr/include/libpq
	for f in ${D}/usr/include/postgresql/libpq-${SLOT}/libpq/*.h
	do
		dosym ../postgresql/libpq-${SLOT}/libpq/$(basename $f) /usr/include/libpq/
	done

	cd ${D}/usr/include/postgresql/libpq-${SLOT}
	for f in $(find * -name '*.h' -print) ; do
		destdir=$(dirname $f)
		if [ ! -d "${D}/usr/include/postgresql/${destdir}" ]; then
			dodir /usr/include/postgresql/${destdir}
		fi
		dosym /usr/include/postgresql/libpq-${SLOT}/${f} /usr/include/postgresql/${destdir}/
	done

	cd "${S}/src/bin/pg_config"
	emake DESTDIR="${D}" install || die "emake install pg_config failed"
}

Re: [gentoo-user] Wrong dependencies to postgresql

[Boyd Stephen Smith Jr.]

[-- Attachment #1: Type: text/plain, Size: 7605 bytes --]

On Sunday 24 December 2006 19:37, Enrico Weigelt <weigelt@metux.de> wrote 
about 'Re: [gentoo-user] Wrong dependencies to postgresql':

First of all, thanks for responding, I was afraid we had lost another savvy 
bugzilla user to the lack of (or discontent with) response from the 
developers.

> * Boyd Stephen Smith Jr. <bss03@volumehost.net> wrote:
> > Jakub is pretty bugzilla savvy, are you sure you bugs weren't
> > closed for valid reasons?
> well, after some discussion @ gentoo-dev, it's now a little bit
> clearer:

> The bugs are not completely valid yet (but soon will be), since
> libpq (<=8.0.8) is still buggy/incomplete (missing pg_config), so
> some packages may still need postgresql for building (not runtime).
> I supplied fixed ebuild to the -dev, but they told me they don't
> see what I'm fixing ;-o

Heh, like I mentioned/alluded to in my previous posting, once a package it 
working, winnowing down the dependencies is sometimes not seen as 
productive.  I sympathizes with your blight and would volunteer my system 
for chroot building were it not for my lack of time (etc.).

> Well, this issue is actually fixed in 8.0.9 (IMHO not completely
> stabelized yet, but soon), while introducing another, even worse at
> the same time: libpq-8.0.9 doesn't wanna play w/ postgresql<=8.0.8,
> but postgresql-8.0.9 wants exactly libpq-8.0.9. So there's no clean
> way to solve this. Someone @ b.g.o suggested completely removing
> postgres+libpq and installing afresh, but that's an absolutely
> no-go for production systems (IMHO). Disabling deps for upgrade
> should work, but is unclean.

Perhaps a dependency on '|| (>=category/libpq-8.0.9 
<category/postgresql-8.0.9)' would be appropriate in this case.  I believe 
that ordering should prefer the library instead of the whole RDBMS.

In addition, a proper blocker should be added to libpq; I know it doesn't 
help automated upgrades move forward, but at least is stops them from 
blocking issues that *could* be problematic.

> My fixed ebuild should fit into the gap by adding pg_config to
> libpq (overwriting the one from postgresql) and changing the
> postgresql ebuild to be satisfied w/ this libpq version.

Hrm, sounds as if you want a single file to be provided by two packages.  
This is not "the Gentoo way".  In Debian, we have the alternatives 
interface; in gentoo, it's assumed you manage such symlinks with either 
eselect or manually.

If the pg_configs are *different*, you should patch both libpq (to install 
pg_config as pg_config-libpq-$SLOT) and postgresql (to install pg_config 
as pg_config-postgresql-$SLOT) and provide an eslect module to choose the 
appropriate pg_config.

If the pg_configs are *the same*, you should (a) add them to libpq and have 
postgresql depend on libqg--not installing it's own pg_config. or (b) have 
pg_config be it's own package, depended on by both libpq and postgresql.

> > Sometime he does jump the gun though,
>
> Not only at me ? Some bit salving ;-)

Oh yes.  Particularly when marking bugs as duplicates.  There were one or 
two recent out-of-tree kernel module bugs that he marked as duplicates of 
the generic "kernel-2.6.18+ breaks sandbox" bug that weren't actuallt 
duplicates.

That said, Jakub is responsible for 50%+ of the hits to bugs.gentoo.org; he 
*is* for all intents and purposes the bugzilla manager.

Normally, he's correct, but don't worry about respectfully disagreeing with 
him.  He's still human and will make mistakes.

> But I had to learn filing bugs
> and talking @ -dev are just a waste my (rare+expensive) time.

Perhaps your time is more valuable than mine, but I've never found 
reporting bugs to be a waste of time.  Surely, patching the offending 
ebuild/source is better, but such patches should be attached to a bugs for 
the developers/other users to use as well.

> > > Lots of packages have an wrong/unnecessary dependency to
> > > postgresql.
> >
> > I don't doubt it.  It's generally better to depend on too much
> > rather than too little, and once things are "working" it's hard
> > to get someone to "fix" it and run the risk of breaking it further.
>
> Wouldn't be such an problem w/ really cleanroom builds + tests
> from the first place. Seems, Gentoo isn't made for that.

Well, there is a catalyst target for smokescreen/tinderbox building.  I'm 
not sure of the quality but it is supposed to start from a clean image for 
each rebuild, and simply use the binary packages previously built.   I'd 
assume it uses the ebuilds's test suite to verify post-install 
correctness, (yes, there is support for unit-sytle testing built into 
emerge) but I'm not sure the quality of those tests on libpq or 
postgresql.

> > > a) probably traditionally depended on the whole postgresql, maybe
> > >    since before libpq was an own package. ie. qt, dovecot, ...
> >
> > Have you confirmed these actually compile just against libpg?
>
> I'm still in the process. The lack of automatic cleanroom builds
> requires me to install an minimal jail before each build.
> (most of my gentoo systems actually have postgresql server running)

Again, just so you know about the tools available, check out catalyst's 
smokescreen/tinderbox target.

But, before you preform such tests, I'd think filing a bug to be premature.  
Make sure there's a fixable issue before you potentially break packages.

> > > b) many apps (ie. webapps like bugzilla) have postgresql as dep.,
> > >    although they do not need it to be installed. (ie. bugzilla does
> > >    not have to do anything directly w/ postgresql, since it uses
> > >    perl-DBD for database access). Of course they maybe want to
> > >    have access to some postgres database, but this obviously does
> > >    not need an local server.
> >
> > Are you sure this isn't bringing in postgresql to satisfy some
> > virtual?
>
> Which virtual should it be ? And why ?

I'm not sure, I didn't do any investigation, but it could be something as 
simple as virtual/db or virtual/storage.  If it's bringing in postgresql 
as a virtual, then it depends on postgresql OR any package that can 
provide that virtual, as controlled by (IIRC) <profile>/virtual 
plus /etc/portage/profile/virutual.  In that case, I don't believe the bug 
would be well-founded, because the dependent package (e.g. bugzilla) 
definitely needs *somewhere* to store it's data.

> > In all cases, you've confirmed that the dependency is direct and not
> > USE flag controlled?
>
> Of course, it is controlled by the postgresql useflag. But that doesn't
> make it better. Bugzilla does not need an local postgres server.
> And I didn't find any sign that it needs pg_config, so this current
> libpq bug also shouldn't be an issue.

USE-flags can bring in non-requisite dependencies, for convenience.  If you 
don't want a package to bring in the USE-flag dependent dependencies, 
simply disable the flag for that package.

That said, if a use flags controls both *support for* and a *dependency on* 
a certain package, it is broken, and I feel you bug could be valid.  E.g. 
package Y without the postgres USE flag can't use a remote postgresql 
server but package Y with the postgres USE flags forces you to install a 
local postgresql server.

-- 
"If there's one thing we've established over the years,
it's that the vast majority of our users don't have the slightest
clue what's best for them in terms of package stability."
-- Gentoo Developer Ciaran McCreesh

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]