From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from pigeon.gentoo.org ([69.77.167.62] helo=lists.gentoo.org) by finch.gentoo.org with esmtp (Exim 4.60) (envelope-from ) id 1JMT41-0003a2-VO for garchives@archives.gentoo.org; Tue, 05 Feb 2008 19:02:38 +0000 Received: from pigeon.gentoo.org (localhost [127.0.0.1]) by pigeon.gentoo.org (Postfix) with SMTP id 82DE7E03ED; Tue, 5 Feb 2008 19:02:36 +0000 (UTC) Received: from smtp.ferdyx.org (170.Red-213-96-222.staticIP.rima-tde.net [213.96.222.170]) by pigeon.gentoo.org (Postfix) with ESMTP id F3CC6E03ED for ; Tue, 5 Feb 2008 19:02:35 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp.ferdyx.org (Postfix) with ESMTP id 124808D306 for ; Tue, 5 Feb 2008 20:12:53 +0100 (CET) X-Virus-Scanned: Debian amavisd-new at ferdyx.org Received: from smtp.ferdyx.org ([127.0.0.1]) by localhost (tungsteno.ferdyx.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id nMSFNlLm+R7b for ; Tue, 5 Feb 2008 20:12:47 +0100 (CET) Received: from snowcone (unknown [213.121.151.206]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by smtp.ferdyx.org (Postfix) with ESMTP id E37028D305 for ; Tue, 5 Feb 2008 20:12:46 +0100 (CET) Date: Tue, 5 Feb 2008 19:02:24 +0000 From: Ciaran McCreesh To: gentoo-dev@lists.gentoo.org Subject: Re: [gentoo-dev] Re: VDB access Message-ID: <20080205190224.59268128@snowcone> In-Reply-To: <20080205084355.45ade352@gentoo.org> References: <20080205010546.38414ec3@snowcone> <20080205084355.45ade352@gentoo.org> X-Mailer: Claws Mail 3.2.0 (GTK+ 2.12.5; x86_64-pc-linux-gnu) Precedence: bulk List-Post: List-Help: List-Unsubscribe: List-Subscribe: List-Id: Gentoo Linux mail X-BeenThere: gentoo-dev@lists.gentoo.org Reply-to: gentoo-dev@lists.gentoo.org Mime-Version: 1.0 Content-Type: multipart/signed; boundary="Sig_/p1cILSayOJlyCIWItubDQfR"; protocol="application/pgp-signature"; micalg=PGP-SHA1 X-Archives-Salt: bb2f66f8-b812-4544-8096-a7a672412bcd X-Archives-Hash: 42a8cbf451bef433073599dc8207603f --Sig_/p1cILSayOJlyCIWItubDQfR Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: quoted-printable On Tue, 5 Feb 2008 08:43:55 +0100 Christian Faulhammer wrote: > > This will let package managers use a format other than VDB. A well > > designed replacement can shave a minute off cold cache command > > times. >=20 > How much gain can be expected? There're two common operations that're really really slow (because they need lots of filesystem access) using VDB: turning a pkg into a cat/pkg and finding all PROVIDEd packages. A redesigned format can, for example, make the former a single filesystem operation and the latter a single directory read. If you're looking for numbers, you can see very roughly how long a VDB load takes off cold cache using: echo 2 | sudo dd of=3D/proc/sys/vm/drop_caches find /var/db/pkg/ -type f -not -name 'environment.bz2' \ | xargs cat >/dev/null A different on-disk format could avoid 90% of those filesystem accesses for many operations. There's a more subtle issue with VDB and the scope of locking required by multithreaded implementations. This one's not an issue for interpreted languages, but it's a minor nuisance in places where there's no language-induced locking. > And what package managers will allow native_built_with_use? I doubt Portage will, since if someone's going to make changes to Portage that'd let it use something other than VDB, they've clearly got enough time to implement use deps as a side project, which would make built_with_use obsolete. Paludis would certainly implement it -- we've already got a redesigned on-disk db format that we use for tracking unpackaged packages, and repurposing that to handle ebuilds wouldn't be very hard. Incidentally, if anyone is still thinking that built_with_use is fine... It doesn't handle IUSE defaults, which means package managers have to store a hacked version of IUSE in VDB for EAPI 1. This isn't documented anywhere and is highly unobvious. --=20 Ciaran McCreesh --Sig_/p1cILSayOJlyCIWItubDQfR Content-Type: application/pgp-signature; name=signature.asc Content-Disposition: attachment; filename=signature.asc -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.7 (GNU/Linux) iD8DBQFHqLLD96zL6DUtXhERAmBAAJ9AwKgcAaEKUkwgcx7mRnDH82lXZwCfVag9 QhRYWQlIft7lDTr/I7mgFww= =GGVi -----END PGP SIGNATURE----- --Sig_/p1cILSayOJlyCIWItubDQfR-- -- gentoo-dev@lists.gentoo.org mailing list