From mboxrd@z Thu Jan 1 00:00:00 1970
Received: from pigeon.gentoo.org ([208.92.234.80] helo=lists.gentoo.org)
by finch.gentoo.org with esmtp (Exim 4.60)
(envelope-from <gentoo-commits+bounces-392055-garchives=archives.gentoo.org@lists.gentoo.org>)
id 1RErbU-0002hm-NM
for garchives@archives.gentoo.org; Fri, 14 Oct 2011 23:55:52 +0000
Received: from pigeon.gentoo.org (localhost [127.0.0.1])
by pigeon.gentoo.org (Postfix) with SMTP id 5C11421C03B;
Fri, 14 Oct 2011 23:55:44 +0000 (UTC)
Received: from smtp.gentoo.org (smtp.gentoo.org [140.211.166.183])
by pigeon.gentoo.org (Postfix) with ESMTP id 1CC6A21C03B
for <gentoo-commits@lists.gentoo.org>; Fri, 14 Oct 2011 23:55:44 +0000 (UTC)
Received: from pelican.gentoo.org (unknown [66.219.59.40])
(using TLSv1 with cipher AECDH-AES256-SHA (256/256 bits))
(No client certificate requested)
by smtp.gentoo.org (Postfix) with ESMTPS id 965701B4018
for <gentoo-commits@lists.gentoo.org>; Fri, 14 Oct 2011 23:55:43 +0000 (UTC)
Received: from localhost.localdomain (localhost [127.0.0.1])
by pelican.gentoo.org (Postfix) with ESMTP id 647B38004F
for <gentoo-commits@lists.gentoo.org>; Fri, 14 Oct 2011 23:55:42 +0000 (UTC)
From: "Zac Medico" <zmedico@gentoo.org>
To: gentoo-commits@lists.gentoo.org
Content-type: text/plain; charset=UTF-8
Reply-To: gentoo-dev@lists.gentoo.org, "Zac Medico" <zmedico@gentoo.org>
Message-ID: <1e8870bd45a4e2a9c43e7f112701c6ae84b0fd56.zmedico@gentoo>
Subject: [gentoo-commits] proj/portage:master commit in: pym/portage/cache/, pym/portage/repository/
X-VCS-Repository: proj/portage
X-VCS-Files: pym/portage/cache/flat_hash.py pym/portage/repository/config.py
X-VCS-Directories: pym/portage/cache/ pym/portage/repository/
X-VCS-Committer: zmedico
X-VCS-Committer-Name: Zac Medico
X-VCS-Revision: 1e8870bd45a4e2a9c43e7f112701c6ae84b0fd56
Date: Fri, 14 Oct 2011 23:55:42 +0000 (UTC)
Precedence: bulk
List-Post: <mailto:gentoo-commits@lists.gentoo.org>
List-Help: <mailto:gentoo-commits+help@lists.gentoo.org>
List-Unsubscribe: <mailto:gentoo-commits+unsubscribe@lists.gentoo.org>
List-Subscribe: <mailto:gentoo-commits+subscribe@lists.gentoo.org>
List-Id: Gentoo Linux mail <gentoo-commits.gentoo.org>
X-BeenThere: gentoo-commits@lists.gentoo.org
Content-Transfer-Encoding: quoted-printable
X-Archives-Salt:
X-Archives-Hash: 9edc3d157fd62700d0b3fd10b1410dbb
commit: 1e8870bd45a4e2a9c43e7f112701c6ae84b0fd56
Author: Brian Harring <ferringb <AT> chromium <DOT> org>
AuthorDate: Fri Oct 14 09:40:00 2011 +0000
Commit: Zac Medico <zmedico <AT> gentoo <DOT> org>
CommitDate: Fri Oct 14 23:50:20 2011 +0000
URL: http://git.overlays.gentoo.org/gitweb/?p=3Dproj/portage.git;a=
=3Dcommit;h=3D1e8870bd
layout.conf: add git friendly pregenerated cache format
Enabled via cache-format =3D md5-dict
This format is essentially just flat_hash, using md5 rather than mtime,
and dropping the path component from _eclasses_ entries.
>From a speed standpoint, the md5 overhead is ~16% in comparison to mtime,
timed on a modern sandybridge; specifically, validating 29k nodes takes
~8.8s for flat_md5, while the pms norm is ~7.7s.
That said, the cache is /usable/ in places PMS is not; in those cases,
it can definitely be a win since even if the cache is partially old,
it's better than regenerating everything from scratch.
(cherry picked from commit 95ddf97e2f7e7d3f6a072604b2df5f77e9298558)
Change-Id: Ic3561369b7a8be7f86480f339ab1686fddea6dff
---
pym/portage/cache/flat_hash.py | 9 +++++++--
pym/portage/repository/config.py | 6 +++++-
2 files changed, 12 insertions(+), 3 deletions(-)
diff --git a/pym/portage/cache/flat_hash.py b/pym/portage/cache/flat_hash=
.py
index b6bc074..2eae9f6 100644
--- a/pym/portage/cache/flat_hash.py
+++ b/pym/portage/cache/flat_hash.py
@@ -31,7 +31,7 @@ class database(fs_template.FsBased):
self.label.lstrip(os.path.sep).rstrip(os.path.sep))
write_keys =3D set(self._known_keys)
write_keys.add("_eclasses_")
- write_keys.add("_mtime_")
+ write_keys.add("_%s_" % (self.validation_chf,))
self._write_keys =3D sorted(write_keys)
if not self.readonly and not os.path.exists(self.location):
self._ensure_dirs()
@@ -69,7 +69,6 @@ class database(fs_template.FsBased):
raise cache_errors.CacheCorruption(cpv, e)
=20
def _setitem(self, cpv, values):
-# import pdb;pdb.set_trace()
s =3D cpv.rfind("/")
fp =3D os.path.join(self.location,cpv[:s],".update.%i.%s" % (os.getpid=
(), cpv[s+1:]))
try:
@@ -153,3 +152,9 @@ class database(fs_template.FsBased):
dirs.append((depth+1, p))
continue
yield p[len_base+1:]
+
+
+class md5_database(database):
+
+ validation_chf =3D 'md5'
+ store_eclass_paths =3D False
diff --git a/pym/portage/repository/config.py b/pym/portage/repository/co=
nfig.py
index a67e7f1..cf268f8 100644
--- a/pym/portage/repository/config.py
+++ b/pym/portage/repository/config.py
@@ -136,9 +136,13 @@ class RepoConfig(object):
format =3D 'pms'
if format =3D=3D 'pms':
from portage.cache.metadata import database
+ name =3D 'metadata/cache'
+ elif format =3D=3D 'md5-dict':
+ from portage.cache.flat_hash import md5_database as database
+ name =3D 'metadata/md5-cache'
else:
return None
- return database(self.location, 'metadata/cache',
+ return database(self.location, name,
auxdbkeys, readonly=3Dreadonly)
=20
def load_manifest(self, *args, **kwds):