From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from pigeon.gentoo.org ([208.92.234.80] helo=lists.gentoo.org) by finch.gentoo.org with esmtp (Exim 4.60) (envelope-from ) id 1QkAiJ-0003OK-Ri for garchives@archives.gentoo.org; Fri, 22 Jul 2011 08:04:04 +0000 Received: from pigeon.gentoo.org (localhost [127.0.0.1]) by pigeon.gentoo.org (Postfix) with SMTP id 8006B21C051; Fri, 22 Jul 2011 08:03:54 +0000 (UTC) Received: from smtp.gentoo.org (smtp.gentoo.org [140.211.166.183]) by pigeon.gentoo.org (Postfix) with ESMTP id 3976721C051 for ; Fri, 22 Jul 2011 08:03:54 +0000 (UTC) Received: from pelican.gentoo.org (unknown [66.219.59.40]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by smtp.gentoo.org (Postfix) with ESMTPS id 9BA3B1B4012 for ; Fri, 22 Jul 2011 08:03:53 +0000 (UTC) Received: from localhost.localdomain (localhost [127.0.0.1]) by pelican.gentoo.org (Postfix) with ESMTP id C102D8003D for ; Fri, 22 Jul 2011 08:03:52 +0000 (UTC) From: "Zac Medico" To: gentoo-commits@lists.gentoo.org Content-type: text/plain; charset=UTF-8 Reply-To: gentoo-dev@lists.gentoo.org, "Zac Medico" Message-ID: <3f8a90683ddc542a0db58e00e55733ee8daaa4ee.zmedico@gentoo> Subject: [gentoo-commits] proj/portage:master commit in: pym/portage/util/_dyn_libs/ X-VCS-Repository: proj/portage X-VCS-Files: pym/portage/util/_dyn_libs/LinkageMapELF.py X-VCS-Directories: pym/portage/util/_dyn_libs/ X-VCS-Committer: zmedico X-VCS-Committer-Name: Zac Medico X-VCS-Revision: 3f8a90683ddc542a0db58e00e55733ee8daaa4ee Date: Fri, 22 Jul 2011 08:03:52 +0000 (UTC) Precedence: bulk List-Post: List-Help: List-Unsubscribe: List-Subscribe: List-Id: Gentoo Linux mail X-BeenThere: gentoo-commits@lists.gentoo.org Content-Transfer-Encoding: quoted-printable X-Archives-Salt: X-Archives-Hash: d47d9b0e9c77bc67b6c1faeeaa6be590 commit: 3f8a90683ddc542a0db58e00e55733ee8daaa4ee Author: Zac Medico gentoo org> AuthorDate: Fri Jul 22 07:58:56 2011 +0000 Commit: Zac Medico gentoo org> CommitDate: Fri Jul 22 07:58:56 2011 +0000 URL: http://git.overlays.gentoo.org/gitweb/?p=3Dproj/portage.git;a= =3Dcommit;h=3D3f8a9068 LinkageMapELF: optimize memory usage This reduces memory consumption by approximately 30%, by replacing mutable set instances with arrays, tuples, and frozensets where appropriate. Also, identical frozenset instances are shared when available. --- pym/portage/util/_dyn_libs/LinkageMapELF.py | 29 ++++++++++++++++++---= ----- 1 files changed, 20 insertions(+), 9 deletions(-) diff --git a/pym/portage/util/_dyn_libs/LinkageMapELF.py b/pym/portage/ut= il/_dyn_libs/LinkageMapELF.py index 218ce2a..49a7d35 100644 --- a/pym/portage/util/_dyn_libs/LinkageMapELF.py +++ b/pym/portage/util/_dyn_libs/LinkageMapELF.py @@ -265,6 +265,10 @@ class LinkageMapELF(object): for x in plibs: lines.append(("plibs", ";".join(['', x, '', '', '']))) =20 + # Share identical frozenset instances when available, + # in order to conserve memory. + frozensets =3D {} + for location, l in lines: l =3D l.rstrip("\n") if not l: @@ -278,21 +282,23 @@ class LinkageMapELF(object): arch =3D fields[0] obj =3D fields[1] soname =3D fields[2] - path =3D set([normalize_path(x) \ + path =3D frozenset(normalize_path(x) \ for x in filter(None, fields[3].replace( "${ORIGIN}", os.path.dirname(obj)).replace( - "$ORIGIN", os.path.dirname(obj)).split(":"))]) - needed =3D [x for x in fields[4].split(",") if x] + "$ORIGIN", os.path.dirname(obj)).split(":"))) + path =3D frozensets.setdefault(path, path) + needed =3D frozenset(x for x in fields[4].split(",") if x) + needed =3D frozensets.setdefault(needed, needed) =20 obj_key =3D self._obj_key(obj) indexed =3D True myprops =3D obj_properties.get(obj_key) if myprops is None: indexed =3D False - myprops =3D (arch, needed, path, soname, set()) + myprops =3D (arch, needed, path, soname, []) obj_properties[obj_key] =3D myprops # All object paths are added into the obj_properties tuple. - myprops[4].add(obj) + myprops[4].append(obj) =20 # Don't index the same file more that once since only one # set of data can be correct and therefore mixing data @@ -309,16 +315,21 @@ class LinkageMapELF(object): soname_map =3D arch_map.get(soname) if soname_map is None: soname_map =3D self._soname_map_class( - providers=3Dset(), consumers=3Dset()) + providers=3D[], consumers=3D[]) arch_map[soname] =3D soname_map - soname_map.providers.add(obj_key) + soname_map.providers.append(obj_key) for needed_soname in needed: soname_map =3D arch_map.get(needed_soname) if soname_map is None: soname_map =3D self._soname_map_class( - providers=3Dset(), consumers=3Dset()) + providers=3D[], consumers=3D[]) arch_map[needed_soname] =3D soname_map - soname_map.consumers.add(obj_key) + soname_map.consumers.append(obj_key) + + for arch, sonames in libs.items(): + for soname_node in sonames.values(): + soname_node.providers =3D tuple(set(soname_node.providers)) + soname_node.consumers =3D tuple(set(soname_node.consumers)) =20 def listBrokenBinaries(self, debug=3DFalse): """