From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from lists.gentoo.org (pigeon.gentoo.org [208.92.234.80]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by finch.gentoo.org (Postfix) with ESMTPS id D1DD71382C5 for ; Sat, 27 Jun 2020 11:34:17 +0000 (UTC) Received: from pigeon.gentoo.org (localhost [127.0.0.1]) by pigeon.gentoo.org (Postfix) with SMTP id 0B54EE09DF; Sat, 27 Jun 2020 11:34:17 +0000 (UTC) Received: from smtp.gentoo.org (smtp.gentoo.org [140.211.166.183]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by pigeon.gentoo.org (Postfix) with ESMTPS id 8E00EE09DF for ; Sat, 27 Jun 2020 11:34:16 +0000 (UTC) Date: Sat, 27 Jun 2020 11:33:36 +0000 User-Agent: K-9 Mail for Android In-Reply-To: <20200627063415.936177-2-cshei@google.com> References: <20200627063415.936177-1-cshei@google.com> <20200627063415.936177-2-cshei@google.com> Precedence: bulk List-Post: List-Help: List-Unsubscribe: List-Subscribe: List-Id: Gentoo Linux mail X-BeenThere: gentoo-portage-dev@lists.gentoo.org Reply-to: gentoo-portage-dev@lists.gentoo.org X-Auto-Response-Suppress: DR, RN, NRN, OOF, AutoReply MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Subject: Re: [gentoo-portage-dev] [PATCH 1/3] Add caching to catpkgsplit function To: gentoo-portage-dev@lists.gentoo.org,Chun-Yu Shei From: =?UTF-8?Q?Micha=C5=82_G=C3=B3rny?= Message-ID: <7E3D00C7-69E5-46DD-8B6B-075408E20327@gentoo.org> X-Archives-Salt: 72ff5c65-6862-4237-973b-c400b68d1080 X-Archives-Hash: 1a927285a34217452bcab2216d22cac3 Dnia June 27, 2020 6:34:13 AM UTC, Chun-Yu Shei napisa= =C5=82(a): >According to cProfile, catpkgsplit is called up to 1-5=2E5 million times >during "emerge -uDvpU --with-bdeps=3Dy @world"=2E Adding a dict to cache >its >results reduces the time for this command from 43=2E53 -> 41=2E53 seconds >-- >a 4=2E8% speedup=2E Not saying caching is wrong for an interim solution but this is the kind o= f function where refactoring may yield even more gain=2E >--- > lib/portage/versions=2Epy | 7 +++++++ > 1 file changed, 7 insertions(+) > >diff --git a/lib/portage/versions=2Epy b/lib/portage/versions=2Epy >index 0c21373cc=2E=2Effec316ce 100644 >--- a/lib/portage/versions=2Epy >+++ b/lib/portage/versions=2Epy >@@ -312,6 +312,7 @@ def _pkgsplit(mypkg, eapi=3DNone): >=20 > _cat_re =3D re=2Ecompile('^%s$' % _cat, re=2EUNICODE) > _missing_cat =3D 'null' >+_catpkgsplit_cache =3D {} >=20 > def catpkgsplit(mydata, silent=3D1, eapi=3DNone): > """ >@@ -331,6 +332,11 @@ def catpkgsplit(mydata, silent=3D1, eapi=3DNone): > return mydata=2Ecpv_split > except AttributeError: > pass >+ >+ cache_entry =3D _catpkgsplit_cache=2Eget(mydata) >+ if cache_entry is not None: >+ return cache_entry >+ > mysplit =3D mydata=2Esplit('/', 1) > p_split =3D None > if len(mysplit) =3D=3D 1: >@@ -343,6 +349,7 @@ def catpkgsplit(mydata, silent=3D1, eapi=3DNone): > if not p_split: > return None > retval =3D (cat, p_split[0], p_split[1], p_split[2]) >+ _catpkgsplit_cache[mydata] =3D retval > return retval >=20 > class _pkg_str(_unicode): -- Best regards,=20 Micha=C5=82 G=C3=B3rny