From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from lists.gentoo.org (pigeon.gentoo.org [208.92.234.80]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by finch.gentoo.org (Postfix) with ESMTPS id D90C91382C5 for ; Sat, 27 Jun 2020 06:34:26 +0000 (UTC) Received: from pigeon.gentoo.org (localhost [127.0.0.1]) by pigeon.gentoo.org (Postfix) with SMTP id 6E32BE08DA; Sat, 27 Jun 2020 06:34:21 +0000 (UTC) Received: from mail-yb1-xb4a.google.com (mail-yb1-xb4a.google.com [IPv6:2607:f8b0:4864:20::b4a]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by pigeon.gentoo.org (Postfix) with ESMTPS id 65670E08DA for ; Sat, 27 Jun 2020 06:34:21 +0000 (UTC) Received: by mail-yb1-xb4a.google.com with SMTP id k127so7821247ybk.11 for ; Fri, 26 Jun 2020 23:34:21 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=Pd4Hrnn6Ox3AqCL6SxVQWHIXXhw7d40ulA4Jem48n2Q=; b=sQHDA5nyZEVxRmX/Szo6/UUp7zOdy8wUaL/5isAmOdx2f3uCYJJTRf0TS7Avyf3/8p qOWeMDPv3ni6D3A5FAUBzqK0ew73aUimBfdcqxqUZejRUiJYobkfDyahjdfYVLfO0GBd Y9PeWtwhU0FXl88zpqFmgQjB2yTb3Pm8+a3WXR9Z8spDy0xsiD5z3I9NT8tV/56gDuCM G1HBGtSbG8WaC/iSGCNYmsW31O3Vqiapl9BbKEDzBs7ugr8cP49FujYBZ85AmTgZcGmn vyXh4bqoeBwIKZzFy67KpE+xbTlxWeNfLeI8XeTDtiuFf/4xswsdntnv3b3XkxNZH1Jx eqGg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=Pd4Hrnn6Ox3AqCL6SxVQWHIXXhw7d40ulA4Jem48n2Q=; b=bI5BJWHna1EhGvojNQgSHQkLr/F0+eJAeZiuJ4ie8/qF1OM5S13xScJbcMjeUYLgPf NZpUNBltWdf7YGeg6RFn+H6jOdfh7luBB7POFXxyj0qUVdZKb6oz3wZOeloBNIVxBGUP Zvv8r6wuWHh5aCyUJg1N4o2GpC+PZsTr+vWwzlvn4xwl/+rV7cs8WHext9K5J+TbWE9h 97SpTN1h65kTJMFI+HLk9GvihBUqj5QGcXgjExaaZfcmOTqliOogQlYoTom9c8D25dwF 4zT9vrXvZ++1T5ClMsuwIOqpil3wDS52PuRmZ8AWxoHhw2mWAFvGUdvU2WiC8qomc3qa OPQQ== X-Gm-Message-State: AOAM530GL+iPXUjlWcziBb2Ja0yrcDDJFXQnWZMZ88hUM/9J0PJzMNlP 4+21o3J/Cj4V0E/d9eqUXoTE6BqCu02jX569cHi6Y8EnYt+AZUvqyVPkHJkKwYA/JFN9hHx5aUB LgiJSV4VFxlNG6s4ZLP9KvRqCUp1awrDpUP9TuIF2rC+ZWkrOJ4uT3PjQib7J1tvMz59Y649Fsc Lb X-Google-Smtp-Source: ABdhPJzXSijCau6rcCwYAzOKX3PWI+gUwGSamG3CuSuKXHuYgQS8s5W4pTXH8W9lECXNEYqpBIFSSs/VFQ== X-Received: by 2002:a25:7c46:: with SMTP id x67mr10811081ybc.279.1593239659650; Fri, 26 Jun 2020 23:34:19 -0700 (PDT) Date: Fri, 26 Jun 2020 23:34:13 -0700 In-Reply-To: <20200627063415.936177-1-cshei@google.com> Message-Id: <20200627063415.936177-2-cshei@google.com> Precedence: bulk List-Post: List-Help: List-Unsubscribe: List-Subscribe: List-Id: Gentoo Linux mail X-BeenThere: gentoo-portage-dev@lists.gentoo.org Reply-to: gentoo-portage-dev@lists.gentoo.org X-Auto-Response-Suppress: DR, RN, NRN, OOF, AutoReply Mime-Version: 1.0 References: <20200627063415.936177-1-cshei@google.com> X-Mailer: git-send-email 2.27.0.212.ge8ba1cc988-goog Subject: [gentoo-portage-dev] [PATCH 1/3] Add caching to catpkgsplit function From: Chun-Yu Shei To: gentoo-portage-dev@lists.gentoo.org Cc: Chun-Yu Shei Content-Type: text/plain; charset="UTF-8" X-Archives-Salt: d77efec8-6303-4cb4-9bf9-32d1dae6aa04 X-Archives-Hash: 17e8e51374aea1dd4e51e8f3622c5a47 According to cProfile, catpkgsplit is called up to 1-5.5 million times during "emerge -uDvpU --with-bdeps=y @world". Adding a dict to cache its results reduces the time for this command from 43.53 -> 41.53 seconds -- a 4.8% speedup. --- lib/portage/versions.py | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/lib/portage/versions.py b/lib/portage/versions.py index 0c21373cc..ffec316ce 100644 --- a/lib/portage/versions.py +++ b/lib/portage/versions.py @@ -312,6 +312,7 @@ def _pkgsplit(mypkg, eapi=None): _cat_re = re.compile('^%s$' % _cat, re.UNICODE) _missing_cat = 'null' +_catpkgsplit_cache = {} def catpkgsplit(mydata, silent=1, eapi=None): """ @@ -331,6 +332,11 @@ def catpkgsplit(mydata, silent=1, eapi=None): return mydata.cpv_split except AttributeError: pass + + cache_entry = _catpkgsplit_cache.get(mydata) + if cache_entry is not None: + return cache_entry + mysplit = mydata.split('/', 1) p_split = None if len(mysplit) == 1: @@ -343,6 +349,7 @@ def catpkgsplit(mydata, silent=1, eapi=None): if not p_split: return None retval = (cat, p_split[0], p_split[1], p_split[2]) + _catpkgsplit_cache[mydata] = retval return retval class _pkg_str(_unicode): -- 2.27.0.212.ge8ba1cc988-goog