public inbox for gentoo-dev@lists.gentoo.org
 help / color / mirror / Atom feed
From: "Michał Górny" <mgorny@gentoo.org>
To: gentoo-dev@lists.gentoo.org
Cc: "Michał Górny" <mgorny@gentoo.org>
Subject: [gentoo-dev] [PATCH 0/5] cargo.eclass: optimizations
Date: Fri, 16 Jun 2023 14:01:03 +0200	[thread overview]
Message-ID: <20230616120739.8656-1-mgorny@gentoo.org> (raw)

Hi,

cargo.eclass is one of the slowest eclasses in ::gentoo, and does some
pretty heavy processing in global scope.  Here's a series of patches
to optimize its behavior.

Unfortunately, the optimizations cannot be done without changing
the API.  However, the series preserves backwards compatibility --
existing ebuilds will simply remain slower.

The two main changes are:

1. Replacing subshell-based `$(cargo_crate_uris)` with global variable
   `${CARGO_CRATE_URIS}`.  This requires that `CRATES` and `GIT_CRATES`
   are defined prior to inherit -- this was already the case
   for the former, and only one case of `GIT_CRATES` does not conform.
   If they're not, then the variable will simply be empty but the old
   function will still work.

2. Supporting separate crate names from versions using a forward slash
   rather than a hyphen.  This makes it possible to parse the crate list
   using a simple cutting rather than regular expressions.  Again,
   existing syntax continues working but it remains much slower.

To compare, the initial timings for the eclass give, using the 52 CRATES
from dev-python/cryptography:

    real  252 it/s
    user  289 it/s

While the timings for fully optimized form (using variable and slashes)
give:

    real  952 it/s
    user  952 it/s


-- 
Best regards,
Michał Górny

Michał Górny (5):
  eclass/tests: Add a minimal benchmark for cargo.eclass
  cargo.eclass: Add variable alternative to $(cargo_crate_uris)
  cargo.eclass: Optimize GIT_CRATES check
  cargo.eclass: Support separating crate names/versions via slash
  cargo.eclass: Mark GIT_CRATES as pre-inherit

 eclass/cargo.eclass         | 127 ++++++++++++++++++++++--------------
 eclass/tests/cargo-bench.sh | 114 ++++++++++++++++++++++++++++++++
 2 files changed, 192 insertions(+), 49 deletions(-)
 create mode 100755 eclass/tests/cargo-bench.sh

-- 
2.41.0



             reply	other threads:[~2023-06-16 12:07 UTC|newest]

Thread overview: 9+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2023-06-16 12:01 Michał Górny [this message]
2023-06-16 12:01 ` [gentoo-dev] [PATCH 1/5] eclass/tests: Add a minimal benchmark for cargo.eclass Michał Górny
2023-06-16 12:01 ` [gentoo-dev] [PATCH 2/5] cargo.eclass: Add variable alternative to $(cargo_crate_uris) Michał Górny
2023-06-16 12:01 ` [gentoo-dev] [PATCH 3/5] cargo.eclass: Optimize GIT_CRATES check Michał Górny
2023-06-16 12:01 ` [gentoo-dev] [PATCH 4/5] cargo.eclass: Support separating crate names/versions via slash Michał Górny
2023-06-16 14:48   ` Denis Lisov
2023-06-16 15:52     ` Michał Górny
2023-06-16 18:06       ` Sam James
2023-06-16 12:01 ` [gentoo-dev] [PATCH 5/5] cargo.eclass: Mark GIT_CRATES as pre-inherit Michał Górny

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20230616120739.8656-1-mgorny@gentoo.org \
    --to=mgorny@gentoo.org \
    --cc=gentoo-dev@lists.gentoo.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox