From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from smtp.gentoo.org (woodpecker.gentoo.org [140.211.166.183]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits)) (No client certificate requested) by finch.gentoo.org (Postfix) with ESMTPS id F11CC1581EE for ; Sun, 30 Mar 2025 09:35:14 +0000 (UTC) Received: from lists.gentoo.org (bobolink.gentoo.org [140.211.166.189]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits)) (No client certificate requested) (Authenticated sender: relay-lists.gentoo.org@gentoo.org) by smtp.gentoo.org (Postfix) with ESMTPSA id D610E3431BE for ; Sun, 30 Mar 2025 09:35:14 +0000 (UTC) Received: from bobolink.gentoo.org (localhost [127.0.0.1]) by bobolink.gentoo.org (Postfix) with ESMTP id 7AA9D1104B9; Sun, 30 Mar 2025 09:34:31 +0000 (UTC) Received: from smtp.gentoo.org (woodpecker.gentoo.org [140.211.166.183]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits)) (No client certificate requested) by bobolink.gentoo.org (Postfix) with ESMTPS id B01C91104B0 for ; Sun, 30 Mar 2025 09:34:30 +0000 (UTC) Received: from localhost (unknown [IPv6:2600:3c00::f03c:92ff:fe12:74e1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) (Authenticated sender: radhermit) by smtp.gentoo.org (Postfix) with ESMTPSA id 5A4AF343162 for ; Sun, 30 Mar 2025 09:34:30 +0000 (UTC) Date: Sun, 30 Mar 2025 03:34:28 -0600 From: Tim Harder To: gentoo-dev@lists.gentoo.org Subject: Re: [gentoo-dev] The uncertain future of repository mirrors Message-ID: Mail-Followup-To: gentoo-dev@lists.gentoo.org References: <6b358608f6e244cb96ce527ad47b3e0483eaf0c6.camel@gentoo.org> <87b23159542cbdeac7970756b83db946@levelnine.at> Precedence: bulk List-Post: List-Help: List-Unsubscribe: List-Subscribe: List-Id: Gentoo Linux mail X-BeenThere: gentoo-dev@lists.gentoo.org Reply-to: gentoo-dev@lists.gentoo.org X-Auto-Response-Suppress: DR, RN, NRN, OOF, AutoReply MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8; format=flowed Content-Disposition: inline In-Reply-To: <87b23159542cbdeac7970756b83db946@levelnine.at> X-Archives-Salt: 8bd78773-a91f-4670-a58d-5210e6bf3cc0 X-Archives-Hash: 34c784640d334a47561856903eef9d51 On 2025-03-30 Sun 02:11, Michael Mair-Keimberger wrote: >I guess i'm a bit late in this discussion but i wanted to let you know >this would also affect my gentoo qa scripts. >(https://gentooqa.levelnine.at). Right now i'm checking the gentoo, >guru, kde, science and pentoo repositories, syncing the repos from >gentoo-mirror for the pre-created metadata. >While the gentoo repository is probably the most valuable, kde, >science and pentoo checks would be non functional and i probably would >have to remove them. (at least until i update my script to create the >metadata myself..) Just to point out, generating metadata can be relatively quick these days for large repos using non-portage tools. Anyone using egencache or `emerge --regen` from portage (especially for new repos without any metadata) is wasting a lot of CPU cycles and memory mainly due to portage's inefficient design executing a new instance of bash per ebuild. Alternatives would be `pmaint regen` from pkgcore (roughly 4x faster than portage) and `pk repo metadata regen` from pkgcraft-tools (roughly 10x faster than portage while using 10x less memory than pkgcore). On semi-decent desktop hardware a full gentoo repo metadata run using pkgcraft takes ~10-20 seconds and less than a second for full verification only. For those interested in more design discussion and benchmarks see [1]. Tim [1]: https://pkgcraft.github.io/posts/metadata-cache-generation/