From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from lists.gentoo.org (pigeon.gentoo.org [208.92.234.80]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by finch.gentoo.org (Postfix) with ESMTPS id C28A3138334 for ; Tue, 29 Oct 2019 14:56:47 +0000 (UTC) Received: from pigeon.gentoo.org (localhost [127.0.0.1]) by pigeon.gentoo.org (Postfix) with SMTP id 2B463E08DB; Tue, 29 Oct 2019 14:56:44 +0000 (UTC) Received: from smtp.gentoo.org (smtp.gentoo.org [140.211.166.183]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by pigeon.gentoo.org (Postfix) with ESMTPS id C2CA4E07A7 for ; Tue, 29 Oct 2019 14:56:43 +0000 (UTC) Received: from gentoo.org (unknown [IPv6:2001:980:3ff0:64:5054:ff:fe0b:7015]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) (Authenticated sender: grobian) by smtp.gentoo.org (Postfix) with ESMTPSA id F1F4334C535 for ; Tue, 29 Oct 2019 14:56:41 +0000 (UTC) Date: Tue, 29 Oct 2019 15:56:37 +0100 From: Fabian Groffen To: gentoo-dev@lists.gentoo.org Subject: Re: [gentoo-dev] New distfile mirror layout Message-ID: <20191029145637.GD22441@gentoo.org> Mail-Followup-To: gentoo-dev@lists.gentoo.org References: <20191029093401.GB22441@gentoo.org> <839D1F8E-35EE-43C9-B513-9895F087D77A@gentoo.org> <13d97d150c38407525e3461e8560ce7f6ed92f14.camel@gentoo.org> <20191029143304.GC22441@gentoo.org> <1e5681ee9c7efc9acadb9593b0a3645812b87651.camel@gentoo.org> Precedence: bulk List-Post: List-Help: List-Unsubscribe: List-Subscribe: List-Id: Gentoo Linux mail X-BeenThere: gentoo-dev@lists.gentoo.org Reply-to: gentoo-dev@lists.gentoo.org X-Auto-Response-Suppress: DR, RN, NRN, OOF, AutoReply MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="VV4b6MQE+OnNyhkM" Content-Disposition: inline In-Reply-To: <1e5681ee9c7efc9acadb9593b0a3645812b87651.camel@gentoo.org> User-Agent: Mutt/1.12.2 (SunOS 5.11, VIM - Vi IMproved 8.1) Organization: Gentoo Foundation, Inc. X-Archives-Salt: d67af17d-80f2-4b49-a272-34688fdba14e X-Archives-Hash: 262a84dbb7b2bbf6478005978ba63303 --VV4b6MQE+OnNyhkM Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On 29-10-2019 15:45:34 +0100, Micha=C5=82 G=C3=B3rny wrote: > On Tue, 2019-10-29 at 15:33 +0100, Fabian Groffen wrote: > > In addition, there are currently files there that aren't referenced from > > ebuilds. Prefix uses these files during bootstrap, local mirrors are > > often much faster than dev.g.o. > >=20 > > If the files don't get mirrored anymore, I guess I can create a dummy > > ebuild that has the files in SRC_URI. >=20 > Ok, this is something I wasn't aware of. I agree that dummy ebuild > should not be necessary here. However, I'm also not sure if distfiles- > local is really the proper way either, especially that I don't see such > files on woodpecker right now. There should be /space/distfiles-local and /space/distfiles-whitelist/prefix with a list of files to retain on the mirror. Thanks, Fabian > I don't think the matter is urgent right now, so let's ponder on it > a bit. In particular, I think we should have a clear indication of who > added which files, when, what for and where they came from. Those are > precisely the things that the current distfiles-local approach misses. >=20 > > If the files get mirrored, but put in a subdir based on the filename > > hash, the original query endpoint on distfiles.g.o changes, much like > > the SRC_URI approach. > >=20 > > Now I can use distfiles.prefix.b.n which redirects to the distfiles.g.o > > URL with subdir for most part I think, but it's sub-optimal from my > > point of view. Calculating the hash is not always feasible due to the > > lack of b2sum or other means. Hence my earlier request to have such > > official translation service on Gentoo hardware. > >=20 > > (I just wrote a small wsgi script that calculates the hash and generates > > the redirect from Python, served via uwsgi/nginx, but there should be > > many ways to achieve the same goals, if and only if a blake2b > > implementation were available for it.) >=20 > This is also something that needs thinking. I personally don't mind > having one but it would be nice if it was able to account for geodns > and such. >=20 > --=20 > Best regards, > Micha=C5=82 G=C3=B3rny >=20 --=20 Fabian Groffen Gentoo on a different level --VV4b6MQE+OnNyhkM Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQEzBAABCgAdFiEELUvHd/Gtp7LaU1vuzpXahU5EQpMFAl24UyUACgkQzpXahU5E QpO9Ogf/Qj4D9Y3sW/lfvOk9ixOpAEzxHkkD7KzN6gBu5reh0tiiMrGAqpI6anb8 TMs7NDLgl9KJPk5ST1mv/M1HE8zHhQ82n3ZNPcuLfhSBrjGW9tFIcYSDKPlzORWM G7xx7GvdpdHpMEJnRjm2tUkb5jfEHL82rzzSLUT+iuJgFIGSn9zDCJWEWMqidc1u KZWq4aaVDKcgcOmm7ybWRyKiywgoGQqB2/OvD/sOGTsLx9Rn8YajiIbwvDiDzO1O MLQJpBDHYhSAQZpx85ZtpfkeBxRCbffybxJkhwhrNFpNkjAao37YMbhkuY3G4kgR x2lBg3TGoOKnrWsAM8Uk4ZAZVsnVKw== =znIU -----END PGP SIGNATURE----- --VV4b6MQE+OnNyhkM--