From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from lists.gentoo.org (pigeon.gentoo.org [208.92.234.80]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by finch.gentoo.org (Postfix) with ESMTPS id A7318138334 for ; Tue, 29 Oct 2019 14:45:44 +0000 (UTC) Received: from pigeon.gentoo.org (localhost [127.0.0.1]) by pigeon.gentoo.org (Postfix) with SMTP id E4DE0E0936; Tue, 29 Oct 2019 14:45:40 +0000 (UTC) Received: from smtp.gentoo.org (smtp.gentoo.org [140.211.166.183]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by pigeon.gentoo.org (Postfix) with ESMTPS id 82FE3E092D for ; Tue, 29 Oct 2019 14:45:40 +0000 (UTC) Received: from pomiot (c134-66.icpnet.pl [85.221.134.66]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) (Authenticated sender: mgorny) by smtp.gentoo.org (Postfix) with ESMTPSA id A219A34C52B; Tue, 29 Oct 2019 14:45:38 +0000 (UTC) Message-ID: <1e5681ee9c7efc9acadb9593b0a3645812b87651.camel@gentoo.org> Subject: Re: [gentoo-dev] New distfile mirror layout From: =?UTF-8?Q?Micha=C5=82_G=C3=B3rny?= To: gentoo-dev@lists.gentoo.org Date: Tue, 29 Oct 2019 15:45:34 +0100 In-Reply-To: <20191029143304.GC22441@gentoo.org> References: <20191029093401.GB22441@gentoo.org> <839D1F8E-35EE-43C9-B513-9895F087D77A@gentoo.org> <13d97d150c38407525e3461e8560ce7f6ed92f14.camel@gentoo.org> <20191029143304.GC22441@gentoo.org> Organization: Gentoo Content-Type: multipart/signed; micalg="pgp-sha512"; protocol="application/pgp-signature"; boundary="=-nlzwb+AcTrppr38yFXAa" User-Agent: Evolution 3.32.4 Precedence: bulk List-Post: List-Help: List-Unsubscribe: List-Subscribe: List-Id: Gentoo Linux mail X-BeenThere: gentoo-dev@lists.gentoo.org Reply-to: gentoo-dev@lists.gentoo.org X-Auto-Response-Suppress: DR, RN, NRN, OOF, AutoReply MIME-Version: 1.0 X-Archives-Salt: 084cff24-edfe-4e0a-9fae-6480807adc5f X-Archives-Hash: 86e4601aa70e466834b341676921fce1 --=-nlzwb+AcTrppr38yFXAa Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable On Tue, 2019-10-29 at 15:33 +0100, Fabian Groffen wrote: > On 29-10-2019 15:17:38 +0100, Ulrich Mueller wrote: > > > > > > > On Tue, 29 Oct 2019, Micha=C5=82 G=C3=B3rny wrote: > > > On Tue, 2019-10-29 at 14:09 +0100, Ulrich Mueller wrote: > > > > > What if the file is hosted at a non-standard tcp port upstream > > > > > (like http://example.org:8080/)? The devmanual says that it _must= _ > > > > > be manually uploaded to /space/distfiles-local/ in such cases. > > > > Or another example, app-emacs/vhdl-mode-3.38.1, where (incompetent, > > > > or nasty?) upstream blocks wget for some reason, but other methods > > > > (e.g., curl, firefox) work? How would I get the file onto the mirro= rs > > > > there? > > > If I were you, I would've explicitly mirrored the file anyway. > > > If upstream blocks wget, then users who do not use GENTOO_MIRRORS wil= l > > > also suffer due to it. > >=20 > > All what I'm saying is that there can be unusual circumstances where > > manual uploading of a file is useful. So please don't take that > > possibility away. >=20 > In addition, there are currently files there that aren't referenced from > ebuilds. Prefix uses these files during bootstrap, local mirrors are > often much faster than dev.g.o. >=20 > If the files don't get mirrored anymore, I guess I can create a dummy > ebuild that has the files in SRC_URI. Ok, this is something I wasn't aware of. I agree that dummy ebuild should not be necessary here. However, I'm also not sure if distfiles- local is really the proper way either, especially that I don't see such files on woodpecker right now. I don't think the matter is urgent right now, so let's ponder on it a bit. In particular, I think we should have a clear indication of who added which files, when, what for and where they came from. Those are precisely the things that the current distfiles-local approach misses. > If the files get mirrored, but put in a subdir based on the filename > hash, the original query endpoint on distfiles.g.o changes, much like > the SRC_URI approach. >=20 > Now I can use distfiles.prefix.b.n which redirects to the distfiles.g.o > URL with subdir for most part I think, but it's sub-optimal from my > point of view. Calculating the hash is not always feasible due to the > lack of b2sum or other means. Hence my earlier request to have such > official translation service on Gentoo hardware. >=20 > (I just wrote a small wsgi script that calculates the hash and generates > the redirect from Python, served via uwsgi/nginx, but there should be > many ways to achieve the same goals, if and only if a blake2b > implementation were available for it.) >=20 This is also something that needs thinking. I personally don't mind having one but it would be nice if it was able to account for geodns and such. --=20 Best regards, Micha=C5=82 G=C3=B3rny --=-nlzwb+AcTrppr38yFXAa Content-Type: application/pgp-signature; name="signature.asc" Content-Description: This is a digitally signed message part Content-Transfer-Encoding: 7bit -----BEGIN PGP SIGNATURE----- iQGTBAABCgB9FiEEx2qEUJQJjSjMiybFY5ra4jKeJA4FAl24UI5fFIAAAAAALgAo aXNzdWVyLWZwckBub3RhdGlvbnMub3BlbnBncC5maWZ0aGhvcnNlbWFuLm5ldEM3 NkE4NDUwOTQwOThEMjhDQzhCMjZDNTYzOUFEQUUyMzI5RTI0MEUACgkQY5ra4jKe JA4ScQgAiV9yIQUy3M6vqh2QMLk8XPfAclKNInx/4xwhTJJ2i6r2A4eDCkDO/eM5 mGPmtJSsuiO841fxqKVNqSzx4fUDpX4twDtTBy3hzEvW/kMreXD1u6pXCkPkimtY SZ8Pvfl3jREqteDHzXzq9KVMt+wlWVWGecIHhIokLjQuKJBf9kShr5bpizRQg6MH xDh1sILl3eF1Adbu0LpIgHxDKPwp2gKwavc9WlUAq14UsNACwbSCjGuPc6CnnovK Y86kSidivUThZyARtMphLZYdyBgEEw1nivujdjzgHaBduxlNPAJbhA56G0l+jopC uOgJ18YNZL1uYRZxeDohkuaHHG1Yow== =tiMJ -----END PGP SIGNATURE----- --=-nlzwb+AcTrppr38yFXAa--