From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from lists.gentoo.org (pigeon.gentoo.org [208.92.234.80]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by finch.gentoo.org (Postfix) with ESMTPS id D6A781382C5 for ; Sat, 27 Jan 2018 11:37:03 +0000 (UTC) Received: from pigeon.gentoo.org (localhost [127.0.0.1]) by pigeon.gentoo.org (Postfix) with SMTP id B8012E0916; Sat, 27 Jan 2018 11:36:56 +0000 (UTC) Received: from smarthost01c.mail.zen.net.uk (smarthost01c.mail.zen.net.uk [212.23.1.5]) (using TLSv1.2 with cipher DHE-RSA-AES128-SHA256 (128/128 bits)) (No client certificate requested) by pigeon.gentoo.org (Postfix) with ESMTPS id 5D38CE08EB for ; Sat, 27 Jan 2018 11:36:55 +0000 (UTC) Received: from [62.3.120.142] (helo=NeddySeagoon_Static) by smarthost01c.mail.zen.net.uk with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.80) (envelope-from ) id 1efOn0-0001uX-Bg for gentoo-dev@lists.gentoo.org; Sat, 27 Jan 2018 11:36:54 +0000 Date: Sat, 27 Jan 2018 11:36:34 +0000 From: Roy Bamford Subject: Re: [gentoo-dev] [pre-GLEP] Split distfile mirror directory structure To: gentoo-dev@lists.gentoo.org In-Reply-To: <1517041828.3816.3.camel@gentoo.org> (from mgorny@gentoo.org on Sat Jan 27 08:30:28 2018) X-Mailer: Balsa 2.5.3 Message-Id: Precedence: bulk List-Post: List-Help: List-Unsubscribe: List-Subscribe: List-Id: Gentoo Linux mail X-BeenThere: gentoo-dev@lists.gentoo.org Reply-to: gentoo-dev@lists.gentoo.org MIME-Version: 1.0 Content-Type: multipart/signed; micalg=PGP-SHA256; protocol="application/pgp-signature"; boundary="=-Rc7ueeH9c8aP2A8Bbw85" X-Originating-smarthost01c-IP: [62.3.120.142] Feedback-ID: 62.3.120.142 X-Archives-Salt: ba078d91-160c-44da-9fee-940738c042a3 X-Archives-Hash: a51359ed80d8a113e7af598878a48552 --=-Rc7ueeH9c8aP2A8Bbw85 Content-Type: text/plain; charset=UTF-8 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On 2018.01.27 08:30, Micha=C5=82 G=C3=B3rny wrote: > W dniu pi=C4=85, 26.01.2018 o godzinie 20=E2=88=B648=E2=80=89-0500, u=C5= =BCytkownik Michael > Orlitzky napisa=C5=82: > > On 01/26/2018 06:24 PM, Micha=C5=82 G=C3=B3rny wrote: > > >=20 > > > The alternate option of using file hash has the advantage of > having > > > a more balanced split. Furthermore, since hashes are stored > > > in Manifests using them is zero-cost. However, this solution has > two > > > significant disadvantages: > > >=20 > > > 1. The hash values are unknown for newly-downloaded distfiles, so > > > ``repoman`` (or an equivalent tool) would have to use a > temporary > > > directory before locating the file in appropriate subdirectory. > > >=20 > > > 2. User-provided distfiles (e.g. for fetch-restricted packages) > with > > > hash mismatches would be placed in the wrong subdirectory, > > > potentially causing confusing errors. > > >=20 > >=20 > > The filename proposal sounds fine, so this is only academic, but: > are > > these two points really disadvantages? > >=20 > > What are we worried about in using a temporary directory? Copying > across > > filesystem boundaries? Except in rare cases, $DISTDIR itself will be > > usable a temporary location (on the same filesystem), won't it? >=20 > Why add the extra complexity when there's no need for one? Note that > there's also the problem of resuming transfers, so in the end we're > talking about permanent temporary directory where we keep unfinished > transfers. >=20 > > For the second point, portage is going to tell me where to put the > file, > > isn't it? Then no matter what garbage I download, won't portage look > for > > it in the right place, because where-to-put-it is determined using > the > > same manifest hash that determines where-to-find-it? >=20 > No, it won't. Why would it? You're going to call something like: >=20 > edistadd foo.tar.gz bar.tar.gz >=20 > ...and it will place the files in the right subdirectories. >=20 > --=20 > Best regards, > Micha=C5=82 G=C3=B3rny >=20 >=20 >=20 >=20 Micha=C5=82, How does this work for fetch restricted files and finding other files no longer on the mirrors? Its no longer a download and move it to $DISTFILES, or is it? Whatever it is, users will need to do it unless files in $DISTFILES are accepted by package managers if they are not found in the main=20 structure. --=20 Regards, Roy Bamford (Neddyseagoon) a member of elections gentoo-ops forum-mods = --=-Rc7ueeH9c8aP2A8Bbw85 Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- iQIzBAABCAAdFiEE8iqN0lnEsghF0U/tVl/TNarc5wkFAlpsZEIACgkQVl/TNarc 5wlWNQ//YnQZDczFrmyhCLQZ/zX9x8nzIRgs+aqmgC87myz1nE0FWcdo2gSPcdNM iyDlufkty7e8cEsIeWfYBoQF7wCfRYwObRv0vYZqgmRQFmK1KHqjv1HGaOUoOjpe BIeRDR6bt9N56uxajq7Kc3iOWodw7NIGE5dn3/n4Dntu4wzjIzDtDkGwfgePXWsM PZ3hJHaI7uM0ybF4OWEelUQocyfczaic/vPpeNg517mZdtjaUKtsQmaBm36V3SFa G0rlmOw8wJIOlzFUGJazkb6k6iGaBUYsUscYm7Dx/4vEUPsUrvZWVV76FPP4qKYY 8oYinMnfTT0W7YeXJnT7n2K4n2JCeDAuOsCrDhENE5Wx90gqpWqdCil2NLhZfaUx dgjwM5I2Bz+c4SdCQ849858aJ2Zi9mMnFvHJqFiJzKkE1RMf/D0uZJtudjkEWRig PsWf3u2rimkdB0tZyegg4e+7/79OApX56TkCITe8KzhW7uGEfSVeyvW6CQBRgQ+i a+hjjDToQno1RJ3kBKRcRd3bv/wWR19tcmPqU8KWXzMCjd5n31aPV5QDsttE8o1Y tr2NVlIWF/dmQQDVlotfQzvHU4m3ngfc2I+VuOOJvIp2yEnoqmzZx3WalXOaP5Uv 5MBTAdEYuaRKG2LoF5++m8kecNWYz4ae8GEfFm9kfDD4eXaLPyM= =HLoU -----END PGP SIGNATURE----- --=-Rc7ueeH9c8aP2A8Bbw85--