public inbox for gentoo-dev@lists.gentoo.org
 help / color / mirror / Atom feed
* [gentoo-dev] New distfile mirror layout
@ 2019-10-18 13:41 Michał Górny
  2019-10-18 19:53 ` Richard Yao
                   ` (2 more replies)
  0 siblings, 3 replies; 56+ messages in thread
From: Michał Górny @ 2019-10-18 13:41 UTC (permalink / raw
  To: gentoo-dev-announce; +Cc: gentoo-dev

[-- Attachment #1: Type: text/plain, Size: 1884 bytes --]

Hi, everybody.

It is my pleasure to announce that yesterday (EU) evening we've switched
to a new distfile mirror layout.  Users will be switching to the new
layout either as they upgrade Portage to 2.3.77 or -- if they upgraded
already -- as their caches expire (24hrs).

The new layout is mostly a bow towards mirror admins, for some of whom
having a 60000+ files in a single directory have been a problem. 
However, I suppose some of you also found e.g. the directory index
hardly usable due to its size.

Throughout a transitional period (whose exact length hasn't been decided
yet), both layouts will be available.  Afterwards, the old layout will
be removed from mirrors.  This has a few implications:

1. Users who don't upgrade their package managers in time will lose
the ability of fetching from Gentoo mirrors.  This shouldn't be that
much of a problem given that the core software needed to upgrade Portage
should all have reliable upstream SRC_URIs.

2. mirror://gentoo/file URIs will stop working.  While technically you
could use mirror://gentoo/XX/file, I'd rather recommend finally
discarding its usage and moving distfiles to devspace.

3. Directly fetching files from distfiles.gentoo.org will become
a little harder.  To fetch a distfile named 'foo-1.tar.gz', you'd have
to use something like:

$ printf '%s' foo-1.tar.gz | b2sum | cut -c1-2
1b
$ wget http://distfiles.gentoo.org/distfiles/1b/foo-1.tar.gz
...


Alternatively, you can:

$ wget http://distfiles.gentoo.org/distfiles/INDEX

and grep for the right path there.  This INDEX is also a more
lightweight alternative to HTML indexes generated by the servers.


If you're interested in more background details and some plots, see [1].

[1] https://dev.gentoo.org/~mgorny/articles/improving-distfile-mirror-structure.html

-- 
Best regards,
Michał Górny


[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 618 bytes --]

^ permalink raw reply	[flat|nested] 56+ messages in thread

end of thread, other threads:[~2019-10-29 16:35 UTC | newest]

Thread overview: 56+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2019-10-18 13:41 [gentoo-dev] New distfile mirror layout Michał Górny
2019-10-18 19:53 ` Richard Yao
2019-10-18 20:49   ` Michał Górny
2019-10-19  1:09     ` Richard Yao
2019-10-19  6:17       ` Michał Górny
2019-10-19  8:20         ` Richard Yao
2019-10-19 19:26       ` Richard Yao
2019-10-19 20:02         ` Michał Górny
2019-10-19 22:48           ` Richard Yao
2019-10-22  0:46   ` James Cloos
2019-10-19 13:31 ` Fabian Groffen
2019-10-19 13:53   ` Michał Górny
2019-10-19 23:24 ` Joshua Kinard
2019-10-19 23:57   ` Alec Warner
2019-10-20  0:14     ` Joshua Kinard
2019-10-20  6:51   ` Michał Górny
2019-10-20  8:25     ` Joshua Kinard
2019-10-20  8:32       ` Michał Górny
2019-10-20  9:21         ` Joshua Kinard
2019-10-20  9:44           ` Michał Górny
2019-10-20 20:57             ` Joshua Kinard
2019-10-21  0:05               ` Joshua Kinard
2019-10-21  5:51                 ` Ulrich Mueller
2019-10-21 10:17                 ` Kent Fredric
2019-10-21 21:34                 ` Mikle Kolyada
2019-10-21 10:13               ` Kent Fredric
2019-10-23  5:16                 ` Joshua Kinard
2019-10-29 16:35                   ` Kent Fredric
2019-10-20 17:09       ` Matt Turner
2019-10-21 16:42     ` Richard Yao
2019-10-21 23:36       ` Matt Turner
2019-10-23  5:18         ` Joshua Kinard
2019-10-23 17:06           ` William Hubbs
2019-10-23 18:38             ` William Hubbs
2019-10-23 22:04           ` William Hubbs
2019-10-24  4:30             ` Michał Górny
2019-10-22  6:51       ` Jaco Kroon
2019-10-22  8:43         ` Ulrich Mueller
2019-10-22  8:46           ` Jaco Kroon
2019-10-23 23:47         ` ext4 readdir performance - was " Richard Yao
2019-10-24  0:01           ` Richard Yao
2019-10-23  1:21       ` Rich Freeman
2019-10-28 23:24     ` Chí-Thanh Christopher Nguyễn
2019-10-29  4:27       ` Michał Górny
2019-10-29  9:34         ` Fabian Groffen
2019-10-29 11:11           ` Michał Górny
2019-10-29 12:23             ` Ulrich Mueller
2019-10-29 12:43               ` Michał Górny
2019-10-29 13:03                 ` Ulrich Mueller
2019-10-29 13:09                   ` Ulrich Mueller
2019-10-29 13:52                     ` Michał Górny
2019-10-29 14:17                       ` Ulrich Mueller
2019-10-29 14:33                         ` Fabian Groffen
2019-10-29 14:45                           ` Michał Górny
2019-10-29 14:56                             ` Fabian Groffen
2019-10-29 13:51                   ` Michał Górny

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox