From: Gordon Pettey <petteyg359@gmail.com>
To: gentoo-dev@lists.gentoo.org
Subject: Re: [gentoo-dev] [pre-GLEP] Split distfile mirror directory structure
Date: Sun, 28 Jan 2018 15:17:10 -0600 [thread overview]
Message-ID: <CAHY5Med=UkgdNy4qFwv8dfM2VakTA3PGexNbHJ8pTn0dj+hVLg@mail.gmail.com> (raw)
In-Reply-To: <1517172228.2114973.1251027256.0A9C8F3C@webmail.messagingengine.com>
On Sun, Jan 28, 2018 at 2:43 PM, Andrew Barchuk <andrew@raindev.io> wrote:
> There's another option to use character ranges for each directory
> computed in a way to have the files distributed evenly. One way to do
> that is to use filename prefix of dynamic length so that each range
> holds the same number of files. E.g. we would have Ab/, Ap/, Ar/ but
> texlive-module-te/, texlive-module-th/, texlive-module-ti/. A similar
> but simpler option is to use file names as range bounds (the same way
> dictionaries use words to demarcate page bounds): each directory will
> have a name of the first file located inside. This way files will be
> distributed evenly and it's still easy to pick a correct directory where
> a file will be located manually.
>
> ...snip...
>
> Using the approach above the files will distributed evenly among the
> directories keeping the possibility to determine the directory for a
> specific file by hand. It's possible if necessary to keep the directory
> structure unchanged for very long time and it will likely stay
> well-balanced. Picking a directory for a file is very cheap. The only
> obvious downside I see is that it's necessary to know list of
> directories to pick the correct one (can be mitigated by caching the
> list of directories if important). If it's desirable to make directory
> names shorter or to look less like file names it's fairly easy to
> achieve by keeping only unique prefixes of directories. For example:
To the contrary, that would not remain balanced, because your
boundaries are entirely dependent on exactly what is in the tree at
the moment you run your script. Now the package manager has to perform
directory listing, sort and find the file name that's closest, open
that directory, find the next closest filename (assuming multiple
levels of hierarchy), and so on, or you have to store yet another
index that duplicates information and takes additional space. Locating
by distfile name hash is effectively free.
next prev parent reply other threads:[~2018-01-28 21:18 UTC|newest]
Thread overview: 44+ messages / expand[flat|nested] mbox.gz Atom feed top
2018-01-26 23:24 [gentoo-dev] [pre-GLEP] Split distfile mirror directory structure Michał Górny
2018-01-27 1:48 ` Michael Orlitzky
2018-01-27 2:44 ` R0b0t1
2018-01-27 8:30 ` Michał Górny
2018-01-27 11:36 ` Roy Bamford
2018-01-27 11:41 ` Michał Górny
2018-01-27 16:42 ` Gordon Pettey
2018-01-27 16:48 ` Michael Orlitzky
2018-01-27 19:01 ` Gordon Pettey
2018-01-27 20:16 ` Michael Orlitzky
2018-01-30 1:21 ` Kent Fredric
2018-01-30 2:53 ` Robin H. Johnson
2018-01-30 7:25 ` Michał Górny
2018-01-30 19:46 ` Kent Fredric
2018-01-27 16:47 ` Michael Orlitzky
2018-01-27 18:14 ` Michał Górny
2018-01-27 18:24 ` Michael Orlitzky
2018-01-27 19:47 ` Michał Górny
2018-01-27 20:30 ` Michael Orlitzky
2018-01-30 1:27 ` Kent Fredric
2018-01-30 7:17 ` Ulrich Mueller
2018-01-28 7:01 ` Jason Zaman
2018-01-28 9:10 ` Michał Górny
2018-01-29 7:33 ` Robin H. Johnson
2018-01-28 10:14 ` Ulrich Mueller
2018-01-28 10:16 ` Michał Górny
2018-01-28 10:22 ` Ulrich Mueller
2018-01-28 10:40 ` Michał Górny
2018-01-28 13:03 ` Ulrich Mueller
2018-01-30 1:41 ` Kent Fredric
2018-01-30 7:11 ` Ulrich Mueller
2018-01-28 20:43 ` Andrew Barchuk
2018-01-28 21:17 ` Gordon Pettey [this message]
2018-01-28 22:00 ` Andrew Barchuk
2018-01-28 22:13 ` Gordon Pettey
2018-01-28 22:14 ` Zac Medico
2018-01-28 22:46 ` Andrew Barchuk
2018-01-29 5:36 ` Michał Górny
2018-01-29 9:22 ` Andrew Barchuk
2018-01-29 19:37 ` [gentoo-dev] [pre-GLEP] Split distfile mirror directory structure (draft v2) Michał Górny
2018-01-29 20:00 ` Robin H. Johnson
2018-01-29 21:09 ` Michał Górny
2018-01-29 20:26 ` R0b0t1
2018-01-29 20:55 ` Alec Warner
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to='CAHY5Med=UkgdNy4qFwv8dfM2VakTA3PGexNbHJ8pTn0dj+hVLg@mail.gmail.com' \
--to=petteyg359@gmail.com \
--cc=gentoo-dev@lists.gentoo.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox