* Re: [gentoo-dev] [pre-GLEP] Split distfile mirror directory structure
@ 2018-01-28 21:17 99% ` Gordon Pettey
0 siblings, 0 replies; 1+ results
From: Gordon Pettey @ 2018-01-28 21:17 UTC (permalink / raw
To: gentoo-dev
On Sun, Jan 28, 2018 at 2:43 PM, Andrew Barchuk <andrew@raindev.io> wrote:
> There's another option to use character ranges for each directory
> computed in a way to have the files distributed evenly. One way to do
> that is to use filename prefix of dynamic length so that each range
> holds the same number of files. E.g. we would have Ab/, Ap/, Ar/ but
> texlive-module-te/, texlive-module-th/, texlive-module-ti/. A similar
> but simpler option is to use file names as range bounds (the same way
> dictionaries use words to demarcate page bounds): each directory will
> have a name of the first file located inside. This way files will be
> distributed evenly and it's still easy to pick a correct directory where
> a file will be located manually.
>
> ...snip...
>
> Using the approach above the files will distributed evenly among the
> directories keeping the possibility to determine the directory for a
> specific file by hand. It's possible if necessary to keep the directory
> structure unchanged for very long time and it will likely stay
> well-balanced. Picking a directory for a file is very cheap. The only
> obvious downside I see is that it's necessary to know list of
> directories to pick the correct one (can be mitigated by caching the
> list of directories if important). If it's desirable to make directory
> names shorter or to look less like file names it's fairly easy to
> achieve by keeping only unique prefixes of directories. For example:
To the contrary, that would not remain balanced, because your
boundaries are entirely dependent on exactly what is in the tree at
the moment you run your script. Now the package manager has to perform
directory listing, sort and find the file name that's closest, open
that directory, find the next closest filename (assuming multiple
levels of hierarchy), and so on, or you have to store yet another
index that duplicates information and takes additional space. Locating
by distfile name hash is effectively free.
^ permalink raw reply [relevance 99%]
Results 1-1 of 1 | reverse | options above
-- pct% links below jump to the message on this page, permalinks otherwise --
2018-01-26 23:24 [gentoo-dev] [pre-GLEP] Split distfile mirror directory structure Michał Górny
2018-01-28 20:43 ` Andrew Barchuk
2018-01-28 21:17 99% ` Gordon Pettey
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox