Re: [gentoo-dev] [pre-GLEP] Split distfile mirror directory structure

[Andrew Barchuk <andrew@raindev.io>]

> To the contrary, that would not remain balanced, because your
> boundaries are entirely dependent on exactly what is in the tree at
> the moment you run your script. Now the package manager has to perform
> directory listing, sort and find the file name that's closest, open
> that directory, find the next closest filename (assuming multiple
> levels of hierarchy), and so on, or you have to store yet another
> index that duplicates information and takes additional space. Locating
> by distfile name hash is effectively free.

Sure, the tree won't be perfectly balanced but it will be pretty close.
E.g. if texlive-* dominates the tree today it will likely continue
dominating it for another 5 years. Statistical distribution of distfile
names will likely be changing very slowly.

Doing a binary search through a list of couple of hundred of directories
is really cheap. I don't see a reason to organize distfiles in a
multi-level hierarchy: e.g. if the goal is to keep no more than 1000
files in a folder than the limit of single level hierarchy is a million
which is more than enough for foreseeable future. The list of 500
directories takes 15kB when using full file names and will be couple of
times smaller when using only unique prefixes.

---
Andrew