From: "Robin H. Johnson" <robbat2@gentoo.org>
To: gentoo-mirrors@lists.gentoo.org
Subject: [gentoo-mirrors] Mirror survey re rsync --hard-link (distfiles/releases/experimental/snapshots)
Date: Sat, 27 Jan 2018 00:47:39 +0000 [thread overview]
Message-ID: <robbat2-20180127T002332-556742341Z@orbis-terrarum.net> (raw)
[-- Attachment #1: Type: text/plain, Size: 1643 bytes --]
Hi,
TL;DR: is your mirror fetching with --hard-link/-H? Would adding it be
problematic for you? Speak up to save yourself bandwidth!
The infrastructure team is having some discussions [1] about splitting apart
the very large distfiles directory, as it now contains ~75000 files, and
this leads some issues on some file-systems.
The distfiles directory is around ~240GB presently, and we would like to
have a reasonable transition period for users, without temporarily
doubling the storage requirement on mirrors.
The best means for this is going to be hard-links, but I don't know how
many mirrors still have that enabled.
None of the rsync setup documentation has ever included it (either right
now or historically). It was strongly encouraged in this mailing list
back in 2004/2005, as part of the new release staging process, but the
most recent mention of hard-linking on the mailing list was Feb 2009.
Ideally, the mirrors that offer rsync service to users for distfiles
would ALSO offer hard-link fetching to users, but that isn't strictly
needed at this point.
To this end, I'm like to know
1. How many mirrors are already fetching from rsync with the
--hard-link/-H option.
2. How many mirrors would NOT be able to enable rsync fetching with
hard-links.
3. Does your mirror offer or explicitly reject hard-link data to users?
[1] https://bugs.gentoo.org/534528
--
Robin Hugh Johnson
Gentoo Linux: Dev, Infra Lead, Foundation Treasurer
E-Mail : robbat2@gentoo.org
GnuPG FP : 11ACBA4F 4778E3F6 E4EDF38E B27B944E 34884E85
GnuPG FP : 7D0B3CEB E9B85B1F 825BCECF EE05E6F6 A48F6136
[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 1113 bytes --]
next reply other threads:[~2018-01-27 0:47 UTC|newest]
Thread overview: 10+ messages / expand[flat|nested] mbox.gz Atom feed top
2018-01-27 0:47 Robin H. Johnson [this message]
2018-01-27 12:28 ` [gentoo-mirrors] Mirror survey re rsync --hard-link (distfiles/releases/experimental/snapshots) Erwin Bronkhorst - Studenten Net Twente
2018-01-27 13:18 ` Alarig Le Lay
2018-01-29 7:54 ` Robin H. Johnson
2018-01-29 8:32 ` SoEasyTo Mirrors Manager
2018-01-28 12:08 ` Carlos Carvalho
2018-01-30 8:08 ` Adrian Sándor
2018-01-30 9:45 ` Alarig Le Lay
2018-01-30 18:24 ` Robin H. Johnson
2018-01-30 18:32 ` Alarig Le Lay
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=robbat2-20180127T002332-556742341Z@orbis-terrarum.net \
--to=robbat2@gentoo.org \
--cc=gentoo-mirrors@lists.gentoo.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox