From: "Xavier Neys" <neysx@lark.gentoo.org>
To: gentoo-doc-cvs@lists.gentoo.org
Subject: [gentoo-doc-cvs] cvs commit: rsync.xml
Date: Fri, 12 May 2006 20:07:08 +0000 [thread overview]
Message-ID: <20060512200708.56DAD6475F@smtp.gentoo.org> (raw)
neysx 06/05/12 20:07:07
Modified: rsync.xml
Log:
Fixed utterly broken coding style and a couple of typos
Revision Changes Path
1.49 xml/htdocs/doc/en/rsync.xml
file : http://www.gentoo.org/cgi-bin/viewcvs.cgi/xml/htdocs/doc/en/rsync.xml?rev=1.49&content-type=text/x-cvsweb-markup&cvsroot=gentoo
plain: http://www.gentoo.org/cgi-bin/viewcvs.cgi/xml/htdocs/doc/en/rsync.xml?rev=1.49&content-type=text/plain&cvsroot=gentoo
diff : http://www.gentoo.org/cgi-bin/viewcvs.cgi/xml/htdocs/doc/en/rsync.xml.diff?r1=1.48&r2=1.49&cvsroot=gentoo
Index: rsync.xml
===================================================================
RCS file: /var/cvsroot/gentoo/xml/htdocs/doc/en/rsync.xml,v
retrieving revision 1.48
retrieving revision 1.49
diff -u -r1.48 -r1.49
--- rsync.xml 12 May 2006 12:59:39 -0000 1.48
+++ rsync.xml 12 May 2006 20:07:07 -0000 1.49
@@ -1,25 +1,19 @@
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE guide SYSTEM "/dtd/guide.dtd">
-<!-- $Header: /var/cvsroot/gentoo/xml/htdocs/doc/en/rsync.xml,v 1.48 2006/05/12 12:59:39 jforman Exp $ -->
+<!-- $Header: /var/cvsroot/gentoo/xml/htdocs/doc/en/rsync.xml,v 1.49 2006/05/12 20:07:07 neysx Exp $ -->
<guide link="/doc/en/rsync.xml" lang="en">
<title>Gentoo Linux rsync Mirrors Policy and Guide</title>
-<author title="Author, new version">
- Tobias Klausmann
-</author>
-
-<author title="Original Author">
+<author title="Author">
<mail link="mirror-admin@gentoo.org">Gentoo Mirror Administrators</mail>
</author>
-<author title="Original Editor">
- <mail link="neysx@gentoo.org">Xavier Neys</mail>
-</author>
-<author title="Contributor">
+<author title="Author">
<mail link="klausman-gentoo@schwarzvogel.de">Tobias Klausmann</mail>
</author>
-
-
+<author title="Editor">
+ <mail link="neysx@gentoo.org">Xavier Neys</mail>
+</author>
<abstract>
This document explains how to set up a official rsync mirror and your own local
@@ -30,7 +24,7 @@
<!-- See http://creativecommons.org/licenses/by-sa/2.5 -->
<license/>
-<version>3.0</version>
+<version>3.1</version>
<date>2006-05-12</date>
<chapter>
@@ -41,21 +35,24 @@
<p>
This guide is intended for people who would like to set up an rsync mirror of
-their own. I caters not only to those who want to run an official rsync mirror
+their own. It caters not only to those who want to run an official rsync mirror
but also those wanting to run private mirrors.
</p>
-<p>There are three kinds of Gentoo rsync mirrors: main rotation mirrors,
-community mirrors and private mirrors. Main rotation mirrors are maintained by
-the Gentoo infrastructure team. They handle the bulk of the Gentoo rsync
-traffic. The community mirrors are run by volunteers from the Gentoo community.
-Private mirrors are mirrors run by individuals which are closed off to the
-public and meant to cut traffic costs and latency for an organization or
-individual.</p>
+<p>
+There are three kinds of Gentoo rsync mirrors: main rotation mirrors, community
+mirrors and private mirrors. Main rotation mirrors are maintained by the Gentoo
+infrastructure team. They handle the bulk of the Gentoo rsync traffic. The
+community mirrors are run by volunteers from the Gentoo community. Private
+mirrors are mirrors run by individuals which are closed off to the public and
+meant to cut traffic costs and latency for an organization or individual.
+</p>
-<p>At this time, we have enough community mirrors and are actively seeking
+<p>
+At this time, we have enough community mirrors and are actively seeking
additional main rotation mirrors. Specifications for main rotation servers
-include:</p>
+include:
+</p>
<ul>
<li>Minimum of a 2GHz Pentium 4 processor (or equivalent)</li>
@@ -65,49 +62,57 @@
<p>
These servers may be donated with bandwidth and colocation space if you have
-them. Otherwise, we can provide these services and you can simply ship the
+them. Otherwise, we can provide these services and you can simply ship the
machine to our colocation facility. Average bandwidth consumption for a main
-rotation mirror is currently ~7Mbps (around 2 TB per month). As the number of
+rotation mirror is currently ~7Mbps (around 2 TB per month). As the number of
main rotation mirrors increases, this number should decrease accordingly.
</p>
<p>
If you would like to donate your machine, please email <mail
-link="jforman@gentoo.org">Jeffrey Forman</mail> with the pertinent information.
+link="jforman@gentoo.org">Jeffrey Forman</mail> with the pertinent
+information.
</p>
</body>
</section>
</chapter>
-<chapter>
+<chapter id="local">
<title>Setting up your own local rsync mirror</title>
<section>
<title>Introduction</title>
<body>
-<p>Many users run Gentoo on several machines and need to sync the portage trees
-on all of them. Using public mirrors is simply a waste of bandwidth at both
-ends. Syncing only one machine against a public mirror and all others against
-that computer would save resources on Gentoo mirrors and save users' bandwidth.
+<p>
+Many users run Gentoo on several machines and need to sync the portage trees on
+all of them. Using public mirrors is simply a waste of bandwidth at both ends.
+Syncing only one machine against a public mirror and all others against that
+computer would save resources on Gentoo mirrors and save users' bandwidth.
</p>
-<p>The same holds true for organizations who would like to control the rsync
-mirror their servers and workstations sync against. Of course, they usually also
-want to save on badnwidth and traffic costs. </p>
-
-<p>All you need to do is select which machine is going to be your own local
-rsync mirror and set it up. You should choose a computer that can handle the CPU
-and disk load that an rsync operation require. Your local mirror also needs to
-be available whenever any of your other computers syncs its portage tree.
+<p>
+The same holds true for organizations who would like to control the rsync
+mirror their servers and workstations sync against. Of course, they usually
+also want to save on bandwidth and traffic costs.
+</p>
+
+<p>
+All you need to do is select which machine is going to be your own local rsync
+mirror and set it up. You should choose a computer that can handle the CPU and
+disk load that an rsync operation requires. Your local mirror also needs to be
+available whenever any of your other computers syncs its portage tree.
Besides, it should have a static IP address or a name that always resolves to
-your server. Configuring a DHCP and/or a DNS server is beyond the scope of this
-guide.</p>
+your server. Configuring a DHCP and/or a DNS server is beyond the scope of this
+guide.
+</p>
-<p>Note that these instructions assume your private rsync mirror is a Gentoo
+<p>
+Note that these instructions assume your private rsync mirror is a Gentoo
machine. If you intend to run it on a different distribution, the guide for
setting up a community mirror might be more helpful. Just don't sync the mirror
-every half hour but once or twice a day.</p>
+every half hour but once or twice a day.
+</p>
</body>
</section>
@@ -115,11 +120,13 @@
<title>Setting up the server</title>
<body>
-<p>There is no extra package to install as the required software is already on
-your computer.Setting up your own local rsync mirror is just a matter of
-configuring the <e>rsyncd</e> daemon to make your <path>/usr/portage</path>
+<p>
+There is no extra package to install as the required software is already on
+your computer. Setting up your own local rsync mirror is just a matter of
+configuring the <c>rsyncd</c> daemon to make your <path>/usr/portage</path>
directory available for syncing. Create the following
-<path>/etc/rsyncd.conf</path> configuration file: </p>
+<path>/etc/rsyncd.conf</path> configuration file:
+</p>
<pre caption="Sample /etc/rsyncd.conf">
pid file = /var/run/rsyncd.pid
@@ -137,16 +144,19 @@
exclude=distfiles/ packages/
</pre>
-<p> You do not need to use the <c>hosts allow</c> and <c>hosts deny</c> options.
-By default, all clients will be allowed to connect. The order in which you write
-the options is not relevant. The server will always check the <c>hosts allow</c>
-option first and grant the connection if the connecting host matches any of the
-listed patterns. The server will then check the <c>hosts deny</c> option and
-refuse the connection if any match is found. Any host that does not match
-anything will be granted a connection. Please read the man page (<c>man
-rsyncd.conf</c>) for more information. </p>
+<p>
+You do not need to use the <c>hosts allow</c> and <c>hosts deny</c> options.
+By default, all clients will be allowed to connect. The order in which you
+write the options is not relevant. The server will always check the <c>hosts
+allow</c> option first and grant the connection if the connecting host matches
+any of the listed patterns. The server will then check the <c>hosts deny</c>
+option and refuse the connection if any match is found. Any host that does not
+match anything will be granted a connection. Please read the man page (<c>man
+rsyncd.conf</c>) for more information.
+</p>
-<p> Now, start your rsync daemon with the following command as the root user:
+<p>
+Now, start your rsync daemon with the following command as the root user:
</p>
<pre caption="Starting the rsync daemon">
@@ -156,9 +166,11 @@
# <i>rc-update add rsyncd default</i>
</pre>
-<p> Let's test your rsync mirror. You do not need to try from another machine
-but it would be a good idea to do so. If your server is not known by name from
-all your computers, you can use its IP address instead. </p>
+<p>
+Let's test your rsync mirror. You do not need to try from another machine but
+it would be a good idea to do so. If your server is not known by name from all
+your computers, you can use its IP address instead.
+</p>
<pre caption="Testing your mirror">
<comment>(You may use the server name or its IP)</comment>
@@ -168,14 +180,18 @@
<comment>(You should see the content of /usr/portage on your mirror)</comment>
</pre>
-<p> Your rsync mirror is now set up. Keep running <c>emerge --sync</c> as you
-have done so far to keep your server up-to-date. If you use cron or similar
+<p>
+Your rsync mirror is now set up. Keep running <c>emerge --sync</c> as you have
+done so far to keep your server up-to-date. If you use cron or similar
facilities to sync regularly, remember to keep it down to a sensible frequency
-like once or twice a day.</p>
+like once or twice a day.
+</p>
-<note> Please note that most public mirror administrators consider syncing more
-than once or twice a day an abuse. Some if not most of them will ban your IP
-from their server if you start abusing their machines.</note>
+<note>
+Please note that most public mirror administrators consider syncing more than
+once or twice a day an abuse. Some if not most of them will ban your IP from
+their server if you start abusing their machines.
+</note>
</body>
</section>
@@ -183,9 +199,11 @@
<title>Configuring your clients</title>
<body>
-<p> Now, make your other computers use your own local rsync mirror instead of a
+<p>
+Now, make your other computers use your own local rsync mirror instead of a
public one. Edit your <path>/etc/make.conf</path> and make the <c>SYNC</c>
-variable point to your server. </p>
+variable point to your server.
+</p>
<pre caption="Define SYNC in /etc/make.conf">
<comment>(Use your server IP addess)</comment>
@@ -194,8 +212,10 @@
SYNC="rsync://<i>your_server_name</i>/gentoo-portage"
</pre>
-<p> You can check that your computer has been properly set up by syncing against
-your own local mirror for the first time: </p>
+<p>
+You can check that your computer has been properly set up by syncing against
+your own local mirror for the first time:
+</p>
<pre caption="Checking and syncing">
<comment>(Check that the SYNC variable has been setup)</comment>
@@ -205,28 +225,32 @@
# <i>emerge --sync</i>
</pre>
-<p> That's it! All your computers will now use your local rsync mirror whenever
-you run <c>emerge --sync</c>. </p>
+<p>
+That's it! All your computers will now use your local rsync mirror whenever you
+run <c>emerge --sync</c>.
+</p>
</body>
</section>
</chapter>
-
<chapter>
<title>Setting up a community rsync server</title>
<section>
<title>Introduction</title>
<body>
-<note> You can find sample configuration and script files in the
-gentoo-rsync-mirror package. Just do <c>emerge gentoo-rsync-mirror</c> </note>
+<note>
+You can find sample configuration and script files in the gentoo-rsync-mirror
+package. Just do <c>emerge gentoo-rsync-mirror</c>.
+</note>
<p>
Right now, mirroring our Portage tree requires around 600Mb, so it isn't space
-intensive; having at least 1Gb free should allow for growing room. Setting
-up a Portage tree mirror is simple -- first, ensure that your mirror has rsync
-installed. Then, set up your <path>rsyncd.conf</path> file to look something like this:
+intensive; having at least 1Gb free should allow for growing room. Setting up a
+Portage tree mirror is simple -- first, ensure that your mirror has rsync
+installed. Then, set up your <path>rsyncd.conf</path> file to look something
+like this:
</p>
<pre caption="rsyncd.conf">
@@ -254,24 +278,32 @@
exclude = distfiles
</pre>
-<p>You can pick your own locations for most of the files, of course. What's
+<p>
+You can pick your own locations for most of the files, of course. What's
important are the section names (<c>[gentoo-portage]</c> and
-<c>[gentoo-x86-portage]</c>). They are the locations that rsync clients will try
-to sync from</p>
+<c>[gentoo-x86-portage]</c>). They are the locations that rsync clients will
+try to sync from.
+</p>
-<p>Above, the gentoo-x86-portage mirror points to the same data as
-gentoo-portage. Although we have recently changed the official name of our
-mirror to gentoo-portage, gentoo-x86-portage is still needed for backwards
-compatibility, so include both entries. Eventually, the gentoo-x86-portage
-location will be removed.</p>
+<p>
+Above, the gentoo-x86-portage mirror points to the same data as gentoo-portage.
+Although we have recently changed the official name of our mirror to
+gentoo-portage, gentoo-x86-portage is still needed for backwards compatibility,
+so include both entries. Eventually, the gentoo-x86-portage location will be
+removed.
+</p>
-<p> For security reasons, the use of a chrooted environment is required! This
-has implications for the logged timestamps -- see the FAQ below.</p>
+<p>
+For security reasons, the use of a chrooted environment is required! This has
+implications for the logged timestamps -- see the FAQ below.
+</p>
-<p> Now, you need to mirror the Gentoo Linux Portage tree. You can use the
-script below to do so. Again, you'll probably want to change some of the file
+<p>
+Now, you need to mirror the Gentoo Linux Portage tree. You can use the script
+below to do so. Again, you'll probably want to change some of the file
locations to suit your needs -- in particular, they should match those of your
-<path>rsyncd.conf</path>.</p>
+<path>rsyncd.conf</path>.
+</p>
<pre caption="rsync-gentoo-portage.sh">
#!/bin/bash
@@ -288,18 +320,22 @@
logger -t rsync "re-rsyncing the gentoo-portage tree"
${RSYNC} ${OPTS} ${SRC} ${DST} >> $0.log 2>&1
-echo "End: "`date` >> $0.log 2>&1
+echo "End: "`date` >> $0.log 2>&1
</pre>
-<p> Your <path>rsyncd.motd</path> should contain your IP address and other
-relevant information about your mirror, such as information about the host
-providing the Portage mirror and an administrative contact.You can now test your
-server as outlined in the "Setting up your own local rsync mirror" section
-above.</p>
+<p>
+Your <path>rsyncd.motd</path> should contain your IP address and other relevant
+information about your mirror, such as information about the host providing the
+Portage mirror and an administrative contact. You can now test your server as
+outlined in the <uri link="#local">Setting up your own local rsync
+mirror</uri> chapter above.
+</p>
-<p> After you have been approved as an official rsync mirror your host will be
+<p>
+After you have been approved as an official rsync mirror, your host will be
aliased with a name of the form: <path>rsync[num].[country
-code].gentoo.org</path> </p>
+code].gentoo.org</path>.
+</p>
</body>
</section>
@@ -311,33 +347,41 @@
<title>Q: Who should I contact regarding rsync issues and maintenance?</title>
<body>
-<p> A: Visit <uri link="http://bugs.gentoo.org">Gentoo Bugzilla</uri> and fill
-out a bug on the product "Mirrors", component "Server Problem". </p>
+<p>
+A: Visit <uri link="http://bugs.gentoo.org">Gentoo Bugzilla</uri> and fill out
+a bug on the product "Mirrors", component "Server Problem".
+</p>
</body>
</section>
<section>
<title>Q: How can I check the freshness of an official rsync server?</title>
-<body>
-<p>The Gentoo infrastructure team monitors all community rsync servers for
-freshness. You can see the results on the <uri link="http://mirrorstats.gentoo.org/rsync">corresponding web page</uri>.
-</p></body>
-</section>
+<body>
+<p>
+The Gentoo infrastructure team monitors all community rsync servers for
+freshness. You can see the results on the <uri
+link="http://mirrorstats.gentoo.org/rsync">corresponding web page</uri>.
+</p>
+
+</body>
+</section>
<section>
<title>Q: I run a private rsync mirror for my company. Can I still access
-rsync1.us.gentoo.org?</title>
+rsync1.us.gentoo.org?</title>
<body>
-<p> A: Because our resources are limited, we need to ensure we allocate them in
-such a way as to provide the maximum amount of benefit to our users. As such, we
-limit connections to our master rsync and distfile mirrors to public mirrors
+<p>
+A: Because our resources are limited, we need to ensure we allocate them in
+such a way as to provide the maximum amount of benefit to our users. As such,
+we limit connections to our master rsync and distfile mirrors to public mirrors
only. Users are welcome to use our regular mirror system to establish a private
rsync mirror, though they are asked to follow certain basic <uri
-link="http://www.gentoo.org/news/en/gwn/20030505-newsletter.xml#doc_chap1_sect3">
-rsync etiquette guidelines</uri>. </p>
+link="/news/en/gwn/20030505-newsletter.xml#doc_chap1_sect3">rsync etiquette
+guidelines</uri>.
+</p>
</body>
</section>
@@ -345,8 +389,9 @@
<title>Q: Is it important that I sync my mirror twice an hour?</title>
<body>
-<p> A: Yes it is important. You do not need to perform the syncs at exactly :00
-and :30 but the syncs should take place in each of the following two windows:
+<p>
+A: Yes it is important. You do not need to perform the syncs at exactly :00 and
+:30 but the syncs should take place in each of the following two windows:
</p>
<ol>
@@ -354,14 +399,14 @@
<li>:30 to :40</li>
</ol>
-<p> Additionally, please make sure that your syncs are exactly 30 minutes apart.
+<p>
+Additionally, please make sure that your syncs are exactly 30 minutes apart.
So, if you schedule the first sync of each hour for :08, please schedule the
second sync of the hour for :38.
</p>
</body>
</section>
-
<section>
<title>Q: Where should I sync my rsync mirror before I become an official Gentoo mirror?</title>
<body>
@@ -379,28 +424,30 @@
<title>Q: How do I find the mirror nearest to me?</title>
<body>
-<p> A: <c>netselect</c> was designed to do this for you. If you haven't already
-run <c>emerge netselect</c> then do it. Then run: <c>netselect
+<p>
+A: <c>netselect</c> was designed to do this for you. If you haven't already run
+<c>emerge netselect</c> then do it. Then run: <c>netselect
rsync.gentoo.org</c>. After a minute or so netselect will print an IP address.
Take this address and use it as the only parameter for rsync with two colons
-appended to it. eg: <c>rsync 1.2.3.4::</c>. You should be able to find out which
-mirror that is from the banner message. Update your <path>/etc/make.conf</path>
-accordingly. </p>
+appended to it. eg: <c>rsync 1.2.3.4::</c>. You should be able to find out
+which mirror that is from the banner message. Update your
+<path>/etc/make.conf</path> accordingly.
+</p>
</body>
</section>
-
<section>
<title>Q: Can I use compression when syncing against rsync1.us.gentoo.org?</title>
<body>
-<p> A: No. Compression utilizes too many resources on the server, so we have
+<p>
+A: No. Compression utilizes too many resources on the server, so we have
forcibly disabled it on <path>rsync1.us.gentoo.org</path>. Please <e>do not</e>
-attempt to use compression when syncing against this server. </p>
+attempt to use compression when syncing against this server.
+</p>
</body>
</section>
-
<section>
<title>Q: I'm seeing a lot of old and probably dead rsync processes, how can I
get rid of them?</title>
@@ -416,7 +463,7 @@
<pre caption="Kill old rsync processes">
/bin/kill -9 `/bin/ps --no-headers -Crsync -o etime,user,pid,command|/bin/grep nobody | \
- /bin/grep "[0-9]\{2\}:[0-9]\{2\}:" |/bin/awk '{print $3}'`
+ /bin/grep "[0-9]\{2\}:[0-9]\{2\}:" |/bin/awk '{print $3}'`
</pre>
</body>
@@ -429,30 +476,30 @@
<p>
In some cases, there are a few inconsiderate users who abuse the rsync mirror
-system by syncing more than 1-2 times per day. In the most extreme cases, users
-schedule cron jobs to sync every 15 minutes or so. This often leads to a Denial
-of Service attack by continually occupying an rsync slot that could have
+system by syncing more than 1-2 times per day. In the most extreme cases,
+users schedule cron jobs to sync every 15 minutes or so. This often leads to a
+Denial of Service attack by continually occupying an rsync slot that could have
otherwise gone to another user. To try and prevent this, you may use the <uri
-link="/proj/en/infrastructure/mirrors/rsyncd.conf_pl.txt">this perl script</uri>
-which will scan through your rsync log files, pick out IP addresses that have
-already connected more than <c>N</c> times that day and dynamically create a
-<path>rsyncd.conf</path> file, including the offending IP addresses in the
+link="/proj/en/infrastructure/mirrors/rsyncd.conf_pl.txt">this perl
+script</uri> which will scan your rsync log files, pick out IP addresses that
+have already connected more than <c>N</c> times that day and dynamically create
+a <path>rsyncd.conf</path> file, including the offending IP addresses in the
'hosts deny' directive. The following line controls what <c>N</c> equals (in
-this case 4): </p>
+this case 4):
+</p>
<pre caption="Define maximum number of connections per IP">
@badhosts=grep {$hash{$_}>4} keys %hash;
</pre>
-<p> If you use this script, please remember to rotate your rsync log files daily
+<p>
+If you use this script, please remember to rotate your rsync log files daily
and modify the script to match the location of your <path>rsyncd.conf</path>
file. This script is tested on Gentoo Linux, but should work suitably on other
-arches that support both rsync and perl. </p>
+arches that support both rsync and perl.
+</p>
</body>
</section>
</chapter>
-
-
-
</guide>
--
gentoo-doc-cvs@gentoo.org mailing list
next reply other threads:[~2006-05-12 20:06 UTC|newest]
Thread overview: 16+ messages / expand[flat|nested] mbox.gz Atom feed top
2006-05-12 20:07 Xavier Neys [this message]
-- strict thread matches above, loose matches on Subject: below --
2007-12-27 17:22 [gentoo-doc-cvs] cvs commit: rsync.xml Shyam Mani
2006-12-25 18:53 Josh Saddler
2006-12-25 18:47 undefined
2006-07-20 15:09 Lukasz Damentko
2006-07-17 18:03 Lukasz Damentko
2006-06-04 19:40 undefined
2006-05-23 18:21 Jan Kundrat
2006-05-23 16:40 undefined
2006-05-12 12:59 undefined
2006-05-12 11:21 undefined
2005-12-12 16:04 Xavier Neys
2005-12-11 10:07 swift
2005-12-11 1:22 undefined
2005-11-07 11:54 Xavier Neys
2005-11-07 11:47 Xavier Neys
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20060512200708.56DAD6475F@smtp.gentoo.org \
--to=neysx@lark.gentoo.org \
--cc=docs-team@lists.gentoo.org \
--cc=gentoo-doc-cvs@lists.gentoo.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox