* Re: [gentoo-mirrors] Gentoo Linux 2007.0 and BitTorrent
2007-05-04 1:44 ` [gentoo-mirrors] Gentoo Linux 2007.0 and BitTorrent Robin H. Johnson
@ 2007-05-04 6:58 ` Martin Eisenhardt
2007-05-04 9:57 ` Georgi Georgiev
` (4 subsequent siblings)
5 siblings, 0 replies; 11+ messages in thread
From: Martin Eisenhardt @ 2007-05-04 6:58 UTC (permalink / raw
To: gentoo-mirrors
[-- Attachment #1: Type: text/plain, Size: 1257 bytes --]
Hello Robin,
hello list,
On Friday 04 May 2007 03:44:06 Robin H. Johnson wrote:
> BitTorrent-only:
> 3.7G livedvd-amd64-installer-2007.0/
> 3.8G livedvd-i686-installer-2007.0/
>
> If you're interested in direct BitTorrent seeding (esp. for the above
> two, even if you aren't seeding anything else), please give me a shout
> so that I can arrange to get them to you.
Shout. :-D
I am interested in seeding the new DVD images, although not at the 50 Mbps you
mentioned in your posting. But a few Mbps might help as well, I guess ...
> For part two of the HTTP-seeding (eg making the BT clients actually use
> the mirrors), I'd like to know which of the experimental mirrors are
> interested in being used for HTTP seeding.
>
> Your webservers will need to support HTTP Range requests for this to
> work. The BitTorrent clients will send range requests for 32K..256K
> blocks (powers of two only).
Assuming that a vanilla Apache 2.0 supports range requests (which it does,
AFAIK), I would like to volunteer for that as well.
Just contact me with the details about how to get the files and what
information you need.
Kind regards
Martin Eisenhardt
---
GnuPG key: 3BBC EBB2 4A36 D2AD E636 556F 30D9 0EE4 C7F1 C1CF
[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]
^ permalink raw reply [flat|nested] 11+ messages in thread* Re: [gentoo-mirrors] Gentoo Linux 2007.0 and BitTorrent
2007-05-04 1:44 ` [gentoo-mirrors] Gentoo Linux 2007.0 and BitTorrent Robin H. Johnson
2007-05-04 6:58 ` Martin Eisenhardt
@ 2007-05-04 9:57 ` Georgi Georgiev
2007-05-04 10:52 ` Robin H. Johnson
2007-05-05 4:07 ` Georgi Georgiev
2007-05-04 10:35 ` Francois Petillon
` (3 subsequent siblings)
5 siblings, 2 replies; 11+ messages in thread
From: Georgi Georgiev @ 2007-05-04 9:57 UTC (permalink / raw
To: gentoo-mirrors
[-- Attachment #1: Type: text/plain, Size: 1137 bytes --]
maillog: 03/05/2007-18:44:06(-0700): Robin H. Johnson types
...
> For part two of the HTTP-seeding (eg making the BT clients actually use
> the mirrors), I'd like to know which of the experimental mirrors are
> interested in being used for HTTP seeding.
>
> Your webservers will need to support HTTP Range requests for this to
> work. The BitTorrent clients will send range requests for 32K..256K
> blocks (powers of two only).
...
How is HTTP BT seeding different than ordinary mirroring (assuming the
server supports Range)? As I don't need to do anything on the server
side (that's my understanding for the moment at least), is there a
particular reason why I should care? We (mirrors) are still going to
serve that same content that we have already agreed to serve.
If this was just out of courtesy, then sure, thanks for asking,
apologies for the above, and feel free to add gg3.net to those torrent
files.
--
(* Georgi Georgiev (* "Go to Heaven for the climate, Hell for (*
*) chutz@gg3.net *) the company." -- Mark Twain *)
(* http://www.gg3.net/ (* (*
[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]
^ permalink raw reply [flat|nested] 11+ messages in thread* Re: [gentoo-mirrors] Gentoo Linux 2007.0 and BitTorrent
2007-05-04 9:57 ` Georgi Georgiev
@ 2007-05-04 10:52 ` Robin H. Johnson
2007-05-05 4:07 ` Georgi Georgiev
1 sibling, 0 replies; 11+ messages in thread
From: Robin H. Johnson @ 2007-05-04 10:52 UTC (permalink / raw
To: gentoo-mirrors
[-- Attachment #1: Type: text/plain, Size: 1800 bytes --]
On Fri, May 04, 2007 at 06:57:21PM +0900, Georgi Georgiev wrote:
> > Your webservers will need to support HTTP Range requests for this to
> > work. The BitTorrent clients will send range requests for 32K..256K
> > blocks (powers of two only).
> How is HTTP BT seeding different than ordinary mirroring (assuming the
> server supports Range)? As I don't need to do anything on the server
> side (that's my understanding for the moment at least), is there a
> particular reason why I should care? We (mirrors) are still going to
> serve that same content that we have already agreed to serve.
I asked it mainly as courtesy of the mirrors, but also because I was
concerned about any cases where it might cause trouble for the mirrors:
- In the best case, if your-choice-httpd supports both KeepAlive and
Range, then the BitTorrent client will pick a point N, and just
sequentially ask for block N, N+1, N+2, ... on that single HTTP
connection - this takes advantage of disk read-ahead on the server
side.
- If your httpd does not support range, there's going to be a separate
connection for each block, which might accidentally trigger some abuse
detection systems. A client with 1Mbit could cause 3 connections per
second to your mirror.
- If your httpd does not support Range, the BitTorrent client would
fail-out.
- From a discussion that was asked of me on #gentoo-mirrors, if your
mirror server used some non-default IO readahead (eg posix_fadvise),
there's also the possibility that it would increase your disk
thrashing since it asks for lots of small chunks instead of big ones.
--
Robin Hugh Johnson
Gentoo Linux Developer & Council Member
E-Mail : robbat2@gentoo.org
GnuPG FP : 11AC BA4F 4778 E3F6 E4ED F38E B27B 944E 3488 4E85
[-- Attachment #2: Type: application/pgp-signature, Size: 321 bytes --]
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [gentoo-mirrors] Gentoo Linux 2007.0 and BitTorrent
2007-05-04 9:57 ` Georgi Georgiev
2007-05-04 10:52 ` Robin H. Johnson
@ 2007-05-05 4:07 ` Georgi Georgiev
1 sibling, 0 replies; 11+ messages in thread
From: Georgi Georgiev @ 2007-05-05 4:07 UTC (permalink / raw
To: gentoo-mirrors
[-- Attachment #1: Type: text/plain, Size: 523 bytes --]
maillog: 04/05/2007-18:57:21(+0900): Георги Георгиев types
>
> If this was just out of courtesy, then sure, thanks for asking,
> apologies for the above, and feel free to add gg3.net to those torrent
> files.
Oops, scratch the above. I just realized that gg3.net is not serving
releases.
--
() Georgi Georgiev () Blow it out your ear. ()
() chutz@gg3.net () ()
() http://www.gg3.net/ () ()
[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [gentoo-mirrors] Gentoo Linux 2007.0 and BitTorrent
2007-05-04 1:44 ` [gentoo-mirrors] Gentoo Linux 2007.0 and BitTorrent Robin H. Johnson
2007-05-04 6:58 ` Martin Eisenhardt
2007-05-04 9:57 ` Georgi Georgiev
@ 2007-05-04 10:35 ` Francois Petillon
2007-05-04 14:52 ` Greg Cornell
` (2 subsequent siblings)
5 siblings, 0 replies; 11+ messages in thread
From: Francois Petillon @ 2007-05-04 10:35 UTC (permalink / raw
To: gentoo-mirrors
Robin H. Johnson wrote:
> BitTorrent-only:
> 3.7G livedvd-amd64-installer-2007.0/
> 3.8G livedvd-i686-installer-2007.0/
After a small discussion with Robin on IRC, I would like to know if I am
the only one who is willing to host these two ISOs on my mirror (FTP
only) or if other people would appreciate to host them too...
François
--
gentoo-mirrors@gentoo.org mailing list
^ permalink raw reply [flat|nested] 11+ messages in thread* RE: [gentoo-mirrors] Gentoo Linux 2007.0 and BitTorrent
2007-05-04 1:44 ` [gentoo-mirrors] Gentoo Linux 2007.0 and BitTorrent Robin H. Johnson
` (2 preceding siblings ...)
2007-05-04 10:35 ` Francois Petillon
@ 2007-05-04 14:52 ` Greg Cornell
2007-05-04 15:38 ` Vladimir Brik
2007-07-04 1:30 ` Philip M. White
5 siblings, 0 replies; 11+ messages in thread
From: Greg Cornell @ 2007-05-04 14:52 UTC (permalink / raw
To: gentoo-mirrors
Hi Robin,
I'm happy to serve up the content via whatever protocols Gentoo needs.
However, I haven't set up bittorrent before so maybe a short howto on
settting up bittorrent (and even checking for the range support in apache)
would be helpful.
Greg
> -----Original Message-----
> From: Robin H. Johnson [mailto:robbat2@gentoo.org]
> Sent: Thursday, May 03, 2007 6:44 PM
> To: gentoo-mirrors@lists.gentoo.org
> Cc: wolf31o2@gentoo.org
> Subject: [gentoo-mirrors] Gentoo Linux 2007.0 and BitTorrent
>
> On Thu, May 03, 2007 at 04:34:19PM -0700, Robin H. Johnson wrote:
> > Would all mirrors that carry the experimental tree please see my
> > follow-up email about BitTorrent HTTP-seeding?
>
> Hi folks,
>
> So now stuff on BitTorrent for the 2007.0 release.
>
> There's two portions to this email, firstly stuff distributed by
> BitTorrent only, and then everything else.
>
> We have 94 torrents in total, 2 of which are only media that is only
> being distributed by BitTorrent.
>
> We needs seeds for all of it, but the bittorrent-only stuff more than
> the rest. I have one sponsor offering 50Mbit * 2 weeks for seeding
> everything so far.
>
> In terms of basic structure, there is a .torrent for every .tar.bz2 or
> .iso in releases or experimental, containing the core file, plus it's
> DIGESTS, signature, and CONTENTS files.
>
> BitTorrent-only:
> 3.7G livedvd-amd64-installer-2007.0/
> 3.8G livedvd-i686-installer-2007.0/
>
> If you're interested in direct BitTorrent seeding (esp. for the above
> two, even if you aren't seeding anything else), please give me a shout
> so that I can arrange to get them to you.
>
> Now for the experimental part...
>
> To help out long-term seeding of the majority of torrents (those that
> aren't bittorrent-only), I'd like to take advantage of the newish HTTP-
> seeding support of BitTorrent [1].
>
> HTTP-seeding kicks in at a MUCH lower priority than regular BitTorrent
> peering, basically helping clients that cannot contact any other source
> for missing pieces. It effectively converts the BitTorrent client to a
> slightly-smart HTTP fetcher that then shares the blocks back over
> BitTorrent to other clients.
>
> For all experimental/ mirrors, that means we're adding a directory tree
> under experimental/ that has directories containing only symlinks. Your
> rsync options will need to have the various symlink-transfer options,
> which I think should be in place already.
>
>
> Here's an example of one.
> experimental/torrent-symlinks/stage3-x86-2007.0:
> lrwxrwxrwx stage3-x86-2007.0.tar.bz2 ->
> ../../releases/x86/2007.0/stages/stage3-x86-2007.0.tar.bz2
> lrwxrwxrwx stage3-x86-2007.0.tar.bz2.CONTENTS ->
> ../../releases/x86/2007.0/stages/stage3-x86-2007.0.tar.bz2.CONTENTS
> lrwxrwxrwx stage3-x86-2007.0.tar.bz2.DIGESTS ->
> ../../releases/x86/2007.0/stages/stage3-x86-2007.0.tar.bz2.DIGESTS
> lrwxrwxrwx stage3-x86-2007.0.tar.bz2.asc ->
> ../../releases/x86/2007.0/stages/stage3-x86-2007.0.tar.bz2.asc
>
> This in itself doesn't make the BitTorrent clients contact you, so even
> if you aren't participating, it doesn't cause harm to have it.
>
> For part two of the HTTP-seeding (eg making the BT clients actually use
> the mirrors), I'd like to know which of the experimental mirrors are
> interested in being used for HTTP seeding.
>
> Your webservers will need to support HTTP Range requests for this to
> work. The BitTorrent clients will send range requests for 32K..256K
> blocks (powers of two only).
>
> I'll be placing the base potion of your URLs into the .torrent files,
> as "$BASEURL/experimental/torrent-symlinks/"
>
> [1] The actual specification for the HTTP-seeding is here:
> http://www.getright.com/seedtorrent.html
>
> --
> Robin Hugh Johnson
> Gentoo Linux Developer & Council Member
> E-Mail : robbat2@gentoo.org
> GnuPG FP : 11AC BA4F 4778 E3F6 E4ED F38E B27B 944E 3488 4E85
--
gentoo-mirrors@gentoo.org mailing list
^ permalink raw reply [flat|nested] 11+ messages in thread* Re: [gentoo-mirrors] Gentoo Linux 2007.0 and BitTorrent
2007-05-04 1:44 ` [gentoo-mirrors] Gentoo Linux 2007.0 and BitTorrent Robin H. Johnson
` (3 preceding siblings ...)
2007-05-04 14:52 ` Greg Cornell
@ 2007-05-04 15:38 ` Vladimir Brik
2007-07-04 1:30 ` Philip M. White
5 siblings, 0 replies; 11+ messages in thread
From: Vladimir Brik @ 2007-05-04 15:38 UTC (permalink / raw
To: gentoo-mirrors; +Cc: wolf31o2
I would be interested to in participating in both direct and HTTP
seeding using gentoo.chem.wisc.edu. The server is on a shared 100Mbps
connection, with about 80% of bandwidth being typically available. I
could also seed bittorrent-only files from a machine on a different
network for a limited time (no HTTP seeding for that one though) if
need be.
Vlad
On 5/3/07, Robin H. Johnson <robbat2@gentoo.org> wrote:
> On Thu, May 03, 2007 at 04:34:19PM -0700, Robin H. Johnson wrote:
> > Would all mirrors that carry the experimental tree please see my
> > follow-up email about BitTorrent HTTP-seeding?
>
> Hi folks,
>
> So now stuff on BitTorrent for the 2007.0 release.
>
> There's two portions to this email, firstly stuff distributed by
> BitTorrent only, and then everything else.
>
> We have 94 torrents in total, 2 of which are only media that is only
> being distributed by BitTorrent.
>
> We needs seeds for all of it, but the bittorrent-only stuff more than
> the rest. I have one sponsor offering 50Mbit * 2 weeks for seeding
> everything so far.
>
> In terms of basic structure, there is a .torrent for every .tar.bz2 or
> .iso in releases or experimental, containing the core file, plus it's
> DIGESTS, signature, and CONTENTS files.
>
> BitTorrent-only:
> 3.7G livedvd-amd64-installer-2007.0/
> 3.8G livedvd-i686-installer-2007.0/
>
> If you're interested in direct BitTorrent seeding (esp. for the above
> two, even if you aren't seeding anything else), please give me a shout
> so that I can arrange to get them to you.
>
> Now for the experimental part...
>
> To help out long-term seeding of the majority of torrents (those that
> aren't bittorrent-only), I'd like to take advantage of the newish
> HTTP-seeding support of BitTorrent [1].
>
> HTTP-seeding kicks in at a MUCH lower priority than regular BitTorrent
> peering, basically helping clients that cannot contact any other source
> for missing pieces. It effectively converts the BitTorrent client to a
> slightly-smart HTTP fetcher that then shares the blocks back over
> BitTorrent to other clients.
>
> For all experimental/ mirrors, that means we're adding a directory tree
> under experimental/ that has directories containing only symlinks. Your
> rsync options will need to have the various symlink-transfer options,
> which I think should be in place already.
>
> Here's an example of one.
> experimental/torrent-symlinks/stage3-x86-2007.0:
> lrwxrwxrwx stage3-x86-2007.0.tar.bz2 -> ../../releases/x86/2007.0/stages/stage3-x86-2007.0.tar.bz2
> lrwxrwxrwx stage3-x86-2007.0.tar.bz2.CONTENTS -> ../../releases/x86/2007.0/stages/stage3-x86-2007.0.tar.bz2.CONTENTS
> lrwxrwxrwx stage3-x86-2007.0.tar.bz2.DIGESTS -> ../../releases/x86/2007.0/stages/stage3-x86-2007.0.tar.bz2.DIGESTS
> lrwxrwxrwx stage3-x86-2007.0.tar.bz2.asc -> ../../releases/x86/2007.0/stages/stage3-x86-2007.0.tar.bz2.asc
>
> This in itself doesn't make the BitTorrent clients contact you, so even
> if you aren't participating, it doesn't cause harm to have it.
>
> For part two of the HTTP-seeding (eg making the BT clients actually use
> the mirrors), I'd like to know which of the experimental mirrors are
> interested in being used for HTTP seeding.
>
> Your webservers will need to support HTTP Range requests for this to
> work. The BitTorrent clients will send range requests for 32K..256K
> blocks (powers of two only).
>
> I'll be placing the base potion of your URLs into the .torrent files, as
> "$BASEURL/experimental/torrent-symlinks/"
>
> [1] The actual specification for the HTTP-seeding is here:
> http://www.getright.com/seedtorrent.html
>
> --
> Robin Hugh Johnson
> Gentoo Linux Developer & Council Member
> E-Mail : robbat2@gentoo.org
> GnuPG FP : 11AC BA4F 4778 E3F6 E4ED F38E B27B 944E 3488 4E85
>
>
--
gentoo-mirrors@gentoo.org mailing list
^ permalink raw reply [flat|nested] 11+ messages in thread* Re: [gentoo-mirrors] Gentoo Linux 2007.0 and BitTorrent
2007-05-04 1:44 ` [gentoo-mirrors] Gentoo Linux 2007.0 and BitTorrent Robin H. Johnson
` (4 preceding siblings ...)
2007-05-04 15:38 ` Vladimir Brik
@ 2007-07-04 1:30 ` Philip M. White
2007-07-04 1:39 ` Robin H. Johnson
5 siblings, 1 reply; 11+ messages in thread
From: Philip M. White @ 2007-07-04 1:30 UTC (permalink / raw
To: gentoo-mirrors
[-- Attachment #1: Type: text/plain, Size: 1226 bytes --]
On Thu, May 03, 2007 at 06:44:06PM -0700, Robin H. Johnson wrote:
> So now stuff on BitTorrent for the 2007.0 release.
>
> There's two portions to this email, firstly stuff distributed by
> BitTorrent only, and then everything else.
>
> We have 94 torrents in total, 2 of which are only media that is only
> being distributed by BitTorrent.
>
> We needs seeds for all of it, but the bittorrent-only stuff more than
> the rest. I have one sponsor offering 50Mbit * 2 weeks for seeding
> everything so far.
>
> In terms of basic structure, there is a .torrent for every .tar.bz2 or
> .iso in releases or experimental, containing the core file, plus it's
> DIGESTS, signature, and CONTENTS files.
>
> BitTorrent-only:
> 3.7G livedvd-amd64-installer-2007.0/
> 3.8G livedvd-i686-installer-2007.0/
>
> If you're interested in direct BitTorrent seeding (esp. for the above
> two, even if you aren't seeding anything else), please give me a shout
> so that I can arrange to get them to you.
I've recently received permission from the network administration of my
mirror to run a BitTorrent client. Where can I get these torrents
(other than downloading them off the web one-by-one)?
--
Philip
[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]
^ permalink raw reply [flat|nested] 11+ messages in thread