* Re: [gentoo-user] The LIGHTEST web server (just for serving files)?
2011-11-12 12:40 ` Pandu Poluan
@ 2011-11-12 13:01 ` Mick
2011-11-12 13:13 ` Stéphane Guedon
2011-11-12 13:22 ` Pandu Poluan
2011-11-12 13:11 ` YoYo Siska
` (2 subsequent siblings)
3 siblings, 2 replies; 22+ messages in thread
From: Mick @ 2011-11-12 13:01 UTC (permalink / raw
To: gentoo-user
[-- Attachment #1: Type: Text/Plain, Size: 1921 bytes --]
On Saturday 12 Nov 2011 12:40:08 Pandu Poluan wrote:
> On Nov 12, 2011 7:00 PM, "Mick" <michaelkintzios@gmail.com> wrote:
> > I've been using boa just for this purpose for years:
> >
> > * www-servers/boa
> >
> > Available versions:
> > ~ 0.94.14_rc21 "~x86 ~sparc ~mips ~ppc ~amd64" [doc]
> >
> > Homepage: http://www.boa.org/
> > Description: A very small and very fast http daemon.
> >
> > It can be easily locked down for internet facing roles.
> >
> > I've also used thttpd (you can throttle its bandwidth if that's important
>
> in
>
> > your network), but it's probably more than required for this purpose:
> >
> > * www-servers/thttpd
> >
> > Available versions:
> > 2.25b-r7 "amd64 ~hppa ~mips ppc sparc x86
>
> ~x86-fbsd" [static]
>
> > ~ 2.25b-r8 "~amd64 ~hppa ~mips ~ppc ~sparc ~x86
>
> ~x86-fbsd"
>
> > [static]
> >
> > Homepage: http://www.acme.com/software/thttpd/
> > Description: Small and fast multiplexing webserver.
>
> Thanks for all the input!
>
> During my drive home, something hit my brain: why not have the 'master'
> server share the distfiles dir via NFS?
>
> So, the question now becomes: what's the drawback/benefit of NFS-sharing vs
> HTTP-sharing? The scenario is back-end LAN at the office, thus, a trusted
> network by definition.
HTTP is not really 'sharing'. It is just 'copying'. Clients download the
distfiles from the home server to minimise load on the gentoo mirrors.
Following the download a client machine will have a local copy of said distfile
in the client://usr/distfile.
With NFS there is only one copy of the file, on the server, shared by other
clients in the LAN.
In my case the server is not always on, so NFS would not be appropriate.
--
Regards,
Mick
[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 198 bytes --]
^ permalink raw reply [flat|nested] 22+ messages in thread* Re: [gentoo-user] The LIGHTEST web server (just for serving files)?
2011-11-12 13:01 ` Mick
@ 2011-11-12 13:13 ` Stéphane Guedon
2011-11-12 13:22 ` Pandu Poluan
1 sibling, 0 replies; 22+ messages in thread
From: Stéphane Guedon @ 2011-11-12 13:13 UTC (permalink / raw
To: gentoo-user
[-- Attachment #1: Type: Text/Plain, Size: 2721 bytes --]
On Saturday 12 November 2011 14:01:54 Mick wrote:
> On Saturday 12 Nov 2011 12:40:08 Pandu Poluan wrote:
> > On Nov 12, 2011 7:00 PM, "Mick" <michaelkintzios@gmail.com> wrote:
> > > I've been using boa just for this purpose for years:
> > >
> > > * www-servers/boa
> > >
> > > Available versions:
> > > ~ 0.94.14_rc21 "~x86 ~sparc ~mips ~ppc ~amd64"
> > > [doc]
> > >
> > > Homepage: http://www.boa.org/
> > > Description: A very small and very fast http daemon.
> > >
> > > It can be easily locked down for internet facing roles.
> > >
> > > I've also used thttpd (you can throttle its bandwidth if that's
> > > important
> >
> > in
> >
> > > your network), but it's probably more than required for this purpose:
> > >
> > > * www-servers/thttpd
> > >
> > > Available versions:
> > > 2.25b-r7 "amd64 ~hppa ~mips ppc sparc x86
> >
> > ~x86-fbsd" [static]
> >
> > > ~ 2.25b-r8 "~amd64 ~hppa ~mips ~ppc ~sparc ~x86
> >
> > ~x86-fbsd"
> >
> > > [static]
> > >
> > > Homepage: http://www.acme.com/software/thttpd/
> > > Description: Small and fast multiplexing webserver.
> >
> > Thanks for all the input!
> >
> > During my drive home, something hit my brain: why not have the 'master'
> > server share the distfiles dir via NFS?
> >
> > So, the question now becomes: what's the drawback/benefit of NFS-sharing
> > vs HTTP-sharing? The scenario is back-end LAN at the office, thus, a
> > trusted network by definition.
>
> HTTP is not really 'sharing'. It is just 'copying'. Clients download the
> distfiles from the home server to minimise load on the gentoo mirrors.
> Following the download a client machine will have a local copy of said
> distfile in the client://usr/distfile.
>
> With NFS there is only one copy of the file, on the server, shared by other
> clients in the LAN.
>
> In my case the server is not always on, so NFS would not be appropriate.
With NFS, there is only one distfiles repo. So, when someone emerge an unknown
program, the distfiles become available to your whole network (if shared of
course).
I share /mnt/portage (the portage tree on a reiserfs) and /mnt/distfiles (on a
ext4 fs). The tree is synced only once also ! It reduce server load (bandwitch
on gentoo master server) and your global diskspace used (because repo are
mutualized).
--
Stéphane Guedon
page web : http://www.22decembre.eu/
carte de visite : http://www.22decembre.eu/downloads/Stephane-Guedon.vcf
clé publique gpg : http://www.22decembre.eu/downloads/Stephane-Guedon.asc
[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 316 bytes --]
^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: [gentoo-user] The LIGHTEST web server (just for serving files)?
2011-11-12 13:01 ` Mick
2011-11-12 13:13 ` Stéphane Guedon
@ 2011-11-12 13:22 ` Pandu Poluan
1 sibling, 0 replies; 22+ messages in thread
From: Pandu Poluan @ 2011-11-12 13:22 UTC (permalink / raw
To: gentoo-user
[-- Attachment #1: Type: text/plain, Size: 2086 bytes --]
On Nov 12, 2011 8:05 PM, "Mick" <michaelkintzios@gmail.com> wrote:
>
> On Saturday 12 Nov 2011 12:40:08 Pandu Poluan wrote:
> > On Nov 12, 2011 7:00 PM, "Mick" <michaelkintzios@gmail.com> wrote:
> > > I've been using boa just for this purpose for years:
> > >
> > > * www-servers/boa
> > >
> > > Available versions:
> > > ~ 0.94.14_rc21 "~x86 ~sparc ~mips ~ppc ~amd64"
[doc]
> > >
> > > Homepage: http://www.boa.org/
> > > Description: A very small and very fast http daemon.
> > >
> > > It can be easily locked down for internet facing roles.
> > >
> > > I've also used thttpd (you can throttle its bandwidth if that's
important
> >
> > in
> >
> > > your network), but it's probably more than required for this purpose:
> > >
> > > * www-servers/thttpd
> > >
> > > Available versions:
> > > 2.25b-r7 "amd64 ~hppa ~mips ppc sparc x86
> >
> > ~x86-fbsd" [static]
> >
> > > ~ 2.25b-r8 "~amd64 ~hppa ~mips ~ppc ~sparc ~x86
> >
> > ~x86-fbsd"
> >
> > > [static]
> > >
> > > Homepage: http://www.acme.com/software/thttpd/
> > > Description: Small and fast multiplexing webserver.
> >
> > Thanks for all the input!
> >
> > During my drive home, something hit my brain: why not have the 'master'
> > server share the distfiles dir via NFS?
> >
> > So, the question now becomes: what's the drawback/benefit of
NFS-sharing vs
> > HTTP-sharing? The scenario is back-end LAN at the office, thus, a
trusted
> > network by definition.
>
> HTTP is not really 'sharing'. It is just 'copying'. Clients download the
> distfiles from the home server to minimise load on the gentoo mirrors.
Yeah, should've put quotes around the sharing part.
> Following the download a client machine will have a local copy of said
distfile
> in the client://usr/distfile.
>
> With NFS there is only one copy of the file, on the server, shared by
other
> clients in the LAN.
>
> In my case the server is not always on, so NFS would not be appropriate.
Hmmm, you have a point there.
Rgds,
[-- Attachment #2: Type: text/html, Size: 3127 bytes --]
^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: [gentoo-user] The LIGHTEST web server (just for serving files)?
2011-11-12 12:40 ` Pandu Poluan
2011-11-12 13:01 ` Mick
@ 2011-11-12 13:11 ` YoYo Siska
2011-11-12 13:23 ` Pandu Poluan
2011-11-14 10:05 ` J. Roeleveld
2011-11-12 13:14 ` [gentoo-user] " Florian Philipp
2011-11-12 22:01 ` [gentoo-user] " Neil Bothwick
3 siblings, 2 replies; 22+ messages in thread
From: YoYo Siska @ 2011-11-12 13:11 UTC (permalink / raw
To: gentoo-user
On Sat, Nov 12, 2011 at 07:40:08PM +0700, Pandu Poluan wrote:
> On Nov 12, 2011 7:00 PM, "Mick" <michaelkintzios@gmail.com> wrote:
> >
> > I've been using boa just for this purpose for years:
> >
> > * www-servers/boa
> > Available versions:
> > ~ 0.94.14_rc21 "~x86 ~sparc ~mips ~ppc ~amd64" [doc]
> > Homepage: http://www.boa.org/
> > Description: A very small and very fast http daemon.
> >
> > It can be easily locked down for internet facing roles.
> >
> > I've also used thttpd (you can throttle its bandwidth if that's important
> in
> > your network), but it's probably more than required for this purpose:
> >
> > * www-servers/thttpd
> > Available versions:
> > 2.25b-r7 "amd64 ~hppa ~mips ppc sparc x86
> ~x86-fbsd" [static]
> > ~ 2.25b-r8 "~amd64 ~hppa ~mips ~ppc ~sparc ~x86
> ~x86-fbsd"
> > [static]
> > Homepage: http://www.acme.com/software/thttpd/
> > Description: Small and fast multiplexing webserver.
>
> Thanks for all the input!
>
> During my drive home, something hit my brain: why not have the 'master'
> server share the distfiles dir via NFS?
>
> So, the question now becomes: what's the drawback/benefit of NFS-sharing vs
> HTTP-sharing? The scenario is back-end LAN at the office, thus, a trusted
> network by definition.
NFS doesn't like when it looses connection to the server. The only
problems I had ever with NFS were because I forgot to unmout it before a
server restart or when I took a computer (laptop) off to another
network...
Otherwise it works well, esp. when mounted ro on the clients, however
for distfiles it might make sense to allow the clients download and save
tarballs that are not there yet ;), though I never used it with many
computer emerging/downloading same same stuff, so can't say if locking
etc works correctly...
And with NFS the clients won't duplicate the files in their own
distfiles directories ;)
yoyo
^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: [gentoo-user] The LIGHTEST web server (just for serving files)?
2011-11-12 13:11 ` YoYo Siska
@ 2011-11-12 13:23 ` Pandu Poluan
2011-11-14 10:05 ` J. Roeleveld
1 sibling, 0 replies; 22+ messages in thread
From: Pandu Poluan @ 2011-11-12 13:23 UTC (permalink / raw
To: gentoo-user
[-- Attachment #1: Type: text/plain, Size: 2264 bytes --]
On Nov 12, 2011 8:16 PM, "YoYo Siska" <yoyo@gl.ksp.sk> wrote:
>
> On Sat, Nov 12, 2011 at 07:40:08PM +0700, Pandu Poluan wrote:
> > On Nov 12, 2011 7:00 PM, "Mick" <michaelkintzios@gmail.com> wrote:
> > >
> > > I've been using boa just for this purpose for years:
> > >
> > > * www-servers/boa
> > > Available versions:
> > > ~ 0.94.14_rc21 "~x86 ~sparc ~mips ~ppc ~amd64"
[doc]
> > > Homepage: http://www.boa.org/
> > > Description: A very small and very fast http daemon.
> > >
> > > It can be easily locked down for internet facing roles.
> > >
> > > I've also used thttpd (you can throttle its bandwidth if that's
important
> > in
> > > your network), but it's probably more than required for this purpose:
> > >
> > > * www-servers/thttpd
> > > Available versions:
> > > 2.25b-r7 "amd64 ~hppa ~mips ppc sparc x86
> > ~x86-fbsd" [static]
> > > ~ 2.25b-r8 "~amd64 ~hppa ~mips ~ppc ~sparc ~x86
> > ~x86-fbsd"
> > > [static]
> > > Homepage: http://www.acme.com/software/thttpd/
> > > Description: Small and fast multiplexing webserver.
> >
> > Thanks for all the input!
> >
> > During my drive home, something hit my brain: why not have the 'master'
> > server share the distfiles dir via NFS?
> >
> > So, the question now becomes: what's the drawback/benefit of
NFS-sharing vs
> > HTTP-sharing? The scenario is back-end LAN at the office, thus, a
trusted
> > network by definition.
>
> NFS doesn't like when it looses connection to the server. The only
> problems I had ever with NFS were because I forgot to unmout it before a
> server restart or when I took a computer (laptop) off to another
> network...
> Otherwise it works well, esp. when mounted ro on the clients, however
> for distfiles it might make sense to allow the clients download and save
> tarballs that are not there yet ;), though I never used it with many
> computer emerging/downloading same same stuff, so can't say if locking
> etc works correctly...
>
> And with NFS the clients won't duplicate the files in their own
> distfiles directories ;)
Yes, that would be beneficial. But if NFS is as finicky as that, what's a
better way to share directories?
Rgds,
[-- Attachment #2: Type: text/html, Size: 3196 bytes --]
^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: [gentoo-user] The LIGHTEST web server (just for serving files)?
2011-11-12 13:11 ` YoYo Siska
2011-11-12 13:23 ` Pandu Poluan
@ 2011-11-14 10:05 ` J. Roeleveld
2011-11-14 10:10 ` microcai
1 sibling, 1 reply; 22+ messages in thread
From: J. Roeleveld @ 2011-11-14 10:05 UTC (permalink / raw
To: gentoo-user
On Sat, November 12, 2011 2:11 pm, YoYo Siska wrote:
> On Sat, Nov 12, 2011 at 07:40:08PM +0700, Pandu Poluan wrote:
>> During my drive home, something hit my brain: why not have the 'master'
>> server share the distfiles dir via NFS?
>>
>> So, the question now becomes: what's the drawback/benefit of NFS-sharing
>> vs
>> HTTP-sharing? The scenario is back-end LAN at the office, thus, a
>> trusted
>> network by definition.
>
> NFS doesn't like when it looses connection to the server. The only
> problems I had ever with NFS were because I forgot to unmout it before a
> server restart or when I took a computer (laptop) off to another
> network...
NFS-shares can work, but these need to be umounted before network goes.
If server goes, problems can occur there as well.
But that is true with any server/client filesharing. (CIFS/Samba, for
instance)
> Otherwise it works well, esp. when mounted ro on the clients, however
> for distfiles it might make sense to allow the clients download and save
> tarballs that are not there yet ;), though I never used it with many
> computer emerging/downloading same same stuff, so can't say if locking
> etc works correctly...
Locking works correctly, have had 5 machines share the same NFS-shared
distfiles and all downloading the source-files.
> And with NFS the clients won't duplicate the files in their own
> distfiles directories ;)
Big plus, for me :)
--
Joost
^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: [gentoo-user] The LIGHTEST web server (just for serving files)?
2011-11-14 10:05 ` J. Roeleveld
@ 2011-11-14 10:10 ` microcai
2011-11-14 14:36 ` Michael Mol
0 siblings, 1 reply; 22+ messages in thread
From: microcai @ 2011-11-14 10:10 UTC (permalink / raw
To: gentoo-user
[-- Attachment #1: Type: text/plain, Size: 1645 bytes --]
http://code.google.com/p/bashttpd/
run with systemd or xinetd
于 2011年11月14日 18:05, J. Roeleveld 写道:
> On Sat, November 12, 2011 2:11 pm, YoYo Siska wrote:
>> On Sat, Nov 12, 2011 at 07:40:08PM +0700, Pandu Poluan wrote:
>>> During my drive home, something hit my brain: why not have the 'master'
>>> server share the distfiles dir via NFS?
>>>
>>> So, the question now becomes: what's the drawback/benefit of NFS-sharing
>>> vs
>>> HTTP-sharing? The scenario is back-end LAN at the office, thus, a
>>> trusted
>>> network by definition.
>>
>> NFS doesn't like when it looses connection to the server. The only
>> problems I had ever with NFS were because I forgot to unmout it before a
>> server restart or when I took a computer (laptop) off to another
>> network...
>
> NFS-shares can work, but these need to be umounted before network goes.
> If server goes, problems can occur there as well.
> But that is true with any server/client filesharing. (CIFS/Samba, for
> instance)
>
>> Otherwise it works well, esp. when mounted ro on the clients, however
>> for distfiles it might make sense to allow the clients download and save
>> tarballs that are not there yet ;), though I never used it with many
>> computer emerging/downloading same same stuff, so can't say if locking
>> etc works correctly...
>
> Locking works correctly, have had 5 machines share the same NFS-shared
> distfiles and all downloading the source-files.
>
>> And with NFS the clients won't duplicate the files in their own
>> distfiles directories ;)
>
> Big plus, for me :)
>
> --
> Joost
>
>
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 490 bytes --]
^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: [gentoo-user] The LIGHTEST web server (just for serving files)?
2011-11-14 10:10 ` microcai
@ 2011-11-14 14:36 ` Michael Mol
2011-11-14 22:01 ` v_2e
2011-11-15 18:40 ` [gentoo-user] " Steven J Long
0 siblings, 2 replies; 22+ messages in thread
From: Michael Mol @ 2011-11-14 14:36 UTC (permalink / raw
To: gentoo-user
Isn't there a kernelland HTTP server? ISTR seeing the option. I don't
know anything about it, though.
On Mon, Nov 14, 2011 at 5:10 AM, microcai <microcai@fedoraproject.org> wrote:
>
> http://code.google.com/p/bashttpd/
>
> run with systemd or xinetd
>
>
>
> 于 2011年11月14日 18:05, J. Roeleveld 写道:
>> On Sat, November 12, 2011 2:11 pm, YoYo Siska wrote:
>>> On Sat, Nov 12, 2011 at 07:40:08PM +0700, Pandu Poluan wrote:
>>>> During my drive home, something hit my brain: why not have the 'master'
>>>> server share the distfiles dir via NFS?
>>>>
>>>> So, the question now becomes: what's the drawback/benefit of NFS-sharing
>>>> vs
>>>> HTTP-sharing? The scenario is back-end LAN at the office, thus, a
>>>> trusted
>>>> network by definition.
>>>
>>> NFS doesn't like when it looses connection to the server. The only
>>> problems I had ever with NFS were because I forgot to unmout it before a
>>> server restart or when I took a computer (laptop) off to another
>>> network...
>>
>> NFS-shares can work, but these need to be umounted before network goes.
>> If server goes, problems can occur there as well.
>> But that is true with any server/client filesharing. (CIFS/Samba, for
>> instance)
>>
>>> Otherwise it works well, esp. when mounted ro on the clients, however
>>> for distfiles it might make sense to allow the clients download and save
>>> tarballs that are not there yet ;), though I never used it with many
>>> computer emerging/downloading same same stuff, so can't say if locking
>>> etc works correctly...
>>
>> Locking works correctly, have had 5 machines share the same NFS-shared
>> distfiles and all downloading the source-files.
>>
>>> And with NFS the clients won't duplicate the files in their own
>>> distfiles directories ;)
>>
>> Big plus, for me :)
>>
>> --
>> Joost
>>
>>
>
>
>
--
:wq
^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: [gentoo-user] The LIGHTEST web server (just for serving files)?
2011-11-14 14:36 ` Michael Mol
@ 2011-11-14 22:01 ` v_2e
2011-11-15 18:40 ` [gentoo-user] " Steven J Long
1 sibling, 0 replies; 22+ messages in thread
From: v_2e @ 2011-11-14 22:01 UTC (permalink / raw
To: gentoo-user
There is a very small web server called "thttpd" which is very
lightweight and lets start serving files very quickly.
It runs on my home router machine with an old Pentium CPU and several
megabytes of RAM and seems to consume about 500 kb of it.
Regards,
Vladimir
On Mon, 14 Nov 2011 09:36:22 -0500
Michael Mol <mikemol@gmail.com> wrote:
> Isn't there a kernelland HTTP server? ISTR seeing the option. I don't
> know anything about it, though.
>
> On Mon, Nov 14, 2011 at 5:10 AM, microcai
> <microcai@fedoraproject.org> wrote:
> >
> > http://code.google.com/p/bashttpd/
> >
> > run with systemd or xinetd
> >
> >
> >
> > 于 2011年11月14日 18:05, J. Roeleveld 写道:
> >> On Sat, November 12, 2011 2:11 pm, YoYo Siska wrote:
> >>> On Sat, Nov 12, 2011 at 07:40:08PM +0700, Pandu Poluan wrote:
> >>>> During my drive home, something hit my brain: why not have the
> >>>> 'master' server share the distfiles dir via NFS?
> >>>>
> >>>> So, the question now becomes: what's the drawback/benefit of
> >>>> NFS-sharing vs
> >>>> HTTP-sharing? The scenario is back-end LAN at the office, thus, a
> >>>> trusted
> >>>> network by definition.
> >>>
> >>> NFS doesn't like when it looses connection to the server. The only
> >>> problems I had ever with NFS were because I forgot to unmout it
> >>> before a server restart or when I took a computer (laptop) off
> >>> to another network...
> >>
> >> NFS-shares can work, but these need to be umounted before network
> >> goes. If server goes, problems can occur there as well.
> >> But that is true with any server/client filesharing. (CIFS/Samba,
> >> for instance)
> >>
> >>> Otherwise it works well, esp. when mounted ro on the clients,
> >>> however for distfiles it might make sense to allow the clients
> >>> download and save tarballs that are not there yet ;), though I
> >>> never used it with many computer emerging/downloading same same
> >>> stuff, so can't say if locking etc works correctly...
> >>
> >> Locking works correctly, have had 5 machines share the same
> >> NFS-shared distfiles and all downloading the source-files.
> >>
> >>> And with NFS the clients won't duplicate the files in their own
> >>> distfiles directories ;)
> >>
> >> Big plus, for me :)
> >>
> >> --
> >> Joost
> >>
> >>
> >
> >
> >
>
>
>
> --
> :wq
>
>
-----
<v_2e@ukr.net>
^ permalink raw reply [flat|nested] 22+ messages in thread* [gentoo-user] Re: The LIGHTEST web server (just for serving files)?
2011-11-14 14:36 ` Michael Mol
2011-11-14 22:01 ` v_2e
@ 2011-11-15 18:40 ` Steven J Long
1 sibling, 0 replies; 22+ messages in thread
From: Steven J Long @ 2011-11-15 18:40 UTC (permalink / raw
To: gentoo-user
Michael Mol wrote:
> Isn't there a kernelland HTTP server? ISTR seeing the option. I don't
> know anything about it, though.
>
Yeah there was; as I recall it got removed a while back.
Google got me:
http://en.wikipedia.org/wiki/TUX_web_server
and khttpd at: http://www.fenrus.demon.nl/
..both of which appear dead. I couldn't find any mention of http in my
kernel config either.
We use lighttpd for our dev stuff; I guess it's that, nginx or thttpd,
last of which doesn't do fastCGI, so might be the best for this
purpose.
http://en.wikipedia.org/wiki/Comparison_of_web_server_software
..might prove helpful.
--
#friendly-coders -- We're friendly, but we're not /that/ friendly ;-)
^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: [gentoo-user] The LIGHTEST web server (just for serving files)?
2011-11-12 12:40 ` Pandu Poluan
2011-11-12 13:01 ` Mick
2011-11-12 13:11 ` YoYo Siska
@ 2011-11-12 13:14 ` Florian Philipp
2011-11-12 13:33 ` Pandu Poluan
2011-11-12 17:05 ` [gentoo-user] " Holger Hoffstaette
2011-11-12 22:01 ` [gentoo-user] " Neil Bothwick
3 siblings, 2 replies; 22+ messages in thread
From: Florian Philipp @ 2011-11-12 13:14 UTC (permalink / raw
To: gentoo-user
[-- Attachment #1: Type: text/plain, Size: 1251 bytes --]
Am 12.11.2011 13:40, schrieb Pandu Poluan:
>
> During my drive home, something hit my brain: why not have the 'master'
> server share the distfiles dir via NFS?
>
> So, the question now becomes: what's the drawback/benefit of NFS-sharing
> vs HTTP-sharing? The scenario is back-end LAN at the office, thus, a
> trusted network by definition.
>
> Rgds,
>
How exactly had you planned to share distfiles? You didn't want to
mirror everything from the offical mirrors, did you? I'm not perfectly
sure how portage handles a mirror that occasionally returns 404 errors
but I think I've seen it fall back to the official mirrors in that case.
Anyway, making educated guesses about what should be on your own mirror
is probably a bit ineffective unless you have a very homogeneous
environment.
What I think you /should/ have wanted is a proxy specifically configured
to cache very large files. net-proxy/http-replicator has been made
specifically for Gentoo distfiles.
NFS has the advantage that it doesn't duplicate distfiles locally on all
machines. It is also easier to set up. Disadvantages? I'm unsure how
portage will handle cases when two machines fetch the same file at the
same time.
Regards,
Florian Philipp
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 262 bytes --]
^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: [gentoo-user] The LIGHTEST web server (just for serving files)?
2011-11-12 13:14 ` [gentoo-user] " Florian Philipp
@ 2011-11-12 13:33 ` Pandu Poluan
2011-11-12 17:05 ` [gentoo-user] " Holger Hoffstaette
1 sibling, 0 replies; 22+ messages in thread
From: Pandu Poluan @ 2011-11-12 13:33 UTC (permalink / raw
To: gentoo-user
[-- Attachment #1: Type: text/plain, Size: 1729 bytes --]
On Nov 12, 2011 8:20 PM, "Florian Philipp" <lists@binarywings.net> wrote:
>
> Am 12.11.2011 13:40, schrieb Pandu Poluan:
> >
> > During my drive home, something hit my brain: why not have the 'master'
> > server share the distfiles dir via NFS?
> >
> > So, the question now becomes: what's the drawback/benefit of NFS-sharing
> > vs HTTP-sharing? The scenario is back-end LAN at the office, thus, a
> > trusted network by definition.
> >
> > Rgds,
> >
>
> How exactly had you planned to share distfiles? You didn't want to
> mirror everything from the offical mirrors, did you? I'm not perfectly
> sure how portage handles a mirror that occasionally returns 404 errors
> but I think I've seen it fall back to the official mirrors in that case.
Yes, portage (at least, 2.2) automatically use the next mirror in the list.
> Anyway, making educated guesses about what should be on your own mirror
> is probably a bit ineffective unless you have a very homogeneous
> environment.
>
> What I think you /should/ have wanted is a proxy specifically configured
> to cache very large files. net-proxy/http-replicator has been made
> specifically for Gentoo distfiles.
>
I had planned on having a script peruse the log file, looking for which box
got a 404, and 1 hour later try to move the file using scp from that box
into the common local subrepo.
But http-replicator sounds mighty better :-)
> NFS has the advantage that it doesn't duplicate distfiles locally on all
> machines. It is also easier to set up. Disadvantages? I'm unsure how
> portage will handle cases when two machines fetch the same file at the
> same time.
I can always stagger the time my boxes fetch the distfiles. That should
prevent locking problems.
Rgds,
[-- Attachment #2: Type: text/html, Size: 2105 bytes --]
^ permalink raw reply [flat|nested] 22+ messages in thread
* [gentoo-user] Re: The LIGHTEST web server (just for serving files)?
2011-11-12 13:14 ` [gentoo-user] " Florian Philipp
2011-11-12 13:33 ` Pandu Poluan
@ 2011-11-12 17:05 ` Holger Hoffstaette
1 sibling, 0 replies; 22+ messages in thread
From: Holger Hoffstaette @ 2011-11-12 17:05 UTC (permalink / raw
To: gentoo-user
On Sat, 12 Nov 2011 14:14:33 +0100, Florian Philipp wrote:
> Disadvantages? I'm unsure how portage will handle cases when two
> machines fetch the same file at the same time.
In my experience concurrent/overlapping fetches work just fine, thanks to
portage's lock file. The second potential downloader just waits or does
something else in the meantime.
-h
^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: [gentoo-user] The LIGHTEST web server (just for serving files)?
2011-11-12 12:40 ` Pandu Poluan
` (2 preceding siblings ...)
2011-11-12 13:14 ` [gentoo-user] " Florian Philipp
@ 2011-11-12 22:01 ` Neil Bothwick
2011-11-13 0:18 ` William Kenworthy
3 siblings, 1 reply; 22+ messages in thread
From: Neil Bothwick @ 2011-11-12 22:01 UTC (permalink / raw
To: gentoo-user
[-- Attachment #1: Type: text/plain, Size: 964 bytes --]
On Sat, 12 Nov 2011 19:40:08 +0700, Pandu Poluan wrote:
> During my drive home, something hit my brain: why not have the 'master'
> server share the distfiles dir via NFS?
No reason at all, I've been doing it for years without a single
problem.
> So, the question now becomes: what's the drawback/benefit of
> NFS-sharing vs HTTP-sharing? The scenario is back-end LAN at the
> office, thus, a trusted network by definition.
The benefit is that everything is centralised. With an HTTP proxy, you
still have to download from the server to each client. The only drawback
that I experience is that if several packages use the same, large source
file, as so many of the KDE packages do, you are repeatedly pulling the
same file over the network, which is a little slower.
--
Neil Bothwick
Keyboard: (n.) a device used by programmers to write software for a mouse
or joystick and by operators for playing games such as 'word processing.'
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 198 bytes --]
^ permalink raw reply [flat|nested] 22+ messages in thread* Re: [gentoo-user] The LIGHTEST web server (just for serving files)?
2011-11-12 22:01 ` [gentoo-user] " Neil Bothwick
@ 2011-11-13 0:18 ` William Kenworthy
0 siblings, 0 replies; 22+ messages in thread
From: William Kenworthy @ 2011-11-13 0:18 UTC (permalink / raw
To: gentoo-user
Its been solved in the past ... designed for just this purpose:
moriah ~ # esearch http-replicator
[ Results for search key : http-replicator ]
[ Applications found : 1 ]
* net-proxy/http-replicator
Latest version available: 3.0-r2
Latest version installed: 3.0-r2
Size of downloaded files: 38 kB
Homepage: http://sourceforge.net/projects/http-replicator
Description: Proxy cache for Gentoo packages
License: GPL-2
moriah ~ #
I chain them together (two levels, avoiding expensive download costs) so
the remote site doesnt have it in its cache, upstream is the master
cache, which if it doesnt have it will fetch from the repo. You can
specify what port it runs on, and then use the http_proxy entry in
make.conf to point the clients to it thus avoiding port 80 and any
existing webserver. Handles concurrent fetches transparently. Overall, I
have found it preferable to NFS which has been a bit flaky at times in
the past.
Recommended!
BillK
On Sat, 2011-11-12 at 22:01 +0000, Neil Bothwick wrote:
> On Sat, 12 Nov 2011 19:40:08 +0700, Pandu Poluan wrote:
>
> > During my drive home, something hit my brain: why not have the 'master'
> > server share the distfiles dir via NFS?
>
> No reason at all, I've been doing it for years without a single
> problem.
>
> > So, the question now becomes: what's the drawback/benefit of
> > NFS-sharing vs HTTP-sharing? The scenario is back-end LAN at the
> > office, thus, a trusted network by definition.
>
> The benefit is that everything is centralised. With an HTTP proxy, you
> still have to download from the server to each client. The only drawback
> that I experience is that if several packages use the same, large source
> file, as so many of the KDE packages do, you are repeatedly pulling the
> same file over the network, which is a little slower.
>
>
^ permalink raw reply [flat|nested] 22+ messages in thread