[gentoo-user] Hosted server as distcc machine

[gentoo-user] Hosted server as distcc machine

[Grant]

Lately I've been thinking about making my remotely hosted server the
distcc machine for all of my systems.  Is there anything wrong with
making a remote machine like that the distcc system?  It's the fastest
system I've got by far.

- Grant

-- 
gentoo-user@gentoo.org mailing list

Re: [gentoo-user] Hosted server as distcc machine

[Boyd Stephen Smith Jr.]

On Monday 20 March 2006 22:25, Grant <emailgrant@gmail.com> wrote about 
'[gentoo-user] Hosted server as distcc machine':
> Is there anything wrong with
> making a remote machine [a] distcc system?

Not really, but you do need to realize that distcc doesn't guarantee that 
jobs will be sent to the remote machines and will not prevent jobs from 
being run locally.  If there are not enough distcc hosts to support the 
number of jobs being run, or the network is down to 1 or more, or other 
such issues, you might end up having too many compiles being run locally.  
This applies even if you put something like localhost/2 in your distcc 
hosts -- when distcc runs out of hosts it unconditionally uses local 
compilation.

Also, distccd is a wide-open security hole: there's little to no 
restriction on what a client can run on the host, and AFAIK only 
ip/host-based restrictions on who can connect.  A few, well-placed IP 
packets with spoofed sources could theoretically result in a rooted box 
(depending on other security features like firewalls, syn cookies, 
restricted shells, chroot jails, and presence of local privilege 
escalation exploits).

It's probably better to use distcc over ssh, using an ssh-agent and PKI 
authentication.  That does involve giving shell access to an account, but 
you probably already have an account that will work. :)  Unfortunately, 
this removes the host's ability to limit simultaneous distcc jobs AFAIK.  
It also makes it quite a bit harder to distcc from cron, but most of the 
time that shouldn't be an issue.

-- 
"If there's one thing we've established over the years,
it's that the vast majority of our users don't have the slightest
clue what's best for them in terms of package stability."
-- Gentoo Developer Ciaran McCreesh
-- 
gentoo-user@gentoo.org mailing list

Re: [gentoo-user] Hosted server as distcc machine

[Grant]

> > Is there anything wrong with
> > making a remote machine [a] distcc system?
>
> Not really, but you do need to realize that distcc doesn't guarantee that
> jobs will be sent to the remote machines and will not prevent jobs from
> being run locally.  If there are not enough distcc hosts to support the
> number of jobs being run, or the network is down to 1 or more, or other
> such issues, you might end up having too many compiles being run locally.
> This applies even if you put something like localhost/2 in your distcc
> hosts -- when distcc runs out of hosts it unconditionally uses local
> compilation.

Good to know for sure.

> Also, distccd is a wide-open security hole: there's little to no
> restriction on what a client can run on the host, and AFAIK only
> ip/host-based restrictions on who can connect.  A few, well-placed IP
> packets with spoofed sources could theoretically result in a rooted box
> (depending on other security features like firewalls, syn cookies,
> restricted shells, chroot jails, and presence of local privilege
> escalation exploits).

Not good.  The remote machine I'm considering using distcc on is my
business's server.  I can't have break-ins there.

> It's probably better to use distcc over ssh, using an ssh-agent and PKI
> authentication.  That does involve giving shell access to an account, but
> you probably already have an account that will work. :)  Unfortunately,
> this removes the host's ability to limit simultaneous distcc jobs AFAIK.
> It also makes it quite a bit harder to distcc from cron, but most of the
> time that shouldn't be an issue.

So using distcc along with ssh and PKI would be sufficient to prevent
the rooted box mentioned above?  How would ssh and PKI be set up in
the workflow?  It isn't mentioned here:

http://www.gentoo.org/doc/en/distcc.xml

- Grant

-- 
gentoo-user@gentoo.org mailing list

Re: [gentoo-user] Hosted server as distcc machine

[Boyd Stephen Smith Jr.]

On Wednesday 22 March 2006 17:10, Grant <emailgrant@gmail.com> wrote about 
'Re: [gentoo-user] Hosted server as distcc machine':
> > > Is there anything wrong with
> > > making a remote machine [a] distcc system?
> >
> > Not really, but you do need to realize that distcc doesn't guarantee
> > that jobs will be sent to the remote machines and will not prevent
> > jobs from being run locally.
>
> Good to know for sure.

I was kinda surprised at the behavior.  I was sort of hoping distcc would 
just sort of "hold" the job until a slot opened up.  It's not a big deal, 
but something that you should be aware of.

> > Also, distccd is a wide-open security hole.
>
> Not good.  The remote machine I'm considering using distcc on is my
> business's server.  I can't have break-ins there.

Then I don't suggest distccd open to the internet (or any public network) 
-- it was never designed to be secure.  It's not a big target ATM for 
hackers AFAIK, but it's still a large vulnerability.

> > It's probably better to use distcc over ssh, using an ssh-agent and
> > PKI authentication.
>
> So using distcc along with ssh and PKI would be sufficient to prevent
> the rooted box mentioned above?

It won't /completely/ prevent it.  But, it will bring down the risk 
significantly.  Random attackers will no longer simply be able to spoof 
IPs, instead the attacker will have to have the username and private key 
of a user known to have shell access.  (Malicious users or a healthy dose 
of paranoia may force you to limit shell access anyway.)

> How would ssh and PKI be set up in
> the workflow?  It isn't mentioned here:
> http://www.gentoo.org/doc/en/distcc.xml

Yeah, I started with that document, too.  distcc/ssh/PKI is not covered, 
since it is moderately advanced.

1) On the server, set up the shell account that will use distcc via ssh.
2) On the client, generate the private key for that account and use 
ssh-copy-id to give the server the public key.  Please, please, give your 
private key a good passphrase -- I've seen some people use an empty 
passphrase!
3) On the server, if possible, disable password logins to force the use of 
the private key for that user.
4) On the client, add a line like shell_account@server to your 
distcc_hosts.  You can leave out the shell_account part if you want to log 
in to the server as the user invoking distcc, but you must include the @ 
since that's how distcc knows the host is accessed via ssh.  You can add 
a :port section if the server runs ssh on a port other than 22; You can 
add a /limit section (after or in lieu of the :port section) to have the 
client limit the number of distcc jobs that will be sent to the server
5) Prior to invoking distcc on the client, start an ssh-agent (I prefer the 
keychain "meta-"agent.) and optionally add your private key to the agent.  
(If you don't start an agent, each compile that goes to an ssh host will 
ask for a password -- very troublesome with parallel make; If you don't 
add your private key to the agent, you'll get prompted for the passphrase 
the first time you need a key -- still moderately troublesome.)

There is no need to run distccd on the server at all.  You /will/ need 
sshd.

Remember, since these are standard ssh connections, you'll limit the number 
of simultaneous jobs on the server by limiting the number of simultaneous 
ssh logins -- not by using any distccd settings.

As far as compile jobs from cron, I just don't suggest them.  If you /have/ 
to use them, have them compile locally.  If they /have/ to use your distcc 
hosts, you'll have to figure out some way to give your cron jobs access to 
your private key without compromising it's security -- not an easy feat.

-- 
"If there's one thing we've established over the years,
it's that the vast majority of our users don't have the slightest
clue what's best for them in terms of package stability."
-- Gentoo Developer Ciaran McCreesh
-- 
gentoo-user@gentoo.org mailing list

Re: [gentoo-user] Hosted server as distcc machine

[Grant]

> > > > Is there anything wrong with
> > > > making a remote machine [a] distcc system?
> > >
> > > Not really, but you do need to realize that distcc doesn't guarantee
> > > that jobs will be sent to the remote machines and will not prevent
> > > jobs from being run locally.
> >
> > Good to know for sure.
>
> I was kinda surprised at the behavior.  I was sort of hoping distcc would
> just sort of "hold" the job until a slot opened up.  It's not a big deal,
> but something that you should be aware of.
>
> > > Also, distccd is a wide-open security hole.
> >
> > Not good.  The remote machine I'm considering using distcc on is my
> > business's server.  I can't have break-ins there.
>
> Then I don't suggest distccd open to the internet (or any public network)
> -- it was never designed to be secure.  It's not a big target ATM for
> hackers AFAIK, but it's still a large vulnerability.
>
> > > It's probably better to use distcc over ssh, using an ssh-agent and
> > > PKI authentication.
> >
> > So using distcc along with ssh and PKI would be sufficient to prevent
> > the rooted box mentioned above?
>
> It won't /completely/ prevent it.  But, it will bring down the risk
> significantly.  Random attackers will no longer simply be able to spoof
> IPs, instead the attacker will have to have the username and private key
> of a user known to have shell access.  (Malicious users or a healthy dose
> of paranoia may force you to limit shell access anyway.)
>
> > How would ssh and PKI be set up in
> > the workflow?  It isn't mentioned here:
> > http://www.gentoo.org/doc/en/distcc.xml
>
> Yeah, I started with that document, too.  distcc/ssh/PKI is not covered,
> since it is moderately advanced.
>
> 1) On the server, set up the shell account that will use distcc via ssh.
> 2) On the client, generate the private key for that account and use
> ssh-copy-id to give the server the public key.  Please, please, give your
> private key a good passphrase -- I've seen some people use an empty
> passphrase!
> 3) On the server, if possible, disable password logins to force the use of
> the private key for that user.
> 4) On the client, add a line like shell_account@server to your
> distcc_hosts.  You can leave out the shell_account part if you want to log
> in to the server as the user invoking distcc, but you must include the @
> since that's how distcc knows the host is accessed via ssh.  You can add
> a :port section if the server runs ssh on a port other than 22; You can
> add a /limit section (after or in lieu of the :port section) to have the
> client limit the number of distcc jobs that will be sent to the server
> 5) Prior to invoking distcc on the client, start an ssh-agent (I prefer the
> keychain "meta-"agent.) and optionally add your private key to the agent.
> (If you don't start an agent, each compile that goes to an ssh host will
> ask for a password -- very troublesome with parallel make; If you don't
> add your private key to the agent, you'll get prompted for the passphrase
> the first time you need a key -- still moderately troublesome.)
>
> There is no need to run distccd on the server at all.  You /will/ need
> sshd.
>
> Remember, since these are standard ssh connections, you'll limit the number
> of simultaneous jobs on the server by limiting the number of simultaneous
> ssh logins -- not by using any distccd settings.
>
> As far as compile jobs from cron, I just don't suggest them.  If you /have/
> to use them, have them compile locally.  If they /have/ to use your distcc
> hosts, you'll have to figure out some way to give your cron jobs access to
> your private key without compromising it's security -- not an easy feat.

That's great, thank you!  It sounds like this would make the remote
distcc idea as secure as ssh and I won't have to worry about the fact
that distcc wasn't built with security in mind.  Is that right?  Also,
I'm the only user on all of my systems so it would be OK to use plain
ssh without PKI right?

- Grant

-- 
gentoo-user@gentoo.org mailing list

Re: [gentoo-user] Hosted server as distcc machine

[Boyd Stephen Smith Jr.]

On Friday 24 March 2006 13:25, Grant <emailgrant@gmail.com> wrote about 
'Re: [gentoo-user] Hosted server as distcc machine':
> > > > It's probably better to use distcc over ssh, using an ssh-agent
> > > > and PKI authentication.
> > > How would ssh and PKI be set up in
> > > the workflow?  It isn't mentioned here:
> > > http://www.gentoo.org/doc/en/distcc.xml
> >
> > 1) On the server, set up the shell account that will use distcc via
> > ssh.
> > 2) On the client, generate the private key for that account and 
> > use ssh-copy-id to give the server the public key.
> > 3) On the server, if possible, disable password logins to force the
> > use of the private key for that user.
> > 4) On the client, add a line like shell_account@server to your
> > distcc_hosts.
> > 5) Prior to invoking distcc on the client, start 
> > an ssh-agent (I prefer the keychain "meta-"agent.) and optionally add
> > your private key to the agent. (If you don't start an agent, each
> > compile that goes to an ssh host will ask for a password -- very
> > troublesome with parallel make; If you don't add your private key to
> > the agent, you'll get prompted for the passphrase the first time you
> > need a key -- still moderately troublesome.)
> >
> > There is no need to run distccd on the server at all.  You /will/ need
> > sshd.
>
> It sounds like this would make the remote
> distcc idea as secure as ssh and I won't have to worry about the fact
> that distcc wasn't built with security in mind.  Is that right?

Yes.  Since you aren't running the distccd server it's lack of security is 
not concern for you.  You'll be depending on the security of ssh.  While 
not completely spotless (e.g. the zlib vulnerability bit openssh) it was, 
at least, designed with security in mind.

> Also, 
> I'm the only user on all of my systems so it would be OK to use plain
> ssh without PKI right?

Unfortunately, no.  Not because it's less secure (though, it might be 
depending on the strength of your passwords vs passphrases), but because 
there's no such thing (AFAIK) as an ssh-password-agent.  This means that 
each compile job has to ask you for the password -- that's not gonna be 
real useful, most likely.  See the parenthetical notes at the end of step 
5.

-- 
"If there's one thing we've established over the years,
it's that the vast majority of our users don't have the slightest
clue what's best for them in terms of package stability."
-- Gentoo Developer Ciaran McCreesh
-- 
gentoo-user@gentoo.org mailing list

Re: [gentoo-user] Hosted server as distcc machine

[Grant]

> > > > > It's probably better to use distcc over ssh, using an ssh-agent
> > > > > and PKI authentication.
> > > > How would ssh and PKI be set up in
> > > > the workflow?  It isn't mentioned here:
> > > > http://www.gentoo.org/doc/en/distcc.xml
> > >
> > > 1) On the server, set up the shell account that will use distcc via
> > > ssh.
> > > 2) On the client, generate the private key for that account and
> > > use ssh-copy-id to give the server the public key.
> > > 3) On the server, if possible, disable password logins to force the
> > > use of the private key for that user.
> > > 4) On the client, add a line like shell_account@server to your
> > > distcc_hosts.
> > > 5) Prior to invoking distcc on the client, start
> > > an ssh-agent (I prefer the keychain "meta-"agent.) and optionally add
> > > your private key to the agent. (If you don't start an agent, each
> > > compile that goes to an ssh host will ask for a password -- very
> > > troublesome with parallel make; If you don't add your private key to
> > > the agent, you'll get prompted for the passphrase the first time you
> > > need a key -- still moderately troublesome.)
> > >
> > > There is no need to run distccd on the server at all.  You /will/ need
> > > sshd.
> >
> > It sounds like this would make the remote
> > distcc idea as secure as ssh and I won't have to worry about the fact
> > that distcc wasn't built with security in mind.  Is that right?
>
> Yes.  Since you aren't running the distccd server it's lack of security is
> not concern for you.  You'll be depending on the security of ssh.  While
> not completely spotless (e.g. the zlib vulnerability bit openssh) it was,
> at least, designed with security in mind.

Nice.

> > Also,
> > I'm the only user on all of my systems so it would be OK to use plain
> > ssh without PKI right?
>
> Unfortunately, no.  Not because it's less secure (though, it might be
> depending on the strength of your passwords vs passphrases), but because
> there's no such thing (AFAIK) as an ssh-password-agent.  This means that
> each compile job has to ask you for the password -- that's not gonna be
> real useful, most likely.  See the parenthetical notes at the end of step
> 5.

So you're saying if I don't use PKI, the remote system is going to
prompt me for a password after I'm already logged in?  You say "each
compile that goes to an ssh host will ask for a password".  At what
point in the emerge process does this happen?

- Grant

-- 
gentoo-user@gentoo.org mailing list

Re: [gentoo-user] Hosted server as distcc machine

[Zac Slade]

On Monday 27 March 2006 12:16, Grant wrote:
> So you're saying if I don't use PKI, the remote system is going to
> prompt me for a password after I'm already logged in?  You say "each
> compile that goes to an ssh host will ask for a password".  At what
> point in the emerge process does this happen?
Anytime you dipatch a job to another system.  If you are using ssh to send 
your jobs (instead of distccd) you will be prompted for a password for every 
job you send to the remote host.  That is of course until distcc realises it 
can't communicate and will then fall back to sending the jobs to localhost.  
If you have a lot of jobs you are sending to distcc then you will see a huge 
increase in your load average as a result.

-- 
Zac Slade
krakrjak@volumehost.net
ICQ:1415282 YM:krakrjak AIM:ttyp99
-- 
gentoo-user@gentoo.org mailing list