[gentoo-user] Load Balancing / Redundant Network ?

[gentoo-user] Load Balancing / Redundant Network ?

[Mal Herring]

Hi List,
Been looking on forums.gentoo.org looking for a way to pair two NIC's so
that should the primary fail, the secondary takes over...

The situation is this:

8 HP DL380G4's all running Gentoo moving into a data centre rack, the
firewall is currently a Netscreen(Juniper) 25 and another will be added
to give us Active/Passive failover on the firewalls.

Redundant switches will be added behind the firewalls to ensure that is
not a single point of failure, all the DL380 have two NIC's in them and
currently only one is active.

All boxes covered with HP on a 24x7x4 Carepaq, redundant power supplies
to be added - DC is a N+1 so fine there...

Biggest issue I am facing and becoming lost on is teaming the NIC's so I
can connect NIC1 to Switch 1 and NIC2 to switch 2 etc...

Can anyone help ?

Ta

Mal

-- 
gentoo-user@gentoo.org mailing list

RE: [gentoo-user] Load Balancing / Redundant Network ?

[Olaf Niermann]

Hi Mal,

> Hi List,
> Been looking on forums.gentoo.org looking for a way to pair 
> two NIC's so
> that should the primary fail, the secondary takes over...


Maybe you can try a lookup onto 'heartbeat' (emerge -s heartbeat) allthough
it might be overloaded for your needs and I personally just use it for
failover between different physical servers.

Bye,
Olaf Niermann

-- 
gentoo-user@gentoo.org mailing list

RE: [gentoo-user] Load Balancing / Redundant Network ?

[A. Khattri]

On Wed, 14 Sep 2005, Olaf Niermann wrote:

> Maybe you can try a lookup onto 'heartbeat' (emerge -s heartbeat) allthough
> it might be overloaded for your needs and I personally just use it for
> failover between different physical servers.

Just wanted to add that heartbeat is part of the HA Linux project - maybe
you'll find useful stuff at their site or on their mailing lists?

http://www.linux-ha.org/

Shame we dont have anything like CARP for Linux yet... (unless someone
knows better?).


-- 

-- 
gentoo-user@gentoo.org mailing list

Re: [gentoo-user] Load Balancing / Redundant Network ?

[Mike Williams]

On Thursday 15 September 2005 19:08, A. Khattri wrote:
> Shame we dont have anything like CARP for Linux yet... (unless someone
> knows better?).

UCARP, but it's fundamentally flawed, as iptables has no method to keep state 
tables in sync between machines.

Personally, I prefer to have iptables set up to allow traffic over connections 
that are already established.
This way you can swap firewalls (and update arp), reboot them, etc, without 
interupting the connection. Far from perfect, but it works to a degree.

-- 
Mike Williams
-- 
gentoo-user@gentoo.org mailing list

Re: [gentoo-user] Load Balancing / Redundant Network ?

[A. Khattri]

On Thu, 15 Sep 2005, Mike Williams wrote:

> On Thursday 15 September 2005 19:08, A. Khattri wrote:
> > Shame we dont have anything like CARP for Linux yet... (unless someone
> > knows better?).
>
> UCARP, but it's fundamentally flawed, as iptables has no method to keep state
> tables in sync between machines.

Yeah, we really need something like pfsync.

> Personally, I prefer to have iptables set up to allow traffic over connections
> that are already established.
> This way you can swap firewalls (and update arp), reboot them, etc, without
> interupting the connection. Far from perfect, but it works to a degree.

Yep, this is what I use.


-- 

-- 
gentoo-user@gentoo.org mailing list

Re: [gentoo-user] Load Balancing / Redundant Network ?

[Bastian Balthazar Bux]

Mal Herring wrote:
> Hi List,
> Been looking on forums.gentoo.org looking for a way to pair two NIC's so
> that should the primary fail, the secondary takes over...
> 
> The situation is this:
> 
> 8 HP DL380G4's all running Gentoo moving into a data centre rack, the
> firewall is currently a Netscreen(Juniper) 25 and another will be added
> to give us Active/Passive failover on the firewalls.
> 
> Redundant switches will be added behind the firewalls to ensure that is
> not a single point of failure, all the DL380 have two NIC's in them and
> currently only one is active.
> 
> All boxes covered with HP on a 24x7x4 Carepaq, redundant power supplies
> to be added - DC is a N+1 so fine there...
> 
> Biggest issue I am facing and becoming lost on is teaming the NIC's so I
> can connect NIC1 to Switch 1 and NIC2 to switch 2 etc...
> 
> Can anyone help ?
> 
> Ta
> 
> Mal
> 
rather dirty but what about put this script in a "*/5" minutes cron ?

------- check_connectivity --------
#! /bin/sh

# counter of failed ping
FAILED_PING=0

# after how much time the script is forced to exit
# warning %s is a gnu extension to date
STOP_TIME=$(( $(date +%s) + 300 -2 ))

# host used for chech
REACHABLE_HOSTS="192.168.1.123 192.168.1.124"

# seconds to wait before to switch (approx)
# left operand = tentatives
# right operand = no of hosts in REACHABLE_HOSTS
TTW=$(( 5 * 2 ))

# if this become "1" call houston
SWITCHED=0

PING_CMD="arping -c1 -Ieth0 -w1 -q"


while [[ "$(date +%s)" -lt "${STOP_TIME}" ]] && [[ ${SWITCHED} -eq 0 ]]
do
        for host in ${REACHABLE_HOSTS}
        do
                if ($PING_CMD $host) ; then
                        FAILED_PING=1
                else
                        FAILED_PING=$(( FAILED_PING + 1 ))
                fi
        done

        if [[ ${FAILED_PING} -gt ${TTW} ]] ; then
                SWITCHED=1
                # put the replace interface fx here
                echo "switch_interface"
        fi
        sleep 1
done
------- check_connectivity --------

Still todo are:
-  the switch_interface() interface function something like relink
   /etc/conf.d/net and restart the net
- the "return to normality" script that once eth0 work again for some
  seconds switch back
- A lock that prevent the script to start if the interface is switched
  (and instead run the check for "normal" status)
- a mail to the admin

P.S. has never checked in real life, do your checks before to use it

Cheers,
Francesco R.
-- 
gentoo-user@gentoo.org mailing list

Re: [gentoo-user] Load Balancing / Redundant Network ?

[kashani]

Mal Herring wrote:
> Hi List,
> Been looking on forums.gentoo.org looking for a way to pair two NIC's so
> that should the primary fail, the secondary takes over...
> 
> The situation is this:
> 
> 8 HP DL380G4's all running Gentoo moving into a data centre rack, the
> firewall is currently a Netscreen(Juniper) 25 and another will be added
> to give us Active/Passive failover on the firewalls.
> 
> Redundant switches will be added behind the firewalls to ensure that is
> not a single point of failure, all the DL380 have two NIC's in them and
> currently only one is active.
> 
> All boxes covered with HP on a 24x7x4 Carepaq, redundant power supplies
> to be added - DC is a N+1 so fine there...
> 
> Biggest issue I am facing and becoming lost on is teaming the NIC's so I
> can connect NIC1 to Switch 1 and NIC2 to switch 2 etc...

I've got three scenarios which may work, but none are close to perfect.

1. Take the hit.
	Build your setup with enough extra capacity that losing a switch and 
all the machines behind it won't cause downtime. The pros are it's easy, 
failover isn't complicated, but you may need more machines than you have 
now.

2. Routers routers everywhere
	Run a routing daemon on each server. Real interfaces become /30 links 
and the server daemons run on a loopback address. The end effect is that 
there are two paths to your loopback, OSPF converges quickly, and src 
IP's and arp addresses never change. The cons is that it's complicated 
and there are a thousand and one ways to shoot yourself in the foot.

3. local VRRP
	Set eth0 as .2 and eth1 as .3 and then have them collude to make .1. 
Assuming things fail in such a way that eth0 fails completely then .3 
should take over as the primary interface and take .1 with it. Better 
machine use, less complicated than OSPF everywhere, but I'm not sure 
it'll work. I suspect that eth0 and eth1 will stay up as far as the 
local machine is concerned and won't move the virtual IP around.

None of these ideas are great, but might get you thinking about 
something that might work.

kashani
-- 
gentoo-user@gentoo.org mailing list

Re: [gentoo-user] Load Balancing / Redundant Network ?

[Scott Storck]

Hallo Mal,

Mal Herring schrieb:

>Hi List,
>Been looking on forums.gentoo.org looking for a way to pair two NIC's so
>that should the primary fail, the secondary takes over...
>  
>
I use the LACP protocol ( IEEE 802.1ad) between my switches and my servers,
it is one of the bonding modes offered by the bonding driver.

LACP is that multiple links between the switch and the server are used 
as one logical link.
If one dies, then the others are still used.
I have tested it using setups of 2 and 4 gigabit links, and I have had 
no problems.

I have only tested it with HP procurve switches, with which I have had 
no problems at all.
HP refers to it as LACP and/or IEE 802.1ad, but I have heard that some 
other switch manufactures call it other names...
I can also recommending checking out the bonding driver in general.

MfG,
Scott

-- 
gentoo-user@gentoo.org mailing list

Re: [gentoo-user] Load Balancing / Redundant Network ?

[kashani]

Scott Storck wrote:
> LACP is that multiple links between the switch and the server are used 
> as one logical link.
> If one dies, then the others are still used.
> I have tested it using setups of 2 and 4 gigabit links, and I have had 
> no problems.
>

In a bonding situation don't both NIC's need to be plugged into the same 
switch or have things changed in the technology recently? I think the 
failure that the original author is trying to plan for is a switch 
failure not a NIC failure.

kashani
-- 
gentoo-user@gentoo.org mailing list

RE: [gentoo-user] Load Balancing / Redundant Network ?

[Mal Herring]

> In a bonding situation don't both NIC's need to be plugged 
> into the same switch or have things changed in the technology 
> recently? I think the failure that the original author is 
> trying to plan for is a switch failure not a NIC failure.

Thanks for all the replies, I was pointed in the direction of the
bonding how-to and this works perfectly on the box, I am now hoping that
I can use two switches for resilience and then will be using two
firewalls as well, so I am trying to plan for all possible failures...

Can the eth ports when using the bonding driver get plugged into
different switches ?

Thanks in advance.

Mal

-- 
gentoo-user@gentoo.org mailing list

Re: [gentoo-user] Load Balancing / Redundant Network ?

[Yoann Pannier]

Mal Herring wrote, On 09/19/2005 09:10 AM:
> Can the eth ports when using the bonding driver get plugged into
> different switches ?

Yes.

from /usr/src/linux/Documentation/networking/bonding.txt:


12.2 High Availability in a Multiple Switch Topology
----------------------------------------------------

             |                                     |
             |port3                           port3|
       +-----+----+                          +-----+----+
       |          |port2       ISL      port2|          |
       | switch A +--------------------------+ switch B |
       |          |                          |          |
       +-----+----+                          +-----++---+
             |port1                           port1|
             |             +-------+               |
             +-------------+ host1 +---------------+
                      eth0 +-------+ eth1

-- 
Yoann Pannier
-- 
gentoo-user@gentoo.org mailing list