[gentoo-user] IP Load Sharing - Per Packet Load Balancing (Linux router)

[gentoo-user] IP Load Sharing - Per Packet Load Balancing (Linux router)

[Nick Khamis]

Hello everyone,

I am looking to put together a linux router for small business, and
was wondering if there was anything the suite (using quagga etc..)
that would allow for load balancing of regular dsl links. Kind of like
cisco with fast ethernet 0,1 and ip sef. If outgoing and incoming
traffic could be balanced, it would be great!

Kind Regards,

Nick.

[gentoo-user] Re: IP Load Sharing - Per Packet Load Balancing (Linux router)

[Nick Khamis]

I missed out some crusial info in my last email. As mentioned this
would be two separate DSL services, connected using separate bridges.
I think I am describing more of a link aggregation or bonding....

Also assuming that the service providers support bonding of the links....


N.

On 5/25/13, Nick Khamis <symack@gmail.com> wrote:
> Hello everyone,
>
> I am looking to put together a linux router for small business, and
> was wondering if there was anything the suite (using quagga etc..)
> that would allow for load balancing of regular dsl links. Kind of like
> cisco with fast ethernet 0,1 and ip sef. If outgoing and incoming
> traffic could be balanced, it would be great!
>
> Kind Regards,
>
> Nick.
>

[gentoo-user] Re: IP Load Sharing - Per Packet Load Balancing (Linux router)

[Nick Khamis]

Any different if the links are VDSL? I have little experience in
working with DSL based connections, and was wondering what was
possible in terms or bridging/bonding etc.. if anything.

N.

On 5/25/13, Nick Khamis <symack@gmail.com> wrote:
> I missed out some crusial info in my last email. As mentioned this
> would be two separate DSL services, connected using separate bridges.
> I think I am describing more of a link aggregation or bonding....
>
> Also assuming that the service providers support bonding of the links....
>
>
> N.
>
> On 5/25/13, Nick Khamis <symack@gmail.com> wrote:
>> Hello everyone,
>>
>> I am looking to put together a linux router for small business, and
>> was wondering if there was anything the suite (using quagga etc..)
>> that would allow for load balancing of regular dsl links. Kind of like
>> cisco with fast ethernet 0,1 and ip sef. If outgoing and incoming
>> traffic could be balanced, it would be great!
>>
>> Kind Regards,
>>
>> Nick.
>>
>

Re: [gentoo-user] IP Load Sharing - Per Packet Load Balancing (Linux router)

[Stroller]

On 25 May 2013, at 22:26, Nick Khamis wrote:
> ... As mentioned this
> would be two separate DSL services, connected using separate bridges.
> I think I am describing more of a link aggregation or bonding....
> 
> Also assuming that the service providers support bonding of the links….

Here in the UK this is a somewhat common thing - there are a number of ISPs which offer bonded xDSL services. 

It's certainly possible to use a Linux router to manage such a connection, although I don't know the details.

http://www22.brinkster.com/findall/bondedcd.html
http://www.automatedhome.co.uk/reviews/adsl-bonding-how-to-and-review.html

Stroller.

Re: [gentoo-user] IP Load Sharing - Per Packet Load Balancing (Linux router)

[J. Roeleveld]

> On 25 May 2013, at 22:26, Nick Khamis wrote:
>> ... As mentioned this
>> would be two separate DSL services, connected using separate bridges.
>> I think I am describing more of a link aggregation or bonding....
>>
>> Also assuming that the service providers support bonding of the links….
>
> Here in the UK this is a somewhat common thing - there are a number of
ISPs which
> offer bonded xDSL services.
>
> It's certainly possible to use a Linux router to manage such a connection,
> although I don't know the details.
>
> http://www22.brinkster.com/findall/bondedcd.html
>
> http://www.automatedhome.co.uk/reviews/adsl-bonding-how-to-and-revie
> w.html

Bonding network devices together is quite simple, but it needs to be
configured on both ends.
In other words, to merge 2 DSL-connections together using bonding, you
need to get both from the same ISP and the ISP would need to support it on
their end.

If bonding can't be done on the ISP-side, you can use seperate
load-balancing/failover using other techniques.

--
Joost

Re: [gentoo-user] IP Load Sharing - Per Packet Load Balancing (Linux router)

[Mick]

[-- Attachment #1: Type: Text/Plain, Size: 1803 bytes --]

On Sunday 26 May 2013 22:35:14 J. Roeleveld wrote:
> > On 25 May 2013, at 22:26, Nick Khamis wrote:
> >> ... As mentioned this
> >> would be two separate DSL services, connected using separate bridges.
> >> I think I am describing more of a link aggregation or bonding....
> >> 
> >> Also assuming that the service providers support bonding of the links….
> > 
> > Here in the UK this is a somewhat common thing - there are a number of
> 
> ISPs which
> 
> > offer bonded xDSL services.
> > 
> > It's certainly possible to use a Linux router to manage such a
> > connection, although I don't know the details.
> > 
> > http://www22.brinkster.com/findall/bondedcd.html
> > 
> > http://www.automatedhome.co.uk/reviews/adsl-bonding-how-to-and-revie
> > w.html
> 
> Bonding network devices together is quite simple, but it needs to be
> configured on both ends.
> In other words, to merge 2 DSL-connections together using bonding, you
> need to get both from the same ISP and the ISP would need to support it on
> their end.
> 
> If bonding can't be done on the ISP-side, you can use seperate
> load-balancing/failover using other techniques.

There's different ways of going about it, without or without MLPPP, depending 
on what your ISP offers:

http://wiki.aa.org.uk/index.php/Linux_upload_bonding_using_multipath_routing

http://wiki.aa.org.uk/index.php/Linux_upload_bonding_using_policy_routing


It used to be the case that Cisco 1800/2800 routers were used at customers' 
premises for MLPPP with certain UK ISPs, but since BT started implementing 
21CN (ADSL2+) they are using ERX core routers (Juniper) and no longer support 
MLPPP.  I understand that MPLS is used instead these days, but have no 
experience in its implementation.
-- 
Regards,
Mick

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 490 bytes --]

Re: [gentoo-user] IP Load Sharing - Per Packet Load Balancing (Linux router)

[Nick Khamis]

Remaining independent from corporate bureaucracy or lack of support
(ISP saying no to MLPP), and proprietary technology (our friends in
blue, purple and green ;). What would be the best way to integrate it
to my linux router to laod balance packets both up and down.
And if not at the packet level, maybe the session would suffice (i.e,
per network session)?
Although per packet would be preferred.

Kind Regards,

Nick.

On 5/26/13, Mick <michaelkintzios@gmail.com> wrote:
> On Sunday 26 May 2013 22:35:14 J. Roeleveld wrote:
>> > On 25 May 2013, at 22:26, Nick Khamis wrote:
>> >> ... As mentioned this
>> >> would be two separate DSL services, connected using separate bridges.
>> >> I think I am describing more of a link aggregation or bonding....
>> >>
>> >> Also assuming that the service providers support bonding of the
>> >> links….
>> >
>> > Here in the UK this is a somewhat common thing - there are a number of
>>
>> ISPs which
>>
>> > offer bonded xDSL services.
>> >
>> > It's certainly possible to use a Linux router to manage such a
>> > connection, although I don't know the details.
>> >
>> > http://www22.brinkster.com/findall/bondedcd.html
>> >
>> > http://www.automatedhome.co.uk/reviews/adsl-bonding-how-to-and-revie
>> > w.html
>>
>> Bonding network devices together is quite simple, but it needs to be
>> configured on both ends.
>> In other words, to merge 2 DSL-connections together using bonding, you
>> need to get both from the same ISP and the ISP would need to support it
>> on
>> their end.
>>
>> If bonding can't be done on the ISP-side, you can use seperate
>> load-balancing/failover using other techniques.
>
> There's different ways of going about it, without or without MLPPP,
> depending
> on what your ISP offers:
>
> http://wiki.aa.org.uk/index.php/Linux_upload_bonding_using_multipath_routing
>
> http://wiki.aa.org.uk/index.php/Linux_upload_bonding_using_policy_routing
>
>
> It used to be the case that Cisco 1800/2800 routers were used at customers'
>
> premises for MLPPP with certain UK ISPs, but since BT started implementing
> 21CN (ADSL2+) they are using ERX core routers (Juniper) and no longer
> support
> MLPPP.  I understand that MPLS is used instead these days, but have no
> experience in its implementation.
> --
> Regards,
> Mick
>

Re: [gentoo-user] IP Load Sharing - Per Packet Load Balancing (Linux router)

[Nick Khamis]

Sorry for the top post.

N.

Re: [gentoo-user] IP Load Sharing - Per Packet Load Balancing (Linux router)

[Nick Khamis]

By downstream, I mean within our own network. Obviously downstream LB
from the ISP's DSLAM would be impossible without MLPP, BGP support...

N

Re: [gentoo-user] IP Load Sharing - Per Packet Load Balancing (Linux router)

[Mick]

[-- Attachment #1: Type: Text/Plain, Size: 893 bytes --]

On Monday 27 May 2013 01:16:31 Nick Khamis wrote:
> By downstream, I mean within our own network. Obviously downstream LB
> from the ISP's DSLAM would be impossible without MLPP, BGP support...
> 
> N

I think that the CONFIG_BONDING option in the kernel will help you do what you 
want.  Have a look at /usr/src/linux/Documentation/networking/bonding.txt

http://www.cyberciti.biz/tips/linux-bond-or-team-multiple-network-interfaces-nic-into-single-interface.html


Without bonding, using Trivial/True link equaliser (teql) you could try 
something like this:

http://compton.nu/2009/12/per-packet-load-balancing-with-linux/

http://lartc.org/howto/lartc.loadshare.html


From a cursory look the kernel bonding seems to be the latest method of doing 
this.  Please let us know what you come up with as the best method for 
achieving what you want.
-- 
Regards,
Mick

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 490 bytes --]

Re: [gentoo-user] IP Load Sharing - Per Packet Load Balancing (Linux router)

[thegeezer]

Hi,
re: load balancing it must be done by the ISP for bonding DSL lines
properly.
what they support is what you will have to implement, typically they
will give you a managed router that you connect to and this will take
care of the bonding for you.

that said, you can do something similar with IPtables and packet marking
and routing tables (see lartc)
in the following iptables I have 2x DSL routers on eth1 and 2x DSL
routers on eth3, which is why I use masquerade -- the kernel knows how
to SNAT based on routing info
then I say "for every NEW connection choose a DSL line"
and then of course if a packet mark should be set then restore it, so
that subsequent connections go out the same direction.

this does mean of course, that you have 4x outgoing IP addresses for the
4x Internet connections
I appreciate this is not same thing as a bonded line, which would give
you 1x outgoing IP address, but it is useful to have this kind of thing
where bonded lines are not supported.

just be careful of some sites, such as Internet banks, authenticate you
against your IP, and if the subsequent connection comes from a differing
IP they immediately log you out.

This setup also means that you can add into the networking up/down and
do things like
# ip rule del from all fwmark 0xa lookup connA
when interfaces go down

the line that reads
-A OUTPUT ! -o eth0 -j redirection
means that if you have squid running it will also use all 4 connections
(not possible in squid.conf)

hope this helps!


IPRULE:
32758:    from 192.168.4.0/24 lookup connD
32759:    from 192.168.3.0/24 lookup connC
32760:    from 192.168.2.0/24 lookup connB
32761:    from 192.168.1.0/24 lookup connA
32762:    from all fwmark 0xd lookup connD
32763:    from all fwmark 0xc lookup connC
32764:    from all fwmark 0xb lookup connB
32765:    from all fwmark 0xa lookup connA
32766:    from all lookup main
32767:    from all lookup default


IPTABLES:
*nat
:PREROUTING ACCEPT
:INPUT ACCEPT
:OUTPUT ACCEPT
:POSTROUTING ACCEPT
-A POSTROUTING -o eth1 -j MASQUERADE
-A POSTROUTING -o eth3 -j MASQUERADE
COMMIT
*mangle
:PREROUTING ACCEPT
:INPUT ACCEPT
:FORWARD ACCEPT
:OUTPUT ACCEPT
:POSTROUTING ACCEPT
:RESTORE
:WAN1
:WAN2
:WAN3
:WAN4
:redirection
-A PREROUTING -j redirection
-A OUTPUT ! -o eth0 -j redirection
-A RESTORE -j CONNMARK --restore-mark --nfmask 0xffffffff --ctmask
0xffffffff
-A RESTORE -j ACCEPT
-A WAN1 -j MARK --set-xmark 0xa/0xffffffff
-A WAN1 -j CONNMARK --save-mark --nfmask 0xffffffff --ctmask 0xffffffff
-A WAN2 -j MARK --set-xmark 0xb/0xffffffff
-A WAN2 -j CONNMARK --save-mark --nfmask 0xffffffff --ctmask 0xffffffff
-A WAN3 -j MARK --set-xmark 0xc/0xffffffff
-A WAN3 -j CONNMARK --save-mark --nfmask 0xffffffff --ctmask 0xffffffff
-A WAN4 -j MARK --set-xmark 0xd/0xffffffff
-A WAN4 -j CONNMARK --save-mark --nfmask 0xffffffff --ctmask 0xffffffff
-A redirection -p tcp -m state --state RELATED,ESTABLISHED -j RESTORE
-A redirection -p tcp -m state --state NEW -m statistic --mode nth
--every 4 --packet 0 -j WAN1
-A redirection -p tcp -m state --state NEW -m statistic --mode nth
--every 4 --packet 1 -j WAN2
-A redirection -p tcp -m state --state NEW -m statistic --mode nth
--every 4 --packet 2 -j WAN3
-A redirection -p tcp -m state --state NEW -m statistic --mode nth
--every 4 --packet 3 -j WAN4
COMMIT
*filter
:INPUT ACCEPT
:FORWARD ACCEPT
:OUTPUT ACCEPT
:fail2ban-SSH
-A INPUT -p tcp -m tcp --dport 22 -j fail2ban-SSH
-A fail2ban-SSH -j RETURN
COMMIT

Re: [gentoo-user] IP Load Sharing - Per Packet Load Balancing (Linux router)

[Nick Khamis]

[-- Attachment #1: Type: text/plain, Size: 265 bytes --]

And who says you can't teach an old man new tricks huh geezer ;)?
Thank you so much for your response!!! That sorts out outgoing
traffic, have you had to setup rules for incoming traffic? I mean
from the outside world to a server for example?

Kind Regards,

Nick.

[-- Attachment #2: Type: text/html, Size: 411 bytes --]

Re: [gentoo-user] IP Load Sharing - Per Packet Load Balancing (Linux router)

[thegeezer]

[-- Attachment #1: Type: text/plain, Size: 2323 bytes --]

On 05/27/2013 02:53 PM, Nick Khamis wrote:
> And who says you can't teach an old man new tricks huh geezer ;)?
> Thank you so much for your response!!! That sorts out outgoing
> traffic, have you had to setup rules for incoming traffic? I mean
> from the outside world to a server for example?
>
> Kind Regards,
>
> Nick.
in this instance for me there is no need for incoming NAT
however, all you need in the mangle table is for each incoming connection:
# iptables -t mangle -I redirection 2 -i eth1 -j WAN1
to ensure that packet marking happens for incoming packets too, but
after the RELATED connections are marked.
in my example of 2 connections on each interface it gets messy and so I
would suggest doing this for only one Internet connect per interface --
otherwise the return packets will be forced out one direction resulting
in TCP handshakes failing.

once you are flagging incoming packets, then normal iptables NAT rules
can be used,
if you have e.g.
iptables -t nat -I PREROUTING -p tcp -m tcp --dport 80 -j DNAT
--to-destination 10.0.0.69
then this would apply for *all* external IP addresses.

in this instance you could have DNS like this
webserver1.domain.tld  A  externalIP-WAN1
                                        A externalIP-WAN2
that would then allow you to to have some resiliency if you were hosting
a web server behind two adsl connections and wanted to ensure a level of
load balancing / robustness


altenatively you can have
iptables -t nat -I PREROUTING -i eth1 -p tcp -m tcp --dport 80 -j DNAT
--to-destination 10.0.0.69
iptables -t nat -I PREROUTING -i eth3 -p tcp -m tcp --dport 80 -j DNAT
--to-destination 10.0.0.70

which would allow you to have a different web server on each adsl
connection.

hope this helps,


PS one thing i've just remembered is to be wary of
/proc/sys/net/ipv4/conf/<interfaces>/rp_filter
as the way it detects reverse paths seemingly is to ignore everything above

/etc/sysctl.conf

net.ipv4.conf.default.rp_filter 

rp_filter - BOOLEAN
	1 - do source validation by reversed path, as specified in RFC1812
	    Recommended option for single homed hosts and stub network
	    routers. Could cause troubles for complicated (not loop free)
	    networks running a slow unreliable protocol (sort of RIP),
	    or using static routes.

	0 - No source validation.




[-- Attachment #2: Type: text/html, Size: 3871 bytes --]