[gentoo-user] Packet Shaping

[gentoo-user] Packet Shaping

[Grant]

Hello, I have packet shaping set up on my Gentoo router with iptables,
shorewall, and "The Wonder Shaper" which is a /etc/shorewall/tcstart
file.  It seems to be working since internet radio is now full of
hiccups.  :)

I'd like to configure it to prioritize voip, ssh, web browsing,
bittorrent downloads, and bittorrent uploads in that order.  I'm using
these docs:

http://www.shorewall.net/2.0/traffic_shaping.htm
http://lartc.org/wondershaper/

since this doesn't include shorewall config info:

http://gentoo-wiki.com/HOWTO_Packet_Shaping

Is anyone else using The Wonder Shaper?  Would anyone recommend I
ditch it and write a tcstart file from scratch?

- Grant
-- 
gentoo-user@gentoo.org mailing list

Re: [gentoo-user] Packet Shaping

[Hans-Werner Hilse]

Hi,

On Fri, 13 Apr 2007 11:24:03 -0700
Grant <emailgrant@gmail.com> wrote:

> Hello, I have packet shaping set up on my Gentoo router with iptables,
> shorewall, and "The Wonder Shaper" which is a /etc/shorewall/tcstart
> file.  It seems to be working since internet radio is now full of
> hiccups.  :)

I never quite understood ingress shaping. Dropping packets always
sounded wrong to me...

> Is anyone else using The Wonder Shaper?  Would anyone recommend I
> ditch it and write a tcstart file from scratch?

No, just edit it (more than those variables on the top). wshaper is
really not that big, well structured and at least somewhat documented.
It makes a good template.

-hwh
-- 
gentoo-user@gentoo.org mailing list

Re: [gentoo-user] Packet Shaping

[Grant]

> > Hello, I have packet shaping set up on my Gentoo router with iptables,
> > shorewall, and "The Wonder Shaper" which is a /etc/shorewall/tcstart
> > file.  It seems to be working since internet radio is now full of
> > hiccups.  :)
>
> I never quite understood ingress shaping. Dropping packets always
> sounded wrong to me...
>
> > Is anyone else using The Wonder Shaper?  Would anyone recommend I
> > ditch it and write a tcstart file from scratch?
>
> No, just edit it (more than those variables on the top). wshaper is
> really not that big, well structured and at least somewhat documented.
> It makes a good template.

After a lot of testing, these numbers seem to give me the best
performance as far as bittorrent download speed.  How can that be?  Is
DOWNLINK my upload and UPLINK my download?

DOWNLINK=425
UPLINK=3450
DEV=ath0

I tried to define the bittorrent ports as a low priority like this:

NOPRIOPORTSRC=6881:6999
NOPRIOPORTDST=6881:6999

but I get this when restarting shorewall:

Illegal "match"

Are you using port ranges in those variable?

- Grant
-- 
gentoo-user@gentoo.org mailing list

Re: [gentoo-user] Packet Shaping

[Hans-Werner Hilse]

Hi,

On Sat, 14 Apr 2007 08:37:19 -0700
Grant <emailgrant@gmail.com> wrote:

> After a lot of testing, these numbers seem to give me the best
> performance as far as bittorrent download speed.
> How can that be?  Is DOWNLINK my upload and UPLINK my download?

Hm, usually not. Are you by chance shaping the internal (i.e. LAN)
interface on a router? Then, of course, it would make sense (except
from the fact that shaping your actual bottle neck, i.e. Internet
connection, would make more sense).

> I tried to define the bittorrent ports as a low priority like this:
> NOPRIOPORTSRC=6881:6999
> NOPRIOPORTDST=6881:6999
> 
> but I get this when restarting shorewall:
> Illegal "match"

In the wshaper source, the action happens here (and the same for *DST):
---snip
for a in $NOPRIOPORTSRC
do
        tc filter add dev $DEV parent 1: protocol ip prio 15 u32 \
           match ip sport $a 0xffff flowid 1:30
done
---snip

In this configuration, it expects a shell-separatable list of ports,
i.e. separated by whitespace. It will create a rule for each one.

The dirty, easy way:
| NOPRIOPORTSRC=$(seq 6881 6999)
| NOPRIOPORTDST=$NOPRIOPORTSRC

But I would rather extend wshaper by another (custom) line and dump your
NOPRIOPORT*-settings.

The syntax is "match ip sport PATTERN MASK". The port of an incoming
packet is AND'ed w/ the MASK and compared to the PATTERN.

e.g. "match ip sport 6880 0xffe0" would match 6880-6911, a further
"match ip sport 6912 0xffc0" would match 6912-6975.

The advantage of this is simply speed/CPU cycles. Alternatively, you
could just use iptables to mark your packets (which probably means even
more precious CPU cycles). The wshaper script, however, doesn't use
iptables.

-hwh
-- 
gentoo-user@gentoo.org mailing list

Re: [gentoo-user] Packet Shaping

[Grant]

> > After a lot of testing, these numbers seem to give me the best
> > performance as far as bittorrent download speed.
> > How can that be?  Is DOWNLINK my upload and UPLINK my download?
>
> Hm, usually not. Are you by chance shaping the internal (i.e. LAN)
> interface on a router? Then, of course, it would make sense (except
> from the fact that shaping your actual bottle neck, i.e. Internet
> connection, would make more sense).

Thanks a lot for that.  I switched the interface to eth0 and reversed
the DOWNLINK and UPLINK values.

> > I tried to define the bittorrent ports as a low priority like this:
> > NOPRIOPORTSRC=6881:6999
> > NOPRIOPORTDST=6881:6999
> >
> > but I get this when restarting shorewall:
> > Illegal "match"
>
> In the wshaper source, the action happens here (and the same for *DST):
> ---snip
> for a in $NOPRIOPORTSRC
> do
>         tc filter add dev $DEV parent 1: protocol ip prio 15 u32 \
>            match ip sport $a 0xffff flowid 1:30
> done
> ---snip
>
> In this configuration, it expects a shell-separatable list of ports,
> i.e. separated by whitespace. It will create a rule for each one.
>
> The dirty, easy way:
> | NOPRIOPORTSRC=$(seq 6881 6999)
> | NOPRIOPORTDST=$NOPRIOPORTSRC
>
> But I would rather extend wshaper by another (custom) line and dump your
> NOPRIOPORT*-settings.
>
> The syntax is "match ip sport PATTERN MASK". The port of an incoming
> packet is AND'ed w/ the MASK and compared to the PATTERN.
>
> e.g. "match ip sport 6880 0xffe0" would match 6880-6911, a further
> "match ip sport 6912 0xffc0" would match 6912-6975.
>
> The advantage of this is simply speed/CPU cycles. Alternatively, you
> could just use iptables to mark your packets (which probably means even
> more precious CPU cycles). The wshaper script, however, doesn't use
> iptables.

I switched to wshaper from wshaper.htb and now ssh and browsing seem a
lot more responsive.  Could that be because I'm missing something in
my kernel that I need for htb?  I don't get any errors when restarting
the firewall.

One other thing is if I don't limit the upload rate within my
bittorrent client, it really goes nuts and everything else suffers.  I
don't see how that's possible with UPLINK and the bittorrent source
and destination ports defined.

What I'd really like to do is limit the bittorrent upload rate so
Verizon doesn't throttle my connection.  Can I do that with The Wonder
Shaper without limiting the total upload rate?  I don't trust the
bittorrent clients I use to limit it.

- Grant
-- 
gentoo-user@gentoo.org mailing list

Re: [gentoo-user] Packet Shaping

[Hans-Werner Hilse]

Hi,

On Mon, 16 Apr 2007 09:00:04 -0700
Grant <emailgrant@gmail.com> wrote:

> > > After a lot of testing, these numbers seem to give me the best
> > > performance as far as bittorrent download speed.
> > > How can that be?  Is DOWNLINK my upload and UPLINK my download?
> >
> > Hm, usually not. Are you by chance shaping the internal (i.e. LAN)
> > interface on a router? Then, of course, it would make sense (except
> > from the fact that shaping your actual bottle neck, i.e. Internet
> > connection, would make more sense).
> 
> Thanks a lot for that.  I switched the interface to eth0 and reversed
> the DOWNLINK and UPLINK values.

:-)

> I switched to wshaper from wshaper.htb and now ssh and browsing seem a
> lot more responsive.  Could that be because I'm missing something in
> my kernel that I need for htb?  I don't get any errors when restarting
> the firewall.

I'm not sure about that. I did only try wshaper.htb and didn't manage
to fit it to my needs completely either (see below). The kernel
configuration help tells a good bunch of info, IIRC.

> One other thing is if I don't limit the upload rate within my
> bittorrent client, it really goes nuts and everything else suffers.  I
> don't see how that's possible with UPLINK and the bittorrent source
> and destination ports defined.

Well, the problem is that limiting inbound traffic is absolutely
unreliable. From the numbers given, I guess you're on DSL, right? (Just
like me, BTW.) If you were on cable, well, there's not a lot you can do
since the media is unreliable w/ regard to your share of it. But I
think you're talking about stable bandwith. If you're not lucky, all
those peers out there flood your inbound traffic line. You can't shape
this on your side, it's absolutely an issue to be resolved on the DSLAM
your DSL modem connects to. OTOH, those routers usually don't do very
sophisticated packet inspection... So it's all about cutting expensive
connections down very early. This is the even more true for
applications that are somewhat hasty in changing their requested and
incoming traffic. So first try cutting down the maximum even more. Take
a few measures and see what is actually saturated: upstream or
downstream. If it's in fact neither, it's a configuration issue.

> What I'd really like to do is limit the bittorrent upload rate so
> Verizon doesn't throttle my connection.  Can I do that with The Wonder
> Shaper without limiting the total upload rate?  I don't trust the
> bittorrent clients I use to limit it.

Did you consider trickle? It's lightweight and easy and works on
application layer (i.e. usermode) by overloading glibc functions... If
you're not trying to manage a whole set of clients behind a router,
that would be an option.

And to be honest: I've dumped QoS on my linux-based router. I've never
managed to fully saturize my link the way I wanted it using it. I'm not
completely sure if I should blame it on the 125MHz the poor CPU's
running at (it's a WiFi AP, the Linksys WAP54g) or the 8MB of RAM...

-hwh
-- 
gentoo-user@gentoo.org mailing list

Re: [gentoo-user] Packet Shaping

[Grant]

> Well, the problem is that limiting inbound traffic is absolutely
> unreliable. From the numbers given, I guess you're on DSL, right? (Just
> like me, BTW.) If you were on cable, well, there's not a lot you can do
> since the media is unreliable w/ regard to your share of it. But I
> think you're talking about stable bandwith. If you're not lucky, all
> those peers out there flood your inbound traffic line. You can't shape
> this on your side, it's absolutely an issue to be resolved on the DSLAM
> your DSL modem connects to. OTOH, those routers usually don't do very
> sophisticated packet inspection... So it's all about cutting expensive
> connections down very early. This is the even more true for
> applications that are somewhat hasty in changing their requested and
> incoming traffic. So first try cutting down the maximum even more. Take
> a few measures and see what is actually saturated: upstream or
> downstream. If it's in fact neither, it's a configuration issue.

It's actually my upload rate that's difficult to limit.  That's not
inbound traffic right?

- Grant
-- 
gentoo-user@gentoo.org mailing list

Re: [gentoo-user] Packet Shaping

[Hans-Werner Hilse]

Hi,

On Mon, 16 Apr 2007 15:19:14 -0700 Grant <emailgrant@gmail.com> wrote:

> It's actually my upload rate that's difficult to limit.  That's not
> inbound traffic right?

Right. You should be able to shape upload quite well. Did you try to
lower allowed upload bandwith further below the nominate bandwith?

-hwh
-- 
gentoo-user@gentoo.org mailing list

Re: [gentoo-user] Packet Shaping

[Grant]

> > It's actually my upload rate that's difficult to limit.  That's not
> > inbound traffic right?
>
> Right. You should be able to shape upload quite well. Did you try to
> lower allowed upload bandwith further below the nominate bandwith?

I switched from The Wonder Shaper and started using shorewall configs
and it's working great.  I can't get ipp2p to identify bittorrent
traffic though, so I have the default set up for really low priority.
Thanks for your help!  This is fun.

- Grant
-- 
gentoo-user@gentoo.org mailing list

Re: [gentoo-user] Packet Shaping

[Elias Probst]

[-- Attachment #1: Type: text/plain, Size: 641 bytes --]

On Wednesday 18 April 2007 02:29:56 Grant wrote:
> I switched from The Wonder Shaper and started using shorewall configs
> and it's working great.  I can't get ipp2p to identify bittorrent
> traffic though, so I have the default set up for really low priority.
> Thanks for your help!  This is fun.
>
> - Grant

Never got Shorewall traffic shaping to do anything.
shorewall show | grep mark | grep -v "mark=0" | wc -l
is 0. It seems to fail when classifying packets. Neither port based 
classifying worked nor ipp2p based classifying.

If you have a working configuration, I'd like to see how you did it.

Regards,
Elias P.

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-user] Packet Shaping

[Grant]

> > I switched from The Wonder Shaper and started using shorewall configs
> > and it's working great.  I can't get ipp2p to identify bittorrent
> > traffic though, so I have the default set up for really low priority.
> > Thanks for your help!  This is fun.
> >
> > - Grant
>
> Never got Shorewall traffic shaping to do anything.
> shorewall show | grep mark | grep -v "mark=0" | wc -l
> is 0. It seems to fail when classifying packets. Neither port based
> classifying worked nor ipp2p based classifying.
>
> If you have a working configuration, I'd like to see how you did it.

Glady.  This works like a dream.  360kbps bittorrent download with a
simultaneous voip call in perfect clarity, remote ssh feeling local,
and extremely responsive browsing.  I do wish I could get ipp2p
working so I don't have to default to mark 4 though.  I'm using
Verizon 3.0Mbps DSL.  No special shorewall.conf config at all.

tcdevices:
eth0    365kbps 150kbps

tcrules:
eth0    1       full    full    1
eth0    2       full/2  full    2
eth0    3       full/3  full*9/10       3
eth0    4       full/25 full/15 4       default

tcrules:
1       0.0.0.0/0       so.me.i.p/32 udp
2       0.0.0.0/0       0.0.0.0/0       tcp     22
2       0.0.0.0/0       0.0.0.0/0       icmp    echo-request
2       0.0.0.0/0       0.0.0.0/0       icmp    echo-reply
2       0.0.0.0/0       so.me.i.p/32 all
3       0.0.0.0/0       0.0.0.0/0       udp     53
3       0.0.0.0/0       0.0.0.0/0       tcp     80,443
3       0.0.0.0/0       0.0.0.0/0       tcp     873
3       0.0.0.0/0       0.0.0.0/0       udp     873

- Grant
-- 
gentoo-user@gentoo.org mailing list