[gentoo-user] OT: default route dependent on dest port?

[gentoo-user] OT: default route dependent on dest port?

[Grant Edwards]

Let's posit two network interfaces net1 (192.168.x.y/16) and net2
(172.16.a.b/16).  There's a NAT/gateway available on each of the
networks. I want to use the 172.16 gateway for TCP connections to port
80 and the 192.168 gateway for everything else.

I'm primarily following this example:

  http://www.tldp.org/HOWTO/Adv-Routing-HOWTO/lartc.netfilter.html

My "main" routing table contains all directly accessible subnets plus
a default route via the 192.168 gateway.
  
I created a second route table named "pmain" which is identical to
"main" except it has a different default route via the 172.16 gateway.

My ip rules are:

  0:      from all lookup local 
  10000:  from all fwmark 0x1 lookup pmain 
  32766:  from all lookup main 
  32767:  from all lookup default 

I then add an iptables rule like this:

  iptables -A OUTPUT -t mangle -p tcp --dport 80 -j MARK --set-mark 1

Now all TCP packets destined for port 80 are sent to the 172.16
gateway, _but_ they're being sent with a 192.168 source address. The
TCP stack is apparently unaware of the advanced routing tricks and
thinks that the packets are going out via the 192.168 gateway.

IOW I've succesfully re-routed TCP _packets_ but not the TCP
_connection_.

How do I tell the TCP stack that it's supposed to use the 172.16
inteface/gateway for connections to port 80?

-- 
Grant Edwards               grant.b.edwards        Yow! I feel partially
                                  at               hydrogenated!
                              gmail.com            

Re: [gentoo-user] OT: default route dependent on dest port?

[Kerin Millar]

On 04/10/2013 21:55, Grant Edwards wrote:
> Let's posit two network interfaces net1 (192.168.x.y/16) and net2
> (172.16.a.b/16).  There's a NAT/gateway available on each of the
> networks. I want to use the 172.16 gateway for TCP connections to port
> 80 and the 192.168 gateway for everything else.
>
> I'm primarily following this example:
>
>    http://www.tldp.org/HOWTO/Adv-Routing-HOWTO/lartc.netfilter.html
>
> My "main" routing table contains all directly accessible subnets plus
> a default route via the 192.168 gateway.
>
> I created a second route table named "pmain" which is identical to
> "main" except it has a different default route via the 172.16 gateway.
>
> My ip rules are:
>
>    0:      from all lookup local
>    10000:  from all fwmark 0x1 lookup pmain
>    32766:  from all lookup main
>    32767:  from all lookup default
>
> I then add an iptables rule like this:
>
>    iptables -A OUTPUT -t mangle -p tcp --dport 80 -j MARK --set-mark 1

It would help if you were to also supply the details of:

   * ip -f inet -o a s
   * ip route show table main
   * ip route show table pmain

>
> Now all TCP packets destined for port 80 are sent to the 172.16
> gateway, _but_ they're being sent with a 192.168 source address. The
> TCP stack is apparently unaware of the advanced routing tricks and
> thinks that the packets are going out via the 192.168 gateway.
>
> IOW I've succesfully re-routed TCP _packets_ but not the TCP
> _connection_.
>
> How do I tell the TCP stack that it's supposed to use the 172.16
> inteface/gateway for connections to port 80?

--Kerin

Re: [gentoo-user] OT: default route dependent on dest port?

[Dragostin Yanev]

On Fri, 4 Oct 2013 20:55:25 +0000 (UTC)
Grant Edwards <grant.b.edwards@gmail.com> wrote:

> Let's posit two network interfaces net1 (192.168.x.y/16) and net2
> (172.16.a.b/16).  There's a NAT/gateway available on each of the
> networks. I want to use the 172.16 gateway for TCP connections to port
> 80 and the 192.168 gateway for everything else.
> 
> I'm primarily following this example:
> 
>   http://www.tldp.org/HOWTO/Adv-Routing-HOWTO/lartc.netfilter.html
> 
> My "main" routing table contains all directly accessible subnets plus
> a default route via the 192.168 gateway.
>   
> I created a second route table named "pmain" which is identical to
> "main" except it has a different default route via the 172.16 gateway.
> 
> My ip rules are:
> 
>   0:      from all lookup local 
>   10000:  from all fwmark 0x1 lookup pmain 
>   32766:  from all lookup main 
>   32767:  from all lookup default 
> 
> I then add an iptables rule like this:
> 
>   iptables -A OUTPUT -t mangle -p tcp --dport 80 -j MARK --set-mark 1
> 
> Now all TCP packets destined for port 80 are sent to the 172.16
> gateway, _but_ they're being sent with a 192.168 source address. The
> TCP stack is apparently unaware of the advanced routing tricks and
> thinks that the packets are going out via the 192.168 gateway.
> 
> IOW I've succesfully re-routed TCP _packets_ but not the TCP
> _connection_.
> 
> How do I tell the TCP stack that it's supposed to use the 172.16
> inteface/gateway for connections to port 80?
> 

Hi,
It's been a while but i believe you want to route via interface not
gateway. Providing more info will make it easier to help you.

[gentoo-user] Re: OT: default route dependent on dest port?

[Grant Edwards]

On 2013-10-04, Kerin Millar <kerframil@fastmail.co.uk> wrote:
> On 04/10/2013 21:55, Grant Edwards wrote:

>> I then add an iptables rule like this:
>>
>>    iptables -A OUTPUT -t mangle -p tcp --dport 80 -j MARK --set-mark 1

I'm about to try adding a second iptables rule to us the nat table to
rewrite the source IP address.  Something like this:

iptables -A POSTROUTING -t nat -o net2 -m mark --mark 1 -j SNAT --to 172.16.1.2

> It would help if you were to also supply the details of:
>
>    * ip -f inet -o a s

$ ip -f inet -o a s
1: lo    inet 127.0.0.1/8 scope host lo
2: net0    inet 192.168.8.4/16 brd 192.168.255.255 scope global net0
3: net1    inet 10.0.0.1/8 brd 10.255.255.255 scope global net1
3: net1    inet 192.168.250.1/24 brd 192.168.250.255 scope global net1
3: net1    inet 192.168.1.1/24 brd 192.168.1.255 scope global net1
3: net1    inet 169.254.1.1/16 brd 169.254.255.255 scope global net1
5: net2    inet 172.16.1.2/16 brd 172.16.255.255 scope global net2

>    * ip route show table main

$ ip route show table main
default via 192.168.0.254 dev net0  metric 2 
10.0.0.0/8 dev net1  proto kernel  scope link  src 10.0.0.1 
127.0.0.0/8 via 127.0.0.1 dev lo 
169.254.0.0/16 dev net1  proto kernel  scope link  src 169.254.1.1 
172.16.0.0/16 dev net2  proto kernel  scope link  src 172.16.1.2 metric 5 
192.168.0.0/16 dev net0  proto kernel  scope link  src 192.168.8.4 
192.168.1.0/24 dev net1  proto kernel  scope link  src 192.168.1.1 
192.168.250.0/24 dev net1  proto kernel  scope link  src 192.168.250.1 

>    * ip route show table pmain

$ ip route show table pmain
default via 172.16.0.34 dev net2  metric 2 
10.0.0.0/8 dev net1  proto kernel  scope link  src 10.0.0.1 
127.0.0.0/8 via 127.0.0.1 dev lo 
169.254.0.0/16 dev net1  proto kernel  scope link  src 169.254.1.1 
172.16.0.0/16 dev net2  proto kernel  scope link  src 172.16.1.2 metric 5 
192.168.0.0/16 dev net0  proto kernel  scope link  src 192.168.8.4 
192.168.1.0/24 dev net1  proto kernel  scope link  src 192.168.1.1 
192.168.250.0/24 dev net1  proto kernel  scope link  src 192.168.250.1 




>
>>
>> Now all TCP packets destined for port 80 are sent to the 172.16
>> gateway, _but_ they're being sent with a 192.168 source address. The
>> TCP stack is apparently unaware of the advanced routing tricks and
>> thinks that the packets are going out via the 192.168 gateway.
>>
>> IOW I've succesfully re-routed TCP _packets_ but not the TCP
>> _connection_.
>>
>> How do I tell the TCP stack that it's supposed to use the 172.16
>> inteface/gateway for connections to port 80?
>
> --Kerin
>
>


-- 
Grant Edwards               grant.b.edwards        Yow! !  I'm in a very
                                  at               clever and adorable INSANE
                              gmail.com            ASYLUM!!

[gentoo-user] Re: OT: default route dependent on dest port?

[Grant Edwards]

On 2013-10-04, Dragostin Yanev <gentoo+user@netixen.com> wrote:
> On Fri, 4 Oct 2013 20:55:25 +0000 (UTC)
>
>> IOW I've succesfully re-routed TCP _packets_ but not the TCP
>> _connection_.
>> 
>> How do I tell the TCP stack that it's supposed to use the 172.16
>> inteface/gateway for connections to port 80?
>
> It's been a while but i believe you want to route via interface not
> gateway. Providing more info will make it easier to help you.

Can you explain what "route via interface" means?

I tried a default route like this:

  ip route add table pmain default dev net2

instead of:

  ip route add table pmain default via <gateway-ip> dev net2
  
But then non-local packets routed via that table don't seem to go out
any interface at all.  

-- 
Grant Edwards               grant.b.edwards        Yow! I'm meditating on
                                  at               the FORMALDEHYDE and the
                              gmail.com            ASBESTOS leaking into my
                                                   PERSONAL SPACE!!

[gentoo-user] Re: OT: default route dependent on dest port?

[Grant Edwards]

On 2013-10-04, Grant Edwards <grant.b.edwards@gmail.com> wrote:
> On 2013-10-04, Kerin Millar <kerframil@fastmail.co.uk> wrote:
>> On 04/10/2013 21:55, Grant Edwards wrote:
>
>>> I then add an iptables rule like this:
>>>
>>>    iptables -A OUTPUT -t mangle -p tcp --dport 80 -j MARK --set-mark 1
>
> I'm about to try adding a second iptables rule to us the nat table to
> rewrite the source IP address.  Something like this:
>
> iptables -A POSTROUTING -t nat -o net2 -m mark --mark 1 -j SNAT --to 172.16.1.2

I also tried 

  iptables -A POSTROUTING -t nat -o net2 -p tcp --dport 80 -j SNAT --to 172.16.1.2

[I don't think the second rule is quite right, though, since it will
also match packets that _don't_ need to have the source IP
re-written.]
  
Both produced the same results: outbound packets look correct (they
have a source address that's valid for the net2 interface).  But,
inbound packets don't seem to reach the TCP stack:

  SYN goes out
  SYN/ACK comes back
  
  SYN gets resent
  SYN/ACK comes back

  SYN gets resent
  SYN/ACK comes back

The src/dst addresses in both the outbound SYN and the inbound SYN/ACK
look right.  Do I need another iptables rule to rewrite the
destination IP on the inbound packets?

-- 
Grant Edwards               grant.b.edwards        Yow! Does someone from
                                  at               PEORIA have a SHORTER
                              gmail.com            ATTENTION span than me?

[gentoo-user] Re: OT: default route dependent on dest port?

[Grant Edwards]

On 2013-10-04, Grant Edwards <grant.b.edwards@gmail.com> wrote:
> On 2013-10-04, Grant Edwards <grant.b.edwards@gmail.com> wrote:
>> On 2013-10-04, Kerin Millar <kerframil@fastmail.co.uk> wrote:
>>> On 04/10/2013 21:55, Grant Edwards wrote:
>>
>>>> I then add an iptables rule like this:
>>>>
>>>>    iptables -A OUTPUT -t mangle -p tcp --dport 80 -j MARK --set-mark 1
>>
>> I'm about to try adding a second iptables rule to us the nat table to
>> rewrite the source IP address.  Something like this:
>>
>> iptables -A POSTROUTING -t nat -o net2 -m mark --mark 1 -j SNAT --to 172.16.1.2
>
> I also tried 
>
>   iptables -A POSTROUTING -t nat -o net2 -p tcp --dport 80 -j SNAT --to 172.16.1.2
>
> [I don't think the second rule is quite right, though, since it will
> also match packets that _don't_ need to have the source IP
> re-written.]
>   
> Both produced the same results: outbound packets look correct (they
> have a source address that's valid for the net2 interface).  But,
> inbound packets don't seem to reach the TCP stack:

If I disable reverse-path filtering then it works. [I'm using the
first SNAT rule that matches based on the mark], but I don't really
like disabling all the reverse path filtering.

Is there a cleaner way to accomplish this that doesn't fall afoul of
rp_filter?

-- 
Grant Edwards               grant.b.edwards        Yow! I have accepted
                                  at               Provolone into my life!
                              gmail.com