[gentoo-user] [OT] Routing Problems

[gentoo-user] [OT] Routing Problems

[Alec Ten Harmsel]

Hey guys,

This is not Gentoo-specific, but one of my roommates just replaced our
router with a DD-WRT routers. For the most part, everything is great and
I love it. There's one problem, that may or may not be cause by said new
router. Between my desktop and my server, I can not ping/SSH/whatever.
The ARP request never gets resolved. Every other connection between any
other pair of machines works, just not desktop to server and vice versa.

The only stuff I could find on Google (which is mainly a front for
searching stackoverflow) is that it's a MAC collision (it's not) or that
it's a hardware problem (which I guess it may be, although I've tried
numerous permutations).

If anyone has any insight, I would appreciate it. I've been banging my
head against this for nearly one week now.

Regards,

Alec

Re: [gentoo-user] [OT] Routing Problems

[Mick]

[-- Attachment #1: Type: Text/Plain, Size: 1202 bytes --]

On Saturday 18 Oct 2014 04:45:21 Alec Ten Harmsel wrote:
> Hey guys,
> 
> This is not Gentoo-specific, but one of my roommates just replaced our
> router with a DD-WRT routers. For the most part, everything is great and
> I love it. There's one problem, that may or may not be cause by said new
> router. Between my desktop and my server, I can not ping/SSH/whatever.
> The ARP request never gets resolved. Every other connection between any
> other pair of machines works, just not desktop to server and vice versa.
> 
> The only stuff I could find on Google (which is mainly a front for
> searching stackoverflow) is that it's a MAC collision (it's not) or that
> it's a hardware problem (which I guess it may be, although I've tried
> numerous permutations).
> 
> If anyone has any insight, I would appreciate it. I've been banging my
> head against this for nearly one week now.

Can you give some additional information on the network topology?

Are we talking about LAN/WAN?  Same subnet?  VLAN?  DMZ?

Any firewall rules or special routing/bridging/etc.?

What do the router logs say?

Have you captured any packets on both ends and in between?

-- 
Regards,
Mick

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 473 bytes --]

Re: [gentoo-user] [OT] Routing Problems

[Alec Ten Harmsel]

On 10/18/2014 04:06 AM, Mick wrote:
> Can you give some additional information on the network topology?
>
> Are we talking about LAN/WAN?  Same subnet?  VLAN?  DMZ?

It's a LAN. Just a simple TP-Link router setup to do DHCP. No VLAN, no
DMZ, only one subnet 192.168.0.0/24. It's at my apartment, so nothing fancy.

>
> Any firewall rules or special routing/bridging/etc.?

Firewall is disabled on the server (CentOS) and my desktop. No special
routing or bridging.

>
> What do the router logs say?

DD-WRT is not very informative. It only has system-type stuff in
/var/log/messages, nothing LAN-related.

>
> Have you captured any packets on both ends and in between?
>
>
>

Capturing packets on my desktop shows strange behavior. When I ping my
server (kwopper), my desktop (greenbeast) starts generating a bunch of
ARPs, none of which get answered. When my laptop pings kwopper, the
first ARP is answered instantly and the pings succeed. Pinging from
kwopper is the same; instantly finds and connects to my laptop, but my
desktop does not see any ARPs or ICMPs from kwopper.

I can attach Wireshark dumps if helpful. tcpdump doesn't seem to want to
work on my server for some reason.

Alec

Re: [gentoo-user] [OT] Routing Problems

[Mick]

[-- Attachment #1: Type: Text/Plain, Size: 1523 bytes --]

On Saturday 18 Oct 2014 16:38:52 Alec Ten Harmsel wrote:
> On 10/18/2014 04:06 AM, Mick wrote:

> > What do the router logs say?
> 
> DD-WRT is not very informative. It only has system-type stuff in
> /var/log/messages, nothing LAN-related.

As James suggested, if you have SSH or telnet access to the router run arp to 
see what the arp tables include.  Also ping the server from the router to see 
if you get any responses.  I expect that these will not reveal anything 
untoward, but it is best to follow a process of elimination a step at a time.


> > Have you captured any packets on both ends and in between?
> 
> Capturing packets on my desktop shows strange behavior. When I ping my
> server (kwopper), my desktop (greenbeast) starts generating a bunch of
> ARPs, none of which get answered. 

Only to state the obvious, that this is not the expected behaviour.  Are you 
sure that the server firewall isn't configured to only allow connections from 
your laptop and/or drop arp packets to avoid arp attacks?  What happens when 
you disable the firewall?


> When my laptop pings kwopper, the
> first ARP is answered instantly and the pings succeed. Pinging from
> kwopper is the same; instantly finds and connects to my laptop, but my
> desktop does not see any ARPs or ICMPs from kwopper.

Using arpscan and arping between desktop and server you should be able to find 
out what is happening, but I suspect something to do with the server 
configuration.

-- 
Regards,
Mick

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 473 bytes --]

Re: [gentoo-user] [OT] Routing Problems

[Alec Ten Harmsel]

On 10/18/2014 12:37 PM, Mick wrote:
> On Saturday 18 Oct 2014 16:38:52 Alec Ten Harmsel wrote:
>> On 10/18/2014 04:06 AM, Mick wrote:
>>> What do the router logs say?
>> DD-WRT is not very informative. It only has system-type stuff in
>> /var/log/messages, nothing LAN-related.
> As James suggested, if you have SSH or telnet access to the router run arp to 
> see what the arp tables include.  Also ping the server from the router to see 
> if you get any responses.  I expect that these will not reveal anything 
> untoward, but it is best to follow a process of elimination a step at a time.

All fine here; pinging from router to server works, and the ARP table
has entries for both desktop and server.

>
>>> Have you captured any packets on both ends and in between?
>> Capturing packets on my desktop shows strange behavior. When I ping my
>> server (kwopper), my desktop (greenbeast) starts generating a bunch of
>> ARPs, none of which get answered. 
> Only to state the obvious, that this is not the expected behaviour.  Are you 
> sure that the server firewall isn't configured to only allow connections from 
> your laptop and/or drop arp packets to avoid arp attacks?  What happens when 
> you disable the firewall?

Firewall is completely disabled on the server, as is SELinux.

>
>> When my laptop pings kwopper, the
>> first ARP is answered instantly and the pings succeed. Pinging from
>> kwopper is the same; instantly finds and connects to my laptop, but my
>> desktop does not see any ARPs or ICMPs from kwopper.
> Using arpscan and arping between desktop and server you should be able to find 
> out what is happening, but I suspect something to do with the server 
> configuration.
>

arpscanning the entire subnet results in 3 responses, with 2 being
displayed and 1 being dropped by the kernel. arpping, even with -D and
-U, returns nothing.

I have no idea what's going on. I think what I'm gonna do is install my
old router behind the new router and plug in all my device to that one
and see if it works, because I absolutely need my desktop and server to
be able to reach each other.

Alec

Re: [gentoo-user] [OT] Routing Problems

[Alec Ten Harmsel]

On 10/18/2014 02:13 PM, Alec Ten Harmsel wrote:
>
> I have no idea what's going on. I think what I'm gonna do is install my
> old router behind the new router and plug in all my device to that one
> and see if it works, because I absolutely need my desktop and server to
> be able to reach each other.
>
> Alec

Well, this fixed it. I'm gonna stick with that.

Thanks for the help.

Alec

Re: [gentoo-user] [OT] Routing Problems

[thegeezer]

On 18/10/14 20:10, Alec Ten Harmsel wrote:
> On 10/18/2014 02:13 PM, Alec Ten Harmsel wrote:
>> I have no idea what's going on. I think what I'm gonna do is install my
>> old router behind the new router and plug in all my device to that one
>> and see if it works, because I absolutely need my desktop and server to
>> be able to reach each other.
>>
>> Alec
> Well, this fixed it. I'm gonna stick with that.
>
> Thanks for the help.
>
> Alec
>
what would have been interesting to note is from your workstation, your
server and the dd-wrt box
# ip -o route
# arp -n
# iptables-save

because putting your devices behind your router fixed it, the issue may
have been dd-wrt, but more likely was something your other room-mates
have going on.  

Re: [gentoo-user] [OT] Routing Problems

[Mick]

[-- Attachment #1: Type: Text/Plain, Size: 383 bytes --]

On Saturday 18 Oct 2014 19:13:23 Alec Ten Harmsel wrote:
> arpscanning the entire subnet results in 3 responses, with 2 being
> displayed and 1 being dropped by the kernel.

Oh!  I wonder if this is your problem.  I can't answer why 1 response is being 
dropped by the kernel.  Have you set up some fancy tcp wrappers or firewall 
rules at the desktop?

-- 
Regards,
Mick

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 473 bytes --]

Re: [gentoo-user] [OT] Routing Problems

[Alec Ten Harmsel]

On 10/19/2014 06:31 AM, Mick wrote:
> On Saturday 18 Oct 2014 19:13:23 Alec Ten Harmsel wrote:
>> arpscanning the entire subnet results in 3 responses, with 2 being
>> displayed and 1 being dropped by the kernel.
> Oh!  I wonder if this is your problem.  I can't answer why 1 response is being 
> dropped by the kernel.  Have you set up some fancy tcp wrappers or firewall 
> rules at the desktop?
>

Nope, running vanilla-sources, no tcp wrappers or firewall. In fact,
iptables wasn't installed until arp-scan or wireshark or another network
tool pulled it in as a dependency.

If you're really interested as to why this happens and can tell me how I
can even log something like this, I'd be more than willing to play
around with it.

Alec

Re: [gentoo-user] [OT] Routing Problems

[Mick]

[-- Attachment #1: Type: Text/Plain, Size: 1830 bytes --]

On Sunday 19 Oct 2014 14:14:03 Alec Ten Harmsel wrote:
> On 10/19/2014 06:31 AM, Mick wrote:
> > On Saturday 18 Oct 2014 19:13:23 Alec Ten Harmsel wrote:
> >> arpscanning the entire subnet results in 3 responses, with 2 being
> >> displayed and 1 being dropped by the kernel.
> > 
> > Oh!  I wonder if this is your problem.  I can't answer why 1 response is
> > being dropped by the kernel.  Have you set up some fancy tcp wrappers or
> > firewall rules at the desktop?
> 
> Nope, running vanilla-sources, no tcp wrappers or firewall. In fact,
> iptables wasn't installed until arp-scan or wireshark or another network
> tool pulled it in as a dependency.
> 
> If you're really interested as to why this happens and can tell me how I
> can even log something like this, I'd be more than willing to play
> around with it.

The information that thegeezer suggested will offer some quick comparisons 
between the three machines.

In addition, for capturing packets:

Router - I think that DD-WRT offers netflow to monitor interfaces.  That 
should show what comes in - goes out, but there may be other network 
diagnostic tools that DD-WRT offers.

On desktop, laptop (for comparison) and server you could use something like:

 tcpdump -i  eth0 -e -l -p -U -vvv -w tcpdump_cap.txt -XX

to record and replace the -w with -r to read the captured file, or use 
wireshark to study the captured file.  Run tcpdump while using arping from 
each machine at a time:

 arping -I eth0 -c 2 192.168.0.10 

You can filter for arp packets and see which machine is the culprit, because 
so far we do not know for sure whether the server refuses to return arp 
responses, or the desktop drops them for some reason (e.g. malformed ARP 
packets), or the router routes them differently.

-- 
Regards,
Mick

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 473 bytes --]

Re: [gentoo-user] [OT] Routing Problems

[wabenbau]

Am Samstag, 18.10.2014 um 11:38
schrieb Alec Ten Harmsel <alec@alectenharmsel.com>:

> 
> On 10/18/2014 04:06 AM, Mick wrote:
> > Can you give some additional information on the network topology?
> >
> > Are we talking about LAN/WAN?  Same subnet?  VLAN?  DMZ?
> 
> It's a LAN. Just a simple TP-Link router setup to do DHCP. No VLAN, no
> DMZ, only one subnet 192.168.0.0/24. It's at my apartment, so nothing
> fancy.

Check the LAN for duplicate IP's. 

Regards
wabe

[gentoo-user] Re: [OT] Routing Problems

[James]

Alec Ten Harmsel <alec <at> alectenharmsel.com> writes:


> This is not Gentoo-specific, but one of my roommates just replaced our
> router with a DD-WRT routers. For the most part, everything is great and
> I love it. There's one problem, that may or may not be cause by said new
> router. Between my desktop and my server, I can not ping/SSH/whatever.
> The ARP request never gets resolved. Every other connection between any
> other pair of machines works, just not desktop to server and vice versa.


First simple thing. Make sure your pc is in the arp tables for the 
router. Log into the router, however, and just ping your unresponsive
manchine:

You shlould be able to just run the 'arp' command:

 # arp


In fact run that command from several nix machines.


If it is not in your dd-wrt OS, then research how to add
it or route the router with  a similar (embedded) OS


hth,
James