public inbox for gentoo-user@lists.gentoo.org
 help / color / mirror / Atom feed
* [gentoo-user] Gentoo router: Conntrack table full
@ 2008-03-23  3:16 Dan Cowsill
  2008-03-23  3:22 ` Andrey Falko
  2008-03-23 13:42 ` Mike Williams
  0 siblings, 2 replies; 6+ messages in thread
From: Dan Cowsill @ 2008-03-23  3:16 UTC (permalink / raw
  To: gentoo-user

Hi folks,

Today I had some really serious problems with my Gentoo router.  I
could ping it, and all the network connections were in place and
functional, but no outside access.  I looked into it and found that
the syslog was flooded with this:


Mar 22 21:25:55 localhost kernel: nf_conntrack: table full, dropping packet.
Mar 22 21:26:00 localhost kernel: printk: 11 messages suppressed.
Mar 22 21:26:00 localhost kernel: nf_conntrack: table full, dropping packet.
Mar 22 21:26:05 localhost kernel: printk: 16 messages suppressed.


These messages spanned a full 20 hours of the log.  I understand that
conntrack is the connection tracking system that iptables uses.  I
also understand that its maximum is something on the order of 65000
simultaneous connections.  For a simple home network, I think we can
agree that I would probably never approach this number of connections
with normal use.

So my question is this:  what could have caused the router's
connection tracker to overflow?
-- 
Dan Cowsill
http://www.danthehat.net
-- 
gentoo-user@lists.gentoo.org mailing list



^ permalink raw reply	[flat|nested] 6+ messages in thread

* Re: [gentoo-user] Gentoo router: Conntrack table full
  2008-03-23  3:16 [gentoo-user] Gentoo router: Conntrack table full Dan Cowsill
@ 2008-03-23  3:22 ` Andrey Falko
  2008-03-23  3:26   ` Dan Cowsill
  2008-03-23 13:42 ` Mike Williams
  1 sibling, 1 reply; 6+ messages in thread
From: Andrey Falko @ 2008-03-23  3:22 UTC (permalink / raw
  To: gentoo-user

On Sat, Mar 22, 2008 at 11:16 PM, Dan Cowsill <danthehat@gmail.com> wrote:
> Hi folks,
>
>  Today I had some really serious problems with my Gentoo router.  I
>  could ping it, and all the network connections were in place and
>  functional, but no outside access.  I looked into it and found that
>  the syslog was flooded with this:
>
>
>  Mar 22 21:25:55 localhost kernel: nf_conntrack: table full, dropping packet.
>  Mar 22 21:26:00 localhost kernel: printk: 11 messages suppressed.
>  Mar 22 21:26:00 localhost kernel: nf_conntrack: table full, dropping packet.
>  Mar 22 21:26:05 localhost kernel: printk: 16 messages suppressed.
>
>
>  These messages spanned a full 20 hours of the log.  I understand that
>  conntrack is the connection tracking system that iptables uses.  I
>  also understand that its maximum is something on the order of 65000
>  simultaneous connections.  For a simple home network, I think we can
>  agree that I would probably never approach this number of connections
>  with normal use.
>
>  So my question is this:  what could have caused the router's
>  connection tracker to overflow?
>  --
>  Dan Cowsill
>  http://www.danthehat.net
>  --
>  gentoo-user@lists.gentoo.org mailing list
>
>

What type of 'net services do you run between your home network and
the outside? Is there a possibility that someone out have put a denial
of service attack on you?
-- 
gentoo-user@lists.gentoo.org mailing list



^ permalink raw reply	[flat|nested] 6+ messages in thread

* Re: [gentoo-user] Gentoo router: Conntrack table full
  2008-03-23  3:22 ` Andrey Falko
@ 2008-03-23  3:26   ` Dan Cowsill
  2008-03-23  9:23     ` Michal 'vorner' Vaner
  0 siblings, 1 reply; 6+ messages in thread
From: Dan Cowsill @ 2008-03-23  3:26 UTC (permalink / raw
  To: gentoo-user

On Sat, Mar 22, 2008 at 11:22 PM, Andrey Falko <ma3oxuct@gmail.com> wrote:
>
> On Sat, Mar 22, 2008 at 11:16 PM, Dan Cowsill <danthehat@gmail.com> wrote:
>  > Hi folks,
>  >
>  >  Today I had some really serious problems with my Gentoo router.  I
>  >  could ping it, and all the network connections were in place and
>  >  functional, but no outside access.  I looked into it and found that
>  >  the syslog was flooded with this:
>  >
>  >
>  >  Mar 22 21:25:55 localhost kernel: nf_conntrack: table full, dropping packet.
>  >  Mar 22 21:26:00 localhost kernel: printk: 11 messages suppressed.
>  >  Mar 22 21:26:00 localhost kernel: nf_conntrack: table full, dropping packet.
>  >  Mar 22 21:26:05 localhost kernel: printk: 16 messages suppressed.
>  >
>  >
>  >  These messages spanned a full 20 hours of the log.  I understand that
>  >  conntrack is the connection tracking system that iptables uses.  I
>  >  also understand that its maximum is something on the order of 65000
>  >  simultaneous connections.  For a simple home network, I think we can
>  >  agree that I would probably never approach this number of connections
>  >  with normal use.
>  >
>  >  So my question is this:  what could have caused the router's
>  >  connection tracker to overflow?
>  >  --
>  >  Dan Cowsill
>  >  http://www.danthehat.net
>  >  --
>  >  gentoo-user@lists.gentoo.org mailing list
>  >
>  >
>
>  What type of 'net services do you run between your home network and
>  the outside? Is there a possibility that someone out have put a denial
>  of service attack on you?
>  --
>  gentoo-user@lists.gentoo.org mailing list
>
>

I have SSH to a server, two open ports for bit torrent connections and
a few ranges for DCC transfers from irc.

The possibility of a DoS attack is pretty real, I imagine.  Is there
any way I could be sure?

-- 
Dan Cowsill
http://www.danthehat.net
-- 
gentoo-user@lists.gentoo.org mailing list



^ permalink raw reply	[flat|nested] 6+ messages in thread

* Re: [gentoo-user] Gentoo router: Conntrack table full
  2008-03-23  3:26   ` Dan Cowsill
@ 2008-03-23  9:23     ` Michal 'vorner' Vaner
  0 siblings, 0 replies; 6+ messages in thread
From: Michal 'vorner' Vaner @ 2008-03-23  9:23 UTC (permalink / raw
  To: gentoo-user

[-- Attachment #1: Type: text/plain, Size: 700 bytes --]

Hello

On Sat, Mar 22, 2008 at 11:26:16PM -0400, Dan Cowsill wrote:
> On Sat, Mar 22, 2008 at 11:22 PM, Andrey Falko <ma3oxuct@gmail.com> wrote:
> I have SSH to a server, two open ports for bit torrent connections and
> a few ranges for DCC transfers from irc.

Torrents can sometimes open thousands of connections (yes, it is very
aggressive way of transferring data). And if you have something like
Skype, they a full table is not something unusual.

So IMO it could be DoS, but it can be some application that doesn't know
what is polite behaviour.

-- 
This is a terroristic email. It will explode in 10 minutes, 
if you do not close it in the meantime.

Michal 'vorner' Vaner

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

^ permalink raw reply	[flat|nested] 6+ messages in thread

* Re: [gentoo-user] Gentoo router: Conntrack table full
  2008-03-23  3:16 [gentoo-user] Gentoo router: Conntrack table full Dan Cowsill
  2008-03-23  3:22 ` Andrey Falko
@ 2008-03-23 13:42 ` Mike Williams
  2008-03-25  6:13   ` Adam Carter
  1 sibling, 1 reply; 6+ messages in thread
From: Mike Williams @ 2008-03-23 13:42 UTC (permalink / raw
  To: gentoo-user

On Sunday 23 March 2008 03:16:16 Dan Cowsill wrote:
>  I
> also understand that its maximum is something on the order of 65000
> simultaneous connections.

That's a significant understatement.
The default limit is based on how much RAM you have, and is set very 
conservatively.
/proc/sys/net/ipv4/netfilter/ip_conntrack_max sets how many connections you 
can track.

You should also 
drop /proc/sys/net/ipv4/netfilter/ip_conntrack_tcp_timeout_established 
significantly. Connections can hang around for weeks, unless properly closed.

On the production linux firewalls I maintain they were happily handling 
~50-60k connections until I dropped ip_conntrack_tcp_timeout_established to 
432000 seconds when the conntrack table dropped to ~30k. I could drop it a 
lot lower, but the machines cope with absolutely no issues.

Personally, I'd drop ip_conntrack_tcp_timeout_established to about a day, or 
even less, as connections won't time out if traffic continues to pass.

-- 
Mike Williams
--
gentoo-user@lists.gentoo.org mailing list



^ permalink raw reply	[flat|nested] 6+ messages in thread

* RE: [gentoo-user] Gentoo router: Conntrack table full
  2008-03-23 13:42 ` Mike Williams
@ 2008-03-25  6:13   ` Adam Carter
  0 siblings, 0 replies; 6+ messages in thread
From: Adam Carter @ 2008-03-25  6:13 UTC (permalink / raw
  To: gentoo-user@lists.gentoo.org

The contents of the connection table is in
/proc/net/ip_conntrack

Example:
tcp      6 65 TIME_WAIT src=192.168.1.4 dst=20.x.y.40 sport=4986 dport=80 src=207.46.109.40 dst=192.168.1.4 sport=80 dport=4986 [ASSURED] mark=0 use=1

So go nuts with grep/awk/sed/sort/uniq etc to find what is consuming all the connections.

+1 for what Mike said about dropping the timeouts to something more sensible. FWIW Checkpoint uses a default TCP timer of 1 hour. Use 4 hours to be conservative.
--
gentoo-user@lists.gentoo.org mailing list



^ permalink raw reply	[flat|nested] 6+ messages in thread

end of thread, other threads:[~2008-03-25  6:15 UTC | newest]

Thread overview: 6+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2008-03-23  3:16 [gentoo-user] Gentoo router: Conntrack table full Dan Cowsill
2008-03-23  3:22 ` Andrey Falko
2008-03-23  3:26   ` Dan Cowsill
2008-03-23  9:23     ` Michal 'vorner' Vaner
2008-03-23 13:42 ` Mike Williams
2008-03-25  6:13   ` Adam Carter

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox