public inbox for gentoo-user-ru@lists.gentoo.org
 help / color / mirror / Atom feed
From: Konstantin Astafjev <konstantin@astafjev.com>
To: gentoo-user-ru@lists.gentoo.org
Subject: [gentoo-user-ru] Kernel Panic при высокой сетевой загрузке
Date: Thu, 26 Feb 2009 23:20:58 +0200	[thread overview]
Message-ID: <616168865.20090226232058@astafjev.com> (raw)

[-- Attachment #1: Type: text/plain, Size: 2158 bytes --]

Приветствую!

Вероятнее всего эта проблема не относится напрямую к Gentoo, но я
хотел бы сначала попросить совета у единственного известного мне
русскоязычного комьюнити. :)

Есть несколько маршрутизаторов на базе Intel Serverboard Aspen 3000AH,
основной функцией которых является шейпинг. Дерево HTB имеет не более
2 тысяч классов. Для фильтров используются hash tables. При суммарной
загрузке свыше 400 Mbps машинка начинает случайным образом
поругиваться и убивать процессы. В конечном итоге это всё приводит к
kernel panic. Пример ругани привожу в приложенном текстовом файле.
wr-up в примере это скрипт, который посредством запуска tc формиурет
классы и фильтры для шейпера. В данном случае он спровоцировал
убийство других процессов, но реально даже если ничего не делать
начинается: swapper: page allocation failure. order:0, mode:0x20
Налицо проблема в том, что кому-то не хватает памяти, судя по top'у
свободная память есть. На более сложные проверки у меня не хватает
знаний.

~ # top
top - 22:49:11 up 3 days,  3:06,  1 user,  load average: 0.06, 0.06, 0.01
Tasks:  85 total,   1 running,  84 sleeping,   0 stopped,   0 zombie
Cpu(s):  0.1%us,  0.1%sy,  0.0%ni, 92.2%id,  0.1%wa,  1.3%hi,  6.1%si,  0.0%st
Mem:   4148644k total,   832628k used,  3316016k free,      852k buffers
Swap:  4000056k total,      160k used,  3999896k free,    13100k cached

Чем больше трафик - тем выше вероятность ошибки. Иногда проходит 3
дня, а иногда достаточно и 5 минут. Вариант сбойного железа исключаю,
так как машинка не одна и на всех наблюдаются одинаковые симптомы тест
и проверку памяти они проходят исправно. Версии ядра менял начиная с
2.6.20, сейчас стоит 2.6.25-hardened-r10. Использование не hardened, а
gentoo ядра не помогло. Для сетевых интерфейсов использую модули e1000
и e1000e. Кажется в модуле e1000e видел совет увеличить значение
vm.min_free_kbytes. Увеличил до 65536, но в моём случае не помогло.

Может быть кто-нибудь сталкивался с такой проблемой и/или знает её
решение? В крайнем случае, подскажите как правильно дебагить такую
проблему и куда лучше обратиться. Спасибо.

-- 
Konstantin Astafjev

[-- Attachment #2: kern_log_error_parts.txt --]
[-- Type: text/plain, Size: 42679 bytes --]

wr-up invoked oom-killer: gfp_mask=0xd0, order=1, oomkilladj=0
Pid: 1874, comm: wr-up Not tainted 2.6.25-hardened-r10 #1
 [<c0440ce7>] oom_kill_process+0x4f/0x194
 [<c0441141>] out_of_memory+0x150/0x183
 [<c0442be6>] __alloc_pages+0x237/0x2bd
 [<c0442ca5>] __get_free_pages+0x39/0x47
 [<c041c0e0>] copy_process+0x97/0x108c
 [<c045c590>] alloc_file+0x24/0x2d
 [<c041d230>] do_fork+0xab/0x1a1
 [<c0420068>] do_wait+0x55c/0x8fb
 [<c0401db4>] sys_clone+0x28/0x2d
 [<c040352d>] sysenter_past_esp+0x66/0x99
 =======================
Mem-info:
DMA per-cpu:
CPU    0: hi:    0, btch:   1 usd:   0
CPU    1: hi:    0, btch:   1 usd:   0
CPU    2: hi:    0, btch:   1 usd:   0
CPU    3: hi:    0, btch:   1 usd:   0
Normal per-cpu:
CPU    0: hi:  186, btch:  31 usd:   0
CPU    1: hi:  186, btch:  31 usd:   0
CPU    2: hi:  186, btch:  31 usd:   0
CPU    3: hi:  186, btch:  31 usd:   0
HighMem per-cpu:
CPU    0: hi:  186, btch:  31 usd:   0
CPU    1: hi:  186, btch:  31 usd:   0
CPU    2: hi:  186, btch:  31 usd:   0
CPU    3: hi:  186, btch:  31 usd:   0
Active:8118 inactive:3847 dirty:0 writeback:0 unstable:0
 free:825642 slab:193769 mapped:2290 pagetables:232 bounce:0
DMA free:4648kB min:1168kB low:1460kB high:1752kB active:0kB inactive:0kB present:16256kB pages_scanned:0 all_unreclaimable? no
lowmem_reserve[]: 0 873 4556 4556
Normal free:75300kB min:64364kB low:80452kB high:96544kB active:300kB inactive:220kB present:894080kB pages_scanned:0 all_unreclaimable? no
lowmem_reserve[]: 0 0 29464 29464
HighMem free:3222620kB min:512kB low:68388kB high:136264kB active:32172kB inactive:15168kB present:3771392kB pages_scanned:0 all_unreclaimable? no
lowmem_reserve[]: 0 0 0 0
DMA: 298*4kB 0*8kB 26*16kB 15*32kB 6*64kB 1*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 4648kB
Normal: 10814*4kB 0*8kB 49*16kB 186*32kB 21*64kB 2*128kB 1*256kB 0*512kB 1*1024kB 1*2048kB 5*4096kB = 75400kB
HighMem: 494*4kB 834*8kB 647*16kB 312*32kB 163*64kB 104*128kB 43*256kB 22*512kB 4*1024kB 3*2048kB 766*4096kB = 3222776kB
6705 total pagecache pages
Swap cache: add 40, delete 0, find 0/0
Free swap  = 3999896kB
Total swap = 4000056kB
Free swap:       3999896kB
1179648 pages of RAM
950272 pages of HIGHMEM
142487 reserved pages
14575 pages shared
40 pages swap cached
0 pages dirty
0 pages writeback
2290 pages mapped
193769 pages slab
232 pages pagetables
Out of memory: kill process 26430 (apache2) score 30872 or a child
Killed process 26430 (apache2)
wr-up invoked oom-killer: gfp_mask=0x84d0, order=0, oomkilladj=0
Pid: 16400, comm: wr-up Not tainted 2.6.25-hardened-r10 #1
 [<c0440ce7>] oom_kill_process+0x4f/0x194
 [<c0441141>] out_of_memory+0x150/0x183
 [<c0442be6>] __alloc_pages+0x237/0x2bd
 [<c0412399>] pte_alloc_one+0x12/0x2d
 [<c044a162>] __pte_alloc+0x14/0xc3
 [<c044a781>] copy_page_range+0x1be/0x456
 [<c041c9fd>] copy_process+0x9b4/0x108c
 [<c041d230>] do_fork+0xab/0x1a1
 [<c0420068>] do_wait+0x55c/0x8fb
 [<c0401db4>] sys_clone+0x28/0x2d
 [<c040352d>] sysenter_past_esp+0x66/0x99
 [<c0680000>] i8042_create_aux_port+0x48/0xc6
 =======================
Mem-info:
DMA per-cpu:
CPU    0: hi:    0, btch:   1 usd:   0
CPU    1: hi:    0, btch:   1 usd:   0
CPU    2: hi:    0, btch:   1 usd:   0
CPU    3: hi:    0, btch:   1 usd:   0
Normal per-cpu:
CPU    0: hi:  186, btch:  31 usd: 181
CPU    1: hi:  186, btch:  31 usd: 172
CPU    2: hi:  186, btch:  31 usd: 208
CPU    3: hi:  186, btch:  31 usd: 142
HighMem per-cpu:
CPU    0: hi:  186, btch:  31 usd:  74
CPU    1: hi:  186, btch:  31 usd: 161
CPU    2: hi:  186, btch:  31 usd: 123
CPU    3: hi:  186, btch:  31 usd: 115
Active:7599 inactive:3866 dirty:0 writeback:0 unstable:0
 free:822956 slab:197269 mapped:2279 pagetables:177 bounce:0
DMA free:4816kB min:1168kB low:1460kB high:1752kB active:0kB inactive:0kB present:16256kB pages_scanned:0 all_unreclaimable? yes
lowmem_reserve[]: 0 873 4556 4556
Normal free:64308kB min:64364kB low:80452kB high:96544kB active:204kB inactive:296kB present:894080kB pages_scanned:845 all_unreclaimable? yes
lowmem_reserve[]: 0 0 29464 29464
HighMem free:3222700kB min:512kB low:68388kB high:136264kB active:30192kB inactive:15168kB present:3771392kB pages_scanned:0 all_unreclaimable? no
lowmem_reserve[]: 0 0 0 0
DMA: 286*4kB 25*8kB 28*16kB 15*32kB 6*64kB 1*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 4832kB
Normal: 6553*4kB 39*8kB 401*16kB 186*32kB 21*64kB 2*128kB 1*256kB 0*512kB 1*1024kB 1*2048kB 5*4096kB = 64300kB
HighMem: 395*4kB 859*8kB 639*16kB 314*32kB 163*64kB 107*128kB 43*256kB 22*512kB 4*1024kB 3*2048kB 766*4096kB = 3222900kB
6700 total pagecache pages
Swap cache: add 40, delete 0, find 0/0
Free swap  = 3999896kB
Total swap = 4000056kB
Free swap:       3999896kB
1179648 pages of RAM
950272 pages of HIGHMEM
142487 reserved pages
12685 pages shared
40 pages swap cached
0 pages dirty
0 pages writeback
2279 pages mapped
197269 pages slab
177 pages pagetables
Out of memory: kill process 26431 (apache2) score 30872 or a child
Killed process 26431 (apache2)
wr-up invoked oom-killer: gfp_mask=0x84d0, order=0, oomkilladj=0
Pid: 17072, comm: wr-up Not tainted 2.6.25-hardened-r10 #1
 [<c0440ce7>] oom_kill_process+0x4f/0x194
 [<c0441141>] out_of_memory+0x150/0x183
 [<c0442be6>] __alloc_pages+0x237/0x2bd
 [<c0442cef>] get_zeroed_page+0x3c/0x4a
 [<c0412715>] pgd_alloc+0x61/0xcc
 [<c041befd>] mm_init+0xa1/0xc3
 [<c041c844>] copy_process+0x7fb/0x108c
 [<c041d230>] do_fork+0xab/0x1a1
 [<c0420068>] do_wait+0x55c/0x8fb
 [<c0401db4>] sys_clone+0x28/0x2d
 [<c040352d>] sysenter_past_esp+0x66/0x99
 [<c0680000>] i8042_create_aux_port+0x48/0xc6
 =======================
Mem-info:
DMA per-cpu:
CPU    0: hi:    0, btch:   1 usd:   0
CPU    1: hi:    0, btch:   1 usd:   0
CPU    2: hi:    0, btch:   1 usd:   0
CPU    3: hi:    0, btch:   1 usd:   0
Normal per-cpu:
CPU    0: hi:  186, btch:  31 usd: 161
CPU    1: hi:  186, btch:  31 usd: 140
CPU    2: hi:  186, btch:  31 usd: 164
CPU    3: hi:  186, btch:  31 usd: 176
HighMem per-cpu:
CPU    0: hi:  186, btch:  31 usd:  50
CPU    1: hi:  186, btch:  31 usd: 158
CPU    2: hi:  186, btch:  31 usd:  49
CPU    3: hi:  186, btch:  31 usd: 138
Active:7011 inactive:3859 dirty:0 writeback:0 unstable:0
 free:823623 slab:197381 mapped:2302 pagetables:154 bounce:0
DMA free:4752kB min:1168kB low:1460kB high:1752kB active:0kB inactive:0kB present:16256kB pages_scanned:0 all_unreclaimable? yes
lowmem_reserve[]: 0 873 4556 4556
Normal free:64272kB min:64364kB low:80452kB high:96544kB active:340kB inactive:268kB present:894080kB pages_scanned:1088 all_unreclaimable? yes
lowmem_reserve[]: 0 0 29464 29464
HighMem free:3225468kB min:512kB low:68388kB high:136264kB active:27704kB inactive:15168kB present:3771392kB pages_scanned:0 all_unreclaimable? no
lowmem_reserve[]: 0 0 0 0
DMA: 284*4kB 25*8kB 29*16kB 16*32kB 7*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 4808kB
Normal: 6572*4kB 40*8kB 401*16kB 186*32kB 21*64kB 2*128kB 1*256kB 0*512kB 1*1024kB 1*2048kB 5*4096kB = 64384kB
HighMem: 668*4kB 890*8kB 616*16kB 323*32kB 174*64kB 108*128kB 45*256kB 22*512kB 4*1024kB 3*2048kB 766*4096kB = 3225504kB
6702 total pagecache pages
Swap cache: add 40, delete 0, find 0/0
Free swap  = 3999896kB
Total swap = 4000056kB
Free swap:       3999896kB
1179648 pages of RAM
950272 pages of HIGHMEM
142487 reserved pages
10550 pages shared
40 pages swap cached
0 pages dirty
0 pages writeback
2302 pages mapped
197381 pages slab
154 pages pagetables
Out of memory: kill process 26426 (apache2) score 3819 or a child
Killed process 26426 (apache2)
wr-up invoked oom-killer: gfp_mask=0x84d0, order=0, oomkilladj=0
Pid: 18061, comm: wr-up Not tainted 2.6.25-hardened-r10 #1
 [<c0440ce7>] oom_kill_process+0x4f/0x194
 [<c0441141>] out_of_memory+0x150/0x183
 [<c0442be6>] __alloc_pages+0x237/0x2bd
 [<c0442cef>] get_zeroed_page+0x3c/0x4a
 [<c0412715>] pgd_alloc+0x61/0xcc
 [<c041befd>] mm_init+0xa1/0xc3
 [<c045ec9b>] bprm_mm_init+0xf/0x143
 [<c045faf3>] do_execve+0xfa/0x350
 [<c044b3ae>] handle_mm_fault+0x603/0x66e
 [<c0684502>] do_page_fault+0x33c/0x769
 [<c0684569>] do_page_fault+0x3a3/0x769
 [<c0401d4b>] sys_execve+0x2a/0x4f
 [<c040352d>] sysenter_past_esp+0x66/0x99
 [<c0680000>] i8042_create_aux_port+0x48/0xc6
 =======================
Mem-info:
DMA per-cpu:
CPU    0: hi:    0, btch:   1 usd:   0
CPU    1: hi:    0, btch:   1 usd:   0
CPU    2: hi:    0, btch:   1 usd:   0
CPU    3: hi:    0, btch:   1 usd:   0
Normal per-cpu:
CPU    0: hi:  186, btch:  31 usd:  70
CPU    1: hi:  186, btch:  31 usd: 114
CPU    2: hi:  186, btch:  31 usd:  42
CPU    3: hi:  186, btch:  31 usd: 181
HighMem per-cpu:
CPU    0: hi:  186, btch:  31 usd:  98
CPU    1: hi:  186, btch:  31 usd: 168
CPU    2: hi:  186, btch:  31 usd: 141
CPU    3: hi:  186, btch:  31 usd:  75
Active:6678 inactive:3879 dirty:37 writeback:0 unstable:0
 free:823851 slab:197598 mapped:2270 pagetables:211 bounce:0
DMA free:4660kB min:1168kB low:1460kB high:1752kB active:0kB inactive:36kB present:16256kB pages_scanned:66 all_unreclaimable? yes
lowmem_reserve[]: 0 873 4556 4556
Normal free:64272kB min:64364kB low:80452kB high:96544kB active:228kB inactive:308kB present:894080kB pages_scanned:1344 all_unreclaimable? yes
lowmem_reserve[]: 0 0 29464 29464
HighMem free:3226472kB min:512kB low:68388kB high:136264kB active:26508kB inactive:15172kB present:3771392kB pages_scanned:0 all_unreclaimable? no
lowmem_reserve[]: 0 0 0 0
DMA: 258*4kB 26*8kB 31*16kB 19*32kB 4*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 4648kB
Normal: 6574*4kB 20*8kB 401*16kB 186*32kB 21*64kB 2*128kB 1*256kB 0*512kB 1*1024kB 1*2048kB 5*4096kB = 64232kB
HighMem: 634*4kB 951*8kB 620*16kB 319*32kB 178*64kB 106*128kB 45*256kB 23*512kB 4*1024kB 3*2048kB 766*4096kB = 3226304kB
6746 total pagecache pages
Swap cache: add 40, delete 0, find 0/0
Free swap  = 3999896kB
Total swap = 4000056kB
Free swap:       3999896kB
1179648 pages of RAM
950272 pages of HIGHMEM
142487 reserved pages
13250 pages shared
40 pages swap cached
37 pages dirty
0 pages writeback
2270 pages mapped
197598 pages slab
211 pages pagetables
Out of memory: kill process 17945 (apache2) score 61039 or a child
Killed process 17945 (apache2)
wr-up invoked oom-killer: gfp_mask=0x84d0, order=0, oomkilladj=0
Pid: 27365, comm: wr-up Not tainted 2.6.25-hardened-r10 #1
 [<c0440ce7>] oom_kill_process+0x4f/0x194
 [<c0441141>] out_of_memory+0x150/0x183
 [<c0442be6>] __alloc_pages+0x237/0x2bd
 [<c0412399>] pte_alloc_one+0x12/0x2d
 [<c044a162>] __pte_alloc+0x14/0xc3
 [<c044a781>] copy_page_range+0x1be/0x456
 [<c041c9fd>] copy_process+0x9b4/0x108c
 [<c041d230>] do_fork+0xab/0x1a1
 [<c0425839>] recalc_sigpending+0xb/0x1d
 [<c0425924>] sigprocmask+0xa1/0xbb
 [<c0401db4>] sys_clone+0x28/0x2d
 [<c040352d>] sysenter_past_esp+0x66/0x99
 [<c0680000>] i8042_create_aux_port+0x48/0xc6
 =======================
Mem-info:
DMA per-cpu:
CPU    0: hi:    0, btch:   1 usd:   0
CPU    1: hi:    0, btch:   1 usd:   0
CPU    2: hi:    0, btch:   1 usd:   0
CPU    3: hi:    0, btch:   1 usd:   0
Normal per-cpu:
CPU    0: hi:  186, btch:  31 usd:  20
CPU    1: hi:  186, btch:  31 usd:   4
CPU    2: hi:  186, btch:  31 usd: 107
CPU    3: hi:  186, btch:  31 usd:   6
HighMem per-cpu:
CPU    0: hi:  186, btch:  31 usd: 110
CPU    1: hi:  186, btch:  31 usd: 153
CPU    2: hi:  186, btch:  31 usd: 138
CPU    3: hi:  186, btch:  31 usd:  81
Active:6757 inactive:3783 dirty:0 writeback:0 unstable:0
 free:823831 slab:197916 mapped:2277 pagetables:211 bounce:0
DMA free:4652kB min:1168kB low:1460kB high:1752kB active:0kB inactive:0kB present:16256kB pages_scanned:4 all_unreclaimable? yes
lowmem_reserve[]: 0 873 4556 4556
Normal free:63980kB min:64364kB low:80452kB high:96544kB active:688kB inactive:0kB present:894080kB pages_scanned:1777 all_unreclaimable? yes
lowmem_reserve[]: 0 0 29464 29464
HighMem free:3226692kB min:512kB low:68388kB high:136264kB active:26344kB inactive:15168kB present:3771392kB pages_scanned:0 all_unreclaimable? no
lowmem_reserve[]: 0 0 0 0
DMA: 288*4kB 8*8kB 30*16kB 18*32kB 5*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 4640kB
Normal: 6494*4kB 7*8kB 405*16kB 186*32kB 21*64kB 2*128kB 1*256kB 0*512kB 1*1024kB 1*2048kB 5*4096kB = 63872kB
HighMem: 639*4kB 951*8kB 620*16kB 320*32kB 178*64kB 107*128kB 45*256kB 23*512kB 4*1024kB 3*2048kB 766*4096kB = 3226484kB
6711 total pagecache pages
Swap cache: add 40, delete 0, find 0/0
Free swap  = 3999896kB
Total swap = 4000056kB
Free swap:       3999896kB
1179648 pages of RAM
950272 pages of HIGHMEM
142487 reserved pages
13039 pages shared
40 pages swap cached
0 pages dirty
0 pages writeback
2277 pages mapped
197916 pages slab
211 pages pagetables
Out of memory: kill process 18742 (apache2) score 61039 or a child
Killed process 18742 (apache2)

..........[skipped]

sshd invoked oom-killer: gfp_mask=0xd0, order=0, oomkilladj=0
Pid: 19486, comm: sshd Not tainted 2.6.25-hardened-r10 #1
 [<c0440ce7>] oom_kill_process+0x4f/0x194
 [<c0441141>] out_of_memory+0x150/0x183
 [<c0442be6>] __alloc_pages+0x237/0x2bd
 [<c063b9d0>] tcp_sendmsg+0x4af/0x997
 [<c04342e2>] tick_dev_program_event+0x1e/0x82
 [<c060f160>] sock_aio_write+0xb3/0xc0
 [<c042f24a>] hrtimer_reprogram+0x83/0x9b
 [<c045b4c8>] do_sync_write+0xbf/0x100
 [<c0626a4c>] qdisc_watchdog_schedule+0x16/0x19
 [<c042cf60>] autoremove_wake_function+0x0/0x2d
 [<c0505920>] security_file_permission+0xc/0xd
 [<c045bbe0>] vfs_write+0x94/0xf6
 [<c045c105>] sys_write+0x3c/0x63
 [<c040352d>] sysenter_past_esp+0x66/0x99
 =======================
Mem-info:
DMA per-cpu:
CPU    0: hi:    0, btch:   1 usd:   0
CPU    1: hi:    0, btch:   1 usd:   0
CPU    2: hi:    0, btch:   1 usd:   0
CPU    3: hi:    0, btch:   1 usd:   0
Normal per-cpu:
CPU    0: hi:  186, btch:  31 usd: 132
CPU    1: hi:  186, btch:  31 usd: 179
CPU    2: hi:  186, btch:  31 usd: 143
CPU    3: hi:  186, btch:  31 usd: 128
HighMem per-cpu:
CPU    0: hi:  186, btch:  31 usd:  61
CPU    1: hi:  186, btch:  31 usd:  33
CPU    2: hi:  186, btch:  31 usd: 108
CPU    3: hi:  186, btch:  31 usd: 118
Active:17244 inactive:2522 dirty:1 writeback:0 unstable:0
 free:813843 slab:198617 mapped:1358 pagetables:115 bounce:0
DMA free:4648kB min:1168kB low:1460kB high:1752kB active:0kB inactive:100kB present:16256kB pages_scanned:177 all_unreclaimable? yes
lowmem_reserve[]: 0 873 4556 4556
Normal free:60360kB min:64364kB low:80452kB high:96544kB active:468kB inactive:136kB present:894080kB pages_scanned:942 all_unreclaimable? yes
lowmem_reserve[]: 0 0 29464 29464
HighMem free:3190364kB min:512kB low:68388kB high:136264kB active:68528kB inactive:9852kB present:3771392kB pages_scanned:0 all_unreclaimable? no
lowmem_reserve[]: 0 0 0 0
DMA: 110*4kB 85*8kB 36*16kB 17*32kB 6*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 4672kB
Normal: 5274*4kB 140*8kB 424*16kB 186*32kB 21*64kB 2*128kB 1*256kB 0*512kB 1*1024kB 1*2048kB 5*4096kB = 60360kB
HighMem: 1*4kB 1*8kB 1*16kB 2*32kB 0*64kB 0*128kB 52*256kB 35*512kB 15*1024kB 3*2048kB 766*4096kB = 3190364kB
4144 total pagecache pages
Swap cache: add 40, delete 0, find 0/0
Free swap  = 3999896kB
Total swap = 4000056kB
Free swap:       3999896kB
1179648 pages of RAM
950272 pages of HIGHMEM
142487 reserved pages
4961 pages shared
40 pages swap cached
1 pages dirty
0 pages writeback
1358 pages mapped
198617 pages slab
115 pages pagetables
Out of memory: kill process 19486 (sshd) score 2256 or a child
Killed process 19489 (bash)
....................................[skipped]...........
Feb  4 19:18:37 kernsrc@router swapper: page allocation failure. order:0, mode:0x20
Feb  4 19:18:37 kernsrc@router Pid: 0, comm: swapper Not tainted 2.6.25-hardened-r10 #1
Feb  4 19:18:37 kernsrc@router [<c0442c58>] __alloc_pages+0x2a9/0x2bd
Feb  4 19:18:37 kernsrc@router [<c0458d8b>] cache_alloc_refill+0x26b/0x454
Feb  4 19:18:37 kernsrc@router [<c0458fd6>] __kmalloc+0x62/0x9c
Feb  4 19:18:37 kernsrc@router [<c0614cb5>] __alloc_skb+0x49/0xf5
Feb  4 19:18:37 kernsrc@router [<c06157f0>] __netdev_alloc_skb+0x14/0x2d
Feb  4 19:18:37 kernsrc@router [<f8965945>] e1000_alloc_rx_buffers+0xa0/0x258 [e1000]
Feb  4 19:18:37 kernsrc@router [<f89663d5>] e1000_clean_rx_irq+0x427/0x460 [e1000]
Feb  4 19:18:37 kernsrc@router [<f8963cea>] e1000_clean+0x51/0x1c8 [e1000]
Feb  4 21:21:07 kernsrc@router [<c042fa86>] ktime_get_ts+0x11/0x3a
Feb  4 21:21:07 kernsrc@router [<c061aa9a>] net_rx_action+0x8a/0x157
Feb  4 21:21:07 kernsrc@router [<c0421be0>] __do_softirq+0x63/0xcc
Feb  4 21:21:07 kernsrc@router [<c0421c7e>] do_softirq+0x35/0x39
Feb  4 21:21:07 kernsrc@router [<c0421e8e>] irq_exit+0x38/0x6a
Feb  4 21:21:07 kernsrc@router [<c04059d1>] do_IRQ+0x73/0x82
Feb  4 21:21:07 kernsrc@router [<c0404503>] common_interrupt+0x23/0x30
Feb  4 21:21:07 kernsrc@router [<c0401e0b>] mwait_idle_with_hints+0x36/0x39
Feb  4 21:21:07 kernsrc@router [<c0401e0e>] mwait_idle+0x0/0xa
Feb  4 21:21:07 kernsrc@router [<c0402139>] cpu_idle+0x6d/0x86
Feb  4 21:21:07 kernsrc@router =======================
Feb  4 21:21:07 kernsrc@router Mem-info:
Feb  4 21:21:07 kernsrc@router DMA per-cpu:
Feb  4 21:21:07 kernsrc@router CPU    0: hi:    0, btch:   1 usd:   0
Feb  4 21:21:07 kernsrc@router CPU    1: hi:    0, btch:   1 usd:   0
Feb  4 21:21:07 kernsrc@router CPU    2: hi:    0, btch:   1 usd:   0
Feb  4 21:21:07 kernsrc@router CPU    3: hi:    0, btch:   1 usd:   0
Feb  4 21:21:07 kernsrc@router Normal per-cpu:
Feb  4 21:21:07 kernsrc@router CPU    0: hi:  186, btch:  31 usd:   0
Feb  4 21:21:07 kernsrc@router CPU    1: hi:  186, btch:  31 usd:   0
Feb  4 21:21:07 kernsrc@router CPU    2: hi:  186, btch:  31 usd:   0
Feb  4 21:21:07 kernsrc@router CPU    3: hi:  186, btch:  31 usd:   0
Feb  4 21:21:07 kernsrc@router HighMem per-cpu:
Feb  4 21:21:07 kernsrc@router CPU    0: hi:  186, btch:  31 usd:   0
Feb  4 21:21:07 kernsrc@router CPU    1: hi:  186, btch:  31 usd:   0
Feb  4 21:21:07 kernsrc@router CPU    2: hi:  186, btch:  31 usd:   0
Feb  4 21:21:07 kernsrc@router CPU    3: hi:  186, btch:  31 usd:   0
Feb  4 21:21:07 kernsrc@router Active:5522 inactive:3048 dirty:0 writeback:0 unstable:0
Feb  4 21:21:07 kernsrc@router free:816292 slab:208254 mapped:1614 pagetables:112 bounce:0
Feb  4 21:21:07 kernsrc@router DMA free:3920kB min:1168kB low:1460kB high:1752kB active:24kB inactive:0kB present:16256kB pages_scanned:36 all_unreclaimable? no
Feb  4 21:21:07 kernsrc@router lowmem_reserve[]: 0 873 4556 4556
Feb  4 21:21:07 kernsrc@router Normal free:24048kB min:64364kB low:80452kB high:96544kB active:640kB inactive:748kB present:894080kB pages_scanned:1291 all_unreclaimable? no
Feb  4 21:21:07 kernsrc@router lowmem_reserve[]: 0 0 29464 29464
Feb  4 21:21:07 kernsrc@router HighMem free:3237200kB min:512kB low:68388kB high:136264kB active:21424kB inactive:11444kB present:3771392kB pages_scanned:0 all_unreclaimable? no
Feb  4 21:21:07 kernsrc@router lowmem_reserve[]: 0 0 0 0
Feb  4 21:21:07 kernsrc@router DMA: 1*4kB 2*8kB 2*16kB 4*32kB 4*64kB 3*128kB 2*256kB 1*512kB 0*1024kB 1*2048kB 0*4096kB = 3892kB
Feb  4 21:21:07 kernsrc@router Normal: 2*4kB 2*8kB 1*16kB 3*32kB 1*64kB 0*128kB 0*256kB 1*512kB 1*1024kB 1*2048kB 5*4096kB = 24264kB
Feb  4 21:21:07 kernsrc@router HighMem: 1038*4kB 505*8kB 243*16kB 83*32kB 39*64kB 12*128kB 6*256kB 5*512kB 5*1024kB 3*2048kB 782*4096kB = 3237200kB
Feb  4 21:21:07 kernsrc@router 6029 total pagecache pages
Feb  4 21:21:07 kernsrc@router Swap cache: add 0, delete 0, find 0/0
Feb  4 21:21:07 kernsrc@router Free swap  = 4000056kB
Feb  4 21:21:07 kernsrc@router Total swap = 4000056kB
Feb  4 21:21:07 kernsrc@router Free swap:       4000056kB
Feb  4 21:21:07 kernsrc@router swapper: page allocation failure. order:0, mode:0x20
Feb  4 21:21:07 kernsrc@router Pid: 0, comm: swapper Not tainted 2.6.25-hardened-r10 #1
Feb  4 21:21:07 kernsrc@router [<c0442c58>] __alloc_pages+0x2a9/0x2bd
Feb  4 21:21:07 kernsrc@router [<c0458d8b>] cache_alloc_refill+0x26b/0x454
Feb  4 21:21:07 kernsrc@router [<c0458fd6>] __kmalloc+0x62/0x9c
Feb  4 21:21:07 kernsrc@router [<f8cb00c3>] match+0x328/0x501 [xt_layer7]
Feb  4 21:21:07 kernsrc@router [<f8ca80c3>] tos_tg_v0+0x6c/0x91 [xt_DSCP]
Feb  4 21:21:07 kernsrc@router [<f8c56278>] ipt_do_table+0x1b7/0x427 [ip_tables]
Feb  4 21:21:07 kernsrc@router [<c062d268>] nf_iterate+0x30/0x63
Feb  4 21:21:07 kernsrc@router [<c062d2f9>] nf_hook_slow+0x3a/0x90
Feb  4 21:21:07 kernsrc@router [<f89a59cc>] br_nf_pre_routing_finish+0x0/0x291 [bridge]
Feb  4 21:21:07 kernsrc@router [<f89a6661>] br_nf_pre_routing+0x520/0x547 [bridge]
Feb  4 21:21:07 kernsrc@router [<f89a59cc>] br_nf_pre_routing_finish+0x0/0x291 [bridge]
Feb  4 21:21:07 kernsrc@router [<c062d268>] nf_iterate+0x30/0x63
Feb  4 21:21:07 kernsrc@router [<f89a2596>] br_handle_frame_finish+0x0/0x104 [bridge]
Feb  4 21:21:07 kernsrc@router [<c062d2f9>] nf_hook_slow+0x3a/0x90
Feb  4 21:21:07 kernsrc@router [<f89a2596>] br_handle_frame_finish+0x0/0x104 [bridge]
Feb  4 21:21:07 kernsrc@router [<f89a283f>] br_handle_frame+0x1a5/0x1ca [bridge]
Feb  4 21:21:07 kernsrc@router [<f89a2596>] br_handle_frame_finish+0x0/0x104 [bridge]
Feb  4 21:21:07 kernsrc@router [<c0618a1f>] netif_receive_skb+0x268/0x391
Feb  4 21:21:07 kernsrc@router [<f89112d7>] e1000_receive_skb+0x12f/0x149 [e1000e]
Feb  4 21:21:07 kernsrc@router [<f8913291>] e1000_clean_rx_irq+0x20e/0x2b1 [e1000e]
Feb  4 21:21:07 kernsrc@router [<f8910429>] e1000_clean+0x4c/0x1b1 [e1000e]
Feb  4 21:21:07 kernsrc@router [<c061aa9a>] net_rx_action+0x8a/0x157
Feb  4 21:21:07 kernsrc@router [<c0421be0>] __do_softirq+0x63/0xcc
Feb  4 21:21:07 kernsrc@router [<c0421c7e>] do_softirq+0x35/0x39
Feb  4 21:21:07 kernsrc@router [<c0421e8e>] irq_exit+0x38/0x6a
Feb  4 21:21:07 kernsrc@router [<c04059d1>] do_IRQ+0x73/0x82
Feb  4 21:21:07 kernsrc@router [<c0404503>] common_interrupt+0x23/0x30
Feb  4 21:21:07 kernsrc@router [<c0401e0b>] mwait_idle_with_hints+0x36/0x39
Feb  4 21:21:07 kernsrc@router [<c0401e0e>] mwait_idle+0x0/0xa
Feb  4 21:21:07 kernsrc@router [<c0402139>] cpu_idle+0x6d/0x86
Feb  4 21:21:07 kernsrc@router =======================
Feb  4 21:21:07 kernsrc@router Mem-info:
Feb  4 21:21:07 kernsrc@router DMA per-cpu:
Feb  4 21:21:07 kernsrc@router CPU    0: hi:    0, btch:   1 usd:   0
Feb  4 21:21:07 kernsrc@router CPU    1: hi:    0, btch:   1 usd:   0
Feb  4 21:21:07 kernsrc@router CPU    2: hi:    0, btch:   1 usd:   0
Feb  4 21:21:07 kernsrc@router CPU    3: hi:    0, btch:   1 usd:   0
Feb  4 21:21:07 kernsrc@router Normal per-cpu:
Feb  4 21:21:07 kernsrc@router CPU    0: hi:  186, btch:  31 usd:   0
Feb  4 21:21:07 kernsrc@router CPU    1: hi:  186, btch:  31 usd:   0
Feb  4 21:21:07 kernsrc@router CPU    2: hi:  186, btch:  31 usd:   0
Feb  4 21:21:07 kernsrc@router CPU    3: hi:  186, btch:  31 usd:   0
Feb  4 21:21:07 kernsrc@router HighMem per-cpu:
Feb  4 21:21:07 kernsrc@router CPU    0: hi:  186, btch:  31 usd:   0
Feb  4 21:21:07 kernsrc@router CPU    1: hi:  186, btch:  31 usd:   0
Feb  4 21:21:07 kernsrc@router CPU    2: hi:  186, btch:  31 usd:   0
Feb  4 21:21:07 kernsrc@router CPU    3: hi:  186, btch:  31 usd:   0
Feb  4 21:21:07 kernsrc@router Active:5522 inactive:3048 dirty:0 writeback:0 unstable:0
Feb  4 21:21:07 kernsrc@router free:816292 slab:208254 mapped:1614 pagetables:112 bounce:0
Feb  4 21:21:07 kernsrc@router DMA free:3920kB min:1168kB low:1460kB high:1752kB active:24kB inactive:0kB present:16256kB pages_scanned:36 all_unreclaimable? no
Feb  4 21:21:07 kernsrc@router lowmem_reserve[]: 0 873 4556 4556
Feb  4 21:21:07 kernsrc@router Normal free:24048kB min:64364kB low:80452kB high:96544kB active:640kB inactive:748kB present:894080kB pages_scanned:1291 all_unreclaimable? no
Feb  4 21:21:07 kernsrc@router lowmem_reserve[]: 0 0 29464 29464
Feb  4 21:21:07 kernsrc@router HighMem free:3237200kB min:512kB low:68388kB high:136264kB active:21424kB inactive:11444kB present:3771392kB pages_scanned:0 all_unreclaimable? no
Feb  4 21:21:07 kernsrc@router lowmem_reserve[]: 0 0 0 0
Feb  4 21:21:07 kernsrc@router DMA: 1*4kB 2*8kB 2*16kB 4*32kB 4*64kB 3*128kB 2*256kB 1*512kB 0*1024kB 1*2048kB 0*4096kB = 3892kB
Feb  4 21:21:07 kernsrc@router Normal: 2*4kB 2*8kB 1*16kB 3*32kB 1*64kB 0*128kB 0*256kB 1*512kB 1*1024kB 1*2048kB 5*4096kB = 24264kB
Feb  4 21:21:07 kernsrc@router HighMem: 1038*4kB 505*8kB 243*16kB 83*32kB 39*64kB 12*128kB 6*256kB 5*512kB 5*1024kB 3*2048kB 782*4096kB = 3237200kB
Feb  4 21:21:07 kernsrc@router 6029 total pagecache pages
Feb  4 21:21:07 kernsrc@router Swap cache: add 0, delete 0, find 0/0
Feb  4 21:21:07 kernsrc@router Free swap  = 4000056kB
Feb  4 21:21:07 kernsrc@router Total swap = 4000056kB
Feb  4 21:21:07 kernsrc@router Free swap:       4000056kB
Feb  4 21:21:07 kernsrc@router 1179648 pages of RAM
Feb  4 21:21:07 kernsrc@router 1179648 pages of RAM
Feb  4 21:21:07 kernsrc@router 950272 pages of HIGHMEM
Feb  4 21:21:07 kernsrc@router 142487 reserved pages
Feb  4 21:21:07 kernsrc@router 9788 pages shared
Feb  4 21:21:07 kernsrc@router 0 pages swap cached
Feb  4 21:21:07 kernsrc@router 0 pages dirty
Feb  4 21:21:07 kernsrc@router 950272 pages of HIGHMEM
Feb  4 21:21:07 kernsrc@router 0 pages writeback
Feb  4 21:21:07 kernsrc@router 1614 pages mapped
Feb  4 21:21:07 kernsrc@router 208254 pages slab
Feb  4 21:21:07 kernsrc@router 112 pages pagetables
Feb  4 21:21:07 kernsrc@router 142487 reserved pages
Feb  4 21:21:07 kernsrc@router 9788 pages shared
Feb  4 21:21:07 kernsrc@router 0 pages swap cached
Feb  4 21:21:07 kernsrc@router 0 pages dirty
Feb  4 21:21:07 kernsrc@router 0 pages writeback
Feb  4 21:21:07 kernsrc@router 1614 pages mapped
Feb  4 21:21:07 kernsrc@router 208254 pages slab
Feb  4 21:21:07 kernsrc@router 112 pages pagetables
Feb  4 21:21:07 kernsrc@router layer7: out of memory in match, bailing.
Feb  4 21:21:07 kernsrc@router swapper: page allocation failure. order:0, mode:0x20
Feb  4 21:21:07 kernsrc@router Pid: 0, comm: swapper Not tainted 2.6.25-hardened-r10 #1
Feb  4 21:21:07 kernsrc@router [<c0442c58>] __alloc_pages+0x2a9/0x2bd
Feb  4 21:21:07 kernsrc@router [<c0458d8b>] cache_alloc_refill+0x26b/0x454
Feb  4 21:21:07 kernsrc@router [<c04045c8>] apic_timer_interrupt+0x28/0x30
Feb  4 21:21:07 kernsrc@router [<c0458fd6>] __kmalloc+0x62/0x9c
Feb  4 21:21:07 kernsrc@router [<f8cb00c3>] match+0x328/0x501 [xt_layer7]
Feb  4 21:21:07 kernsrc@router [<f8ca80c3>] tos_tg_v0+0x6c/0x91 [xt_DSCP]
Feb  4 21:21:07 kernsrc@router [<f8c56278>] ipt_do_table+0x1b7/0x427 [ip_tables]
Feb  4 21:21:07 kernsrc@router [<c062d268>] nf_iterate+0x30/0x63
Feb  4 21:21:07 kernsrc@router [<c062d2f9>] nf_hook_slow+0x3a/0x90
Feb  4 21:21:07 kernsrc@router [<f89a59cc>] br_nf_pre_routing_finish+0x0/0x291 [bridge]
Feb  4 21:21:07 kernsrc@router [<f89a6661>] br_nf_pre_routing+0x520/0x547 [bridge]
Feb  4 21:21:07 kernsrc@router [<f89a59cc>] br_nf_pre_routing_finish+0x0/0x291 [bridge]
Feb  4 21:21:07 kernsrc@router [<c062d268>] nf_iterate+0x30/0x63
Feb  4 21:21:07 kernsrc@router [<f89a2596>] br_handle_frame_finish+0x0/0x104 [bridge]
Feb  4 21:21:07 kernsrc@router [<c062d2f9>] nf_hook_slow+0x3a/0x90
Feb  4 21:21:07 kernsrc@router [<f89a2596>] br_handle_frame_finish+0x0/0x104 [bridge]
Feb  4 21:21:07 kernsrc@router [<f89a283f>] br_handle_frame+0x1a5/0x1ca [bridge]
Feb  4 21:21:07 kernsrc@router [<f89a2596>] br_handle_frame_finish+0x0/0x104 [bridge]
Feb  4 21:21:07 kernsrc@router [<c0618a1f>] netif_receive_skb+0x268/0x391
Feb  4 21:21:07 kernsrc@router [<f89112d7>] e1000_receive_skb+0x12f/0x149 [e1000e]
Feb  4 21:21:07 kernsrc@router [<f8913291>] e1000_clean_rx_irq+0x20e/0x2b1 [e1000e]
Feb  4 21:21:07 kernsrc@router [<f8910429>] e1000_clean+0x4c/0x1b1 [e1000e]
Feb  4 21:21:07 kernsrc@router [<c061aa9a>] net_rx_action+0x8a/0x157
Feb  4 21:21:07 kernsrc@router [<c0421be0>] __do_softirq+0x63/0xcc
Feb  4 21:21:07 kernsrc@router [<c0421c7e>] do_softirq+0x35/0x39
Feb  4 21:21:07 kernsrc@router [<c0421e8e>] irq_exit+0x38/0x6a
Feb  4 21:21:07 kernsrc@router [<c04059d1>] do_IRQ+0x73/0x82
Feb  4 21:21:07 kernsrc@router [<c0404503>] common_interrupt+0x23/0x30
Feb  4 21:21:07 kernsrc@router [<c0401e0b>] mwait_idle_with_hints+0x36/0x39
Feb  4 21:21:07 kernsrc@router [<c0401e0e>] mwait_idle+0x0/0xa
Feb  4 21:21:07 kernsrc@router [<c0402139>] cpu_idle+0x6d/0x86
Feb  4 21:21:07 kernsrc@router =======================
Feb  4 21:21:07 kernsrc@router Mem-info:
Feb  4 21:21:07 kernsrc@router DMA per-cpu:
Feb  4 21:21:07 kernsrc@router CPU    0: hi:    0, btch:   1 usd:   0
Feb  4 21:21:07 kernsrc@router CPU    1: hi:    0, btch:   1 usd:   0
Feb  4 21:21:07 kernsrc@router CPU    2: hi:    0, btch:   1 usd:   0
Feb  4 21:21:07 kernsrc@router CPU    3: hi:    0, btch:   1 usd:   0
Feb  4 21:21:07 kernsrc@router Normal per-cpu:
Feb  4 21:21:07 kernsrc@router CPU   swap = 4000056kB
Feb  4 21:21:07 kernsrc@router Free swap:       4000056kB
Feb  4 21:21:07 kernsrc@router 1179648 pages of RAM
Feb  4 21:21:07 kernsrc@router 950272 pages of HIGHMEM
Feb  4 21:21:07 kernsrc@router 142487 reserved pages
Feb  4 21:21:07 kernsrc@router 9793 pages shared
Feb  4 21:21:07 kernsrc@router 0 pages swap cached
Feb  4 21:21:07 kernsrc@router 0 pages dirty
Feb  4 21:21:07 kernsrc@router 0 pages writeback
Feb  4 21:21:07 kernsrc@router 1619 pages mapped
Feb  4 21:21:07 kernsrc@router 208296 pages slab
Feb  4 21:21:07 kernsrc@router 112 pages pagetables
Feb  4 21:21:07 kernsrc@router printk: 102551 messages suppressed.
Feb  4 21:21:07 kernsrc@router swapper: page allocation failure. order:0, mode:0x20
Feb  4 21:21:07 kernsrc@router Pid: 0, comm: swapper Not tainted 2.6.25-hardened-r10 #1
Feb  4 21:21:07 kernsrc@router [<c0442c58>] __alloc_pages+0x2a9/0x2bd
Feb  4 21:21:07 kernsrc@router [<c0458d8b>] cache_alloc_refill+0x26b/0x454
Feb  4 21:21:07 kernsrc@router [<f89a283f>] br_handle_frame+0x1a5/0x1ca [bridge]
Feb  4 21:21:07 kernsrc@router [<c0458fd6>] __kmalloc+0x62/0x9c
Feb  4 21:21:07 kernsrc@router [<c0614cb5>] __alloc_skb+0x49/0xf5
Feb  4 21:21:07 kernsrc@router [<c06157f0>] __netdev_alloc_skb+0x14/0x2d
Feb  4 21:21:07 kernsrc@router [<f8966178>] e1000_clean_rx_irq+0x1ca/0x460 [e1000]
Feb  4 21:21:07 kernsrc@router [<f8963cea>] e1000_clean+0x51/0x1c8 [e1000]
Feb  4 21:21:07 kernsrc@router [<c061aa9a>] net_rx_action+0x8a/0x157
Feb  4 21:21:07 kernsrc@router [<c0421be0>] __do_softirq+0x63/0xcc
Feb  4 21:21:07 kernsrc@router [<c0421c7e>] do_softirq+0x35/0x39
Feb  4 21:21:07 kernsrc@router [<c0421e8e>] irq_exit+0x38/0x6a
Feb  4 21:21:07 kernsrc@router [<c040ed01>] smp_apic_timer_interrupt+0x75/0x80
Feb  4 21:21:07 kernsrc@router [<c04045c8>] apic_timer_interrupt+0x28/0x30
Feb  4 21:21:07 kernsrc@router [<c0401e0b>] mwait_idle_with_hints+0x36/0x39
Feb  4 21:21:07 kernsrc@router [<c0401e0e>] mwait_idle+0x0/0xa
Feb  4 21:21:07 kernsrc@router [<c0402139>] cpu_idle+0x6d/0x86
Feb  4 21:21:07 kernsrc@router =======================
Feb  4 21:21:07 kernsrc@router Mem-info:
Feb  4 21:21:07 kernsrc@router DMA per-cpu:
Feb  4 21:21:07 kernsrc@router CPU    0: hi:    0, btch:   1 usd:   0
Feb  4 21:21:07 kernsrc@router CPU    1: hi:    0, btch:   1 usd:   0
Feb  4 21:21:07 kernsrc@router CPU    2: hi:    0, btch:   1 usd:   0
Feb  4 21:21:07 kernsrc@router CPU    3: hi:    0, btch:   1 usd:   0
Feb  4 21:21:07 kernsrc@router Normal per-cpu:
Feb  4 21:21:07 kernsrc@router CPU    0: hi:  186, btch:  31 usd:   0
Feb  4 21:21:07 kernsrc@router CPU    1: hi:  186, btch:  31 usd:   0
Feb  4 21:21:07 kernsrc@router CPU    2: hi:  186, btch:  31 usd:   0
Feb  4 21:21:07 kernsrc@router CPU    3: hi:  186, btch:  31 usd:   0
Feb  4 21:21:07 kernsrc@router HighMem per-cpu:
Feb  4 21:21:07 kernsrc@router CPU    0: hi:  186, btch:  31 usd:   0
Feb  4 21:21:07 kernsrc@router CPU    1: hi:  186, btch:  31 usd:   0
Feb  4 21:21:07 kernsrc@router CPU    2: hi:  186, btch:  31 usd:   0
Feb  4 21:21:07 kernsrc@router CPU    3: hi:  186, btch:  31 usd:   0
Feb  4 21:21:07 kernsrc@router Active:5590 inactive:3009 dirty:0 writeback:0 unstable:0
Feb  4 21:21:07 kernsrc@router free:816313 slab:208296 mapped:1619 pagetables:112 bounce:0
Feb  4 21:21:07 kernsrc@router DMA free:3912kB min:1168kB low:1460kB high:1752kB active:12kB inactive:36kB present:16256kB pages_scanned:99 all_unreclaimable? no
Feb  4 21:21:07 kernsrc@router lowmem_reserve[]: 0 873 4556 4556
Feb  4 21:21:07 kernsrc@router Normal free:24140kB min:64364kB low:80452kB high:96544kB active:904kB inactive:576kB present:894080kB pages_scanned:2415 all_unreclaimable? yes
Feb  4 21:21:07 kernsrc@router lowmem_reserve[]: 0 0 29464 29464
Feb  4 21:21:07 kernsrc@router HighMem free:3237200kB min:512kB low:68388kB high:136264kB active:21444kB inactive:11424kB present:3771392kB pages_scanned:0 all_unreclaimable? no
Feb  4 21:21:07 kernsrc@router lowmem_reserve[]: 0 0 0 0
Feb  4 21:21:07 kernsrc@router DMA: 6*4kB 2*8kB 2*16kB 4*32kB 4*64kB 3*128kB 2*256kB 1*512kB 0*1024kB 1*2048kB 0*4096kB = 3912kB
Feb  4 21:21:07 kernsrc@router Normal: 31*4kB 0*8kB 1*16kB 0*32kB 1*64kB 1*128kB 1*256kB 0*512kB 1*1024kB 1*2048kB 5*4096kB = 24140kB
Feb  4 21:21:07 kernsrc@router HighMem: 1038*4kB 505*8kB 243*16kB 83*32kB 39*64kB 12*128kB 6*256kB 5*512kB 5*1024kB 3*2048kB 782*4096kB = 3237200kB
Feb  4 21:21:07 kernsrc@router 6029 total pagecache pages
Feb  4 21:21:07 kernsrc@router Swap cache: add 0, delete 0, find 0/0
Feb  4 21:21:07 kernsrc@router Free swap  = 4000056kB
Feb  4 21:21:07 kernsrc@router Total swap = 4000056kB
Feb  4 21:21:07 kernsrc@router Free swap:       4000056kB
Feb  4 21:21:07 kernsrc@router 1179648 pages of RAM
Feb  4 21:21:07 kernsrc@router 950272 pages of HIGHMEM
Feb  4 21:21:07 kernsrc@router 142487 reserved pages
Feb  4 21:21:07 kernsrc@router 9793 pages shared
Feb  4 21:21:07 kernsrc@router 0 pages swap cached
Feb  4 21:21:07 kernsrc@router 0 pages dirty
Feb  4 21:21:07 kernsrc@router 0 pages writeback
Feb  4 21:21:07 kernsrc@router 1619 pages mapped
Feb  4 21:21:07 kernsrc@router 208296 pages slab
Feb  4 21:21:07 kernsrc@router 112 pages pagetables
Feb  4 21:21:07 kernsrc@router printk: 95995 messages suppressed.
Feb  4 21:21:07 kernsrc@router swapper: page allocation failure. order:0, mode:0x20
Feb  4 21:21:07 kernsrc@router Pid: 0, comm: swapper Not tainted 2.6.25-hardened-r10 #1
Feb  4 21:21:07 kernsrc@router [<c0442c58>] __alloc_pages+0x2a9/0x2bd
Feb  4 21:21:07 kernsrc@router [<c0458d8b>] cache_alloc_refill+0x26b/0x454
Feb  4 21:21:07 kernsrc@router [<f89a283f>] br_handle_frame+0x1a5/0x1ca [bridge]
Feb  4 21:21:07 kernsrc@router [<c0458fd6>] __kmalloc+0x62/0x9c
Feb  4 21:21:07 kernsrc@router [<c0614cb5>] __alloc_skb+0x49/0xf5
Feb  4 21:21:07 kernsrc@router [<c06157f0>] __netdev_alloc_skb+0x14/0x2d
Feb  4 21:21:07 kernsrc@router [<f8966178>] e1000_clean_rx_irq+0x1ca/0x460 [e1000]
Feb  4 21:21:07 kernsrc@router [<f8963cea>] e1000_clean+0x51/0x1c8 [e1000]
Feb  4 21:21:07 kernsrc@router [<c061aa9a>] net_rx_action+0x8a/0x157
Feb  4 21:21:07 kernsrc@router [<c0421be0>] __do_softirq+0x63/0xcc
Feb  4 21:21:07 kernsrc@router [<c0421c7e>] do_softirq+0x35/0x39
Feb  4 21:21:07 kernsrc@router [<c0421e8e>] irq_exit+0x38/0x6a
Feb  4 21:21:07 kernsrc@router [<c040ed01>] smp_apic_timer_interrupt+0x75/0x80
Feb  4 21:21:07 kernsrc@router [<c04045c8>] apic_timer_interrupt+0x28/0x30
Feb  4 21:21:07 kernsrc@router [<c0401e0b>] mwait_idle_with_hints+0x36/0x39
Feb  4 21:21:07 kernsrc@router [<c0401e0e>] mwait_idle+0x0/0xa
Feb  4 21:21:07 kernsrc@router [<c0402139>] cpu_idle+0x6d/0x86
Feb  4 21:21:07 kernsrc@router =======================
Feb  4 21:21:07 kernsrc@router Mem-info:
Feb  4 21:21:07 kernsrc@router DMA per-cpu:
Feb  4 21:21:07 kernsrc@router CPU    0: hi:    0, btch:   1 usd:   0
Feb  4 21:21:07 kernsrc@router CPU    1: hi:    0, btch:   1 usd:   0
Feb  4 21:21:07 kernsrc@router CPU    2: hi:    0, btch:   1 usd:   0
Feb  4 21:21:07 kernsrc@router CPU    3: hi:    0, btch:   1 usd:   0
Feb  4 21:21:07 kernsrc@router Normal per-cpu:
Feb  4 21:21:07 kernsrc@router CPU    0: hi:  186, btch:  31 usd:   0
Feb  4 21:21:07 kernsrc@router CPU    1: hi:  186, btch:  31 usd:   0
Feb  4 21:21:07 kernsrc@router CPU    2: hi:  186, btch:  31 usd:   0
Feb  4 21:21:07 kernsrc@router CPU    3: hi:  186, btch:  31 usd:   0
Feb  4 21:21:07 kernsrc@router HighMem per-cpu:
Feb  4 21:21:07 kernsrc@router CPU    0: hi:  186, btch:  31 usd:   0
Feb  4 21:21:07 kernsrc@router CPU    1: hi:  186, btch:  31 usd:   0
Feb  4 21:21:07 kernsrc@router CPU    2: hi:  186, btch:  31 usd:   0
Feb  4 21:21:07 kernsrc@router CPU    3: hi:  186, btch:  31 usd:   0
Feb  4 21:21:07 kernsrc@router Active:5581 inactive:3000 dirty:0 writeback:0 unstable:0
Feb  4 21:21:07 kernsrc@router free:816289 slab:208295 mapped:1619 pagetables:112 bounce:0
Feb  4 21:21:07 kernsrc@router DMA free:3912kB min:1168kB low:1460kB high:1752kB active:4kB inactive:36kB present:16256kB pages_scanned:72 all_unreclaimable? yes
Feb  4 21:21:07 kernsrc@router lowmem_reserve[]: 0 873 4556 4556
Feb  4 21:21:07 kernsrc@router Normal free:24044kB min:64364kB low:80452kB high:96544kB active:876kB inactive:540kB present:894080kB pages_scanned:2535 all_unreclaimable? yes
Feb  4 21:21:07 kernsrc@router lowmem_reserve[]: 0 0 29464 29464
Feb  4 21:21:07 kernsrc@router HighMem free:3237200kB min:512kB low:68388kB high:136264kB active:21444kB inactive:11424kB present:3771392kB pages_scanned:0 all_unreclaimable? no
Feb  4 21:21:07 kernsrc@router lowmem_reserve[]: 0 0 0 0
Feb  4 21:21:07 kernsrc@router DMA: 6*4kB 2*8kB 2*16kB 4*32kB 4*64kB 3*128kB 2*256kB 1*512kB 0*1024kB 1*2048kB 0*4096kB = 3912kB
Feb  4 21:21:07 kernsrc@router Normal: 31*4kB 0*8kB 1*16kB 0*32kB 1*64kB 1*128kB 1*256kB 0*512kB 1*1024kB 1*2048kB 5*4096kB = 24140kB
Feb  4 21:21:07 kernsrc@router HighMem: 1038*4kB 505*8kB 243*16kB 83*32kB 39*64kB 12*128kB 6*256kB 5*512kB 5*1024kB 3*2048kB 782*4096kB = 3237200kB
Feb  4 21:21:07 kernsrc@router 6029 total pagecache pages
Feb  4 21:21:07 kernsrc@router Swap cache: add 0, delete 0, find 0/0
Feb  4 21:21:07 kernsrc@router Free swap  = 4000056kB
Feb  4 21:21:07 kernsrc@router Total swap = 4000056kB
Feb  4 21:21:07 kernsrc@router Free swap:       4000056kB
Feb  4 21:21:07 kernsrc@router 1179648 pages of RAM
Feb  4 21:21:07 kernsrc@router 950272 pages of HIGHMEM
Feb  4 21:21:07 kernsrc@router 142487 reserved pages
Feb  4 21:21:07 kernsrc@router 9793 pages shared
Feb  4 21:21:07 kernsrc@router 0 pages swap cached
Feb  4 21:21:07 kernsrc@router 0 pages dirty
Feb  4 21:21:07 kernsrc@router 0 pages writeback
Feb  4 21:21:07 kernsrc@router 1619 pages mapped
Feb  4 21:21:07 kernsrc@router 208295 pages slab
Feb  4 21:21:07 kernsrc@router 112 pages pagetables
Feb  4 21:21:07 kernsrc@router printk: 53439 messages suppressed.
Feb  4 21:21:07 kernsrc@router swapper: page allocation failure. order:0, mode:0x20
Feb  4 21:21:07 kernsrc@router Pid: 0, comm: swapper Not tainted 2.6.25-hardened-r10 #1
Feb  4 21:21:07 kernsrc@router [<c0442c58>] __alloc_pages+0x2a9/0x2bd
Feb  4 21:21:07 kernsrc@router [<c04342e2>] tick_dev_program_event+0x1e/0x82
Feb  4 21:21:07 kernsrc@router [<c0458d8b>] cache_alloc_refill+0x26b/0x454
Feb  4 21:21:07 kernsrc@router [<c0434395>] tick_program_event+0x1f/0x23
Feb  4 21:21:07 kernsrc@router [<c0458fd6>] __kmalloc+0x62/0x9c
Feb  4 21:21:07 kernsrc@router [<c0614cb5>] __alloc_skb+0x49/0xf5
Feb  4 21:21:07 kernsrc@router [<c06157f0>] __netdev_alloc_skb+0x14/0x2d
Feb  4 21:21:07 kernsrc@router [<f8966178>] e1000_clean_rx_irq+0x1ca/0x460 [e1000]
Feb  4 21:21:07 kernsrc@router [<f8963cea>] e1000_clean+0x51/0x1c8 [e1000]
Feb  4 21:21:07 kernsrc@router [<c042fa86>] ktime_get_ts+0x11/0x3a
Feb  4 21:21:07 kernsrc@router [<c061aa9a>] net_rx_action+0x8a/0x157
Feb  4 21:21:07 kernsrc@router [<c0421be0>] __do_softirq+0x63/0xcc
Feb  4 21:21:07 kernsrc@router [<c0421c7e>] do_softirq+0x35/0x39
Feb  4 21:21:07 kernsrc@router [<c0421e8e>] irq_exit+0x38/0x6a
Feb  4 21:21:07 kernsrc@router [<c04059d1>] do_IRQ+0x73/0x82
Feb  4 21:21:07 kernsrc@router [<c0404503>] common_interrupt+0x23/0x30
Feb  4 21:21:07 kernsrc@router [<c0401e0b>] mwait_idle_with_hints+0x36/0x39
Feb  4 21:21:07 kernsrc@router [<c0401e0e>] mwait_idle+0x0/0xa
Feb  4 21:21:07 kernsrc@router [<c0402139>] cpu_idle+0x6d/0x86
Feb  4 21:21:07 kernsrc@router =======================
Feb  4 21:21:07 kernsrc@router Mem-info:
Feb  4 21:21:07 kernsrc@router DMA per-cpu:
Feb  4 21:21:07 kernsrc@router CPU    0: hi:    0, btch:   1 usd:   0
Feb  4 21:21:07 kernsrc@router CPU    1: hi:    0, btch:   1 usd:   0
Feb  4 21:21:07 kernsrc@router CPU    2: hi:    0, btch:   1 usd:   0
Feb  4 21:21:07 kernsrc@router CPU    3: hi:    0, btch:   1 usd:   0
Feb  4 21:21:07 kernsrc@router Normal per-cpu:
Feb  4 21:21:07 kernsrc@router CPU    0: hi:  186, btch:  31 usd:   0
Feb  4 21:21:07 kernsrc@router CPU    1: hi:  186, btch:  31 usd:   0
Feb  4 21:21:07 kernsrc@router CPU    2: hi:  186, btch:  31 usd:   0
Feb  4 21:21:07 kernsrc@router CPU    3: hi:  186, btch:  31 usd:   0
Feb  4 21:21:07 kernsrc@router HighMem per-cpu:
Feb  4 21:21:07 kernsrc@router CPU    0: hi:  186, btch:  31 usd:   0
Feb  4 21:21:07 kernsrc@router CPU    1: hi:  186, btch:  31 usd:   0
Feb  4 21:21:07 kernsrc@router CPU    2: hi:  186, btch:  31 usd:   0
Feb  4 21:21:07 kernsrc@router CPU    3: hi:  186, btch:  31 usd:   0
Feb  4 21:21:07 kernsrc@router Active:5606 inactive:2936 dirty:0 writeback:0 unstable:0
Feb  4 21:21:07 kernsrc@router free:816313 slab:208296 mapped:1619 pagetables:112 bounce:0
Feb  4 21:21:07 kernsrc@router DMA free:3912kB min:1168kB low:1460kB high:1752kB active:12kB inactive:0kB present:16256kB pages_scanned:45 all_unreclaimable? yes
Feb  4 21:21:07 kernsrc@router lowmem_reserve[]: 0 873 4556 4556
Feb  4 21:21:07 kernsrc@router Normal free:24140kB min:64364kB low:80452kB high:96544kB active:968kB inactive:320kB present:894080kB pages_scanned:2514 all_unreclaimable? no
Feb  4 21:21:07 kernsrc@router lowmem_reserve[]: 0 0 29464 29464
Feb  4 21:21:07 kernsrc@router HighMem free:3237200kB min:512kB low:68388kB high:136264kB active:21444kB inactive:11424kB present:3771392kB pages_scanned:0 all_unreclaimable? no
Feb  4 21:21:07 kernsrc@router lowmem_reserve[]: 0 0 0 0
...............................[skipped]

             reply	other threads:[~2009-02-26 21:21 UTC|newest]

Thread overview: 5+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2009-02-26 21:20 Konstantin Astafjev [this message]
2009-02-26 21:22 ` [gentoo-user-ru] Kernel Panic при высокой сетевой загрузке Алексей Белкин
2009-02-27 15:40 ` [gentoo-user-ru] " alexey conovalov
2009-02-28 12:12 ` Peter Volkov
2009-02-28 12:34   ` Dmitry Pisarev

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=616168865.20090226232058@astafjev.com \
    --to=konstantin@astafjev.com \
    --cc=gentoo-user-ru@lists.gentoo.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox