From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from lists.gentoo.org (pigeon.gentoo.org [208.92.234.80]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by finch.gentoo.org (Postfix) with ESMTPS id 7CD9C1382C5 for ; Thu, 13 May 2021 13:10:33 +0000 (UTC) Received: from pigeon.gentoo.org (localhost [127.0.0.1]) by pigeon.gentoo.org (Postfix) with SMTP id 4F151E08C8; Thu, 13 May 2021 13:10:21 +0000 (UTC) Received: from mail-qv1-xf2c.google.com (mail-qv1-xf2c.google.com [IPv6:2607:f8b0:4864:20::f2c]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by pigeon.gentoo.org (Postfix) with ESMTPS id E140EE0874 for ; Thu, 13 May 2021 13:10:20 +0000 (UTC) Received: by mail-qv1-xf2c.google.com with SMTP id w9so13689958qvi.13 for ; Thu, 13 May 2021 06:10:20 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to; bh=v2xDHwRnS2mIaVrTV1nSh4ElpQ6L3NtjUVWOmjKLVZc=; b=nP4pm/EQl+qbSpRGe4dU7Tl+svANhNrxn2baRkH+NFPU0F+bqXSaYLZ57ik38GF6nZ etIkVBhMcxkP0zazsnjzfs5pJxZNOvpYnCOCXJRIjTLrW2c0VX1P4Vvbs1+Yw94SaFsj RKliOFx0wkcZHlf/2L0+5L2smKinjWc9OVNPH7HdJlGVr2FS5lN2BpfPd3WGlGgqqKSn Y/vEE+QuzQPZlrpEGTEt/4sN3orOwJVErsCJHQ3gCZkVRREnvFSTcFHUatY3IiUov6Bs xLpEUAtk0icvPWVaMPwEK9qlVwh08UfiqeQ9T47AA31DrqDr7XwQuIESi53zPg7qHxB8 RAqg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to; bh=v2xDHwRnS2mIaVrTV1nSh4ElpQ6L3NtjUVWOmjKLVZc=; b=TNhnjVetmwYt91sB3GWSbb2QO3/MrhLw61gP6LjmFYrLw7hoFxjsEJy7yTTtfKQn3s 0q7+nO3w6GlXJ0w1E+7GQe0FpHELSsEa00n8Kb4XZ3Mz0xq83/u7tu/kcvKjdGo4pbyr 6d+sHrkVnXtzdAmXBN7f6pKG4gIIIzyJNlnEJw2DNz0apzpwH75fS+QcVH2aSLlUUYjA K+0IsW/pMMD1XqUWmj+6aXxYMzD8U2zN0U35Z0idNjFIfNuZIDoEDuDtn51DGK2+mgBP Iblrx3J2MqBiuB5QVpkaKUZsH3MV6ihCAtURJHl9rohJ+GmDWY2d13y0ebI6fsYSV7P7 Kvrg== X-Gm-Message-State: AOAM530kTeRtYq04lcxisfryL0Dm3duAyGzYTNW48oJFbB/NucZI0X/V OEvq2sISnckfVT1ewEVYvRxkZXd1OZ+tsa+/70z/E9mHl4E= X-Google-Smtp-Source: ABdhPJwu2dYCAIJTo6G+O9//I4bsynQoh4y+oQxgrcwQc7btHPqQE295Vh3xi3Htt4CRkYJXz2ITCw3vREsqtz24GIQ= X-Received: by 2002:a0c:e052:: with SMTP id y18mr40490199qvk.15.1620911419647; Thu, 13 May 2021 06:10:19 -0700 (PDT) Precedence: bulk List-Post: List-Help: List-Unsubscribe: List-Subscribe: List-Id: Gentoo Linux mail X-BeenThere: gentoo-user@lists.gentoo.org Reply-to: gentoo-user@lists.gentoo.org X-Auto-Response-Suppress: DR, RN, NRN, OOF, AutoReply MIME-Version: 1.0 References: In-Reply-To: From: Mark Knecht Date: Thu, 13 May 2021 06:10:08 -0700 Message-ID: Subject: Re: [gentoo-user] Approx monthly hard lockups To: Gentoo User Content-Type: multipart/alternative; boundary="00000000000061699e05c235d934" X-Archives-Salt: b27cf996-85a3-48e7-9d9d-107a37904be7 X-Archives-Hash: 2503db9e4ff52d3ea59e4c15b129ffde --00000000000061699e05c235d934 Content-Type: text/plain; charset="UTF-8" On Thu, May 13, 2021 at 12:20 AM Walter Dnes wrote: > > On Wed, May 12, 2021 at 09:50:29PM -0500, Dale wrote > > Walter Dnes wrote: > > > On Wed, May 12, 2021 at 11:16:56AM -0700, Mark Knecht wrote > > >> Is there nothing in the system logs? What's near the end before whatever > > >> you get from booting after this issue? > > > Where would those logfiles be? "dmesg" starts off with... > > > > > > > > > Most likely in /var/log/messages. The bad thing about dmesg, I think it > > resets when rebooting which erases previous info. > > > > As Mark pointed out, you need to go back to the time before it starts > > adding the boot up process. > > No help. /var/log/messages goes straight from iptables messages to > bootup with no indication of shutdown... > > May 12 11:26:31 i3 kernel: FECESBOOK:IN= OUT=eth0 SRC=192.168.1.249 DST=31.13.80.12 LEN=60 TOS=0x00 PREC=0x00 TTL=64 ID=5824 DF PROTO=TCP SPT=50566 DPT=443 WINDOW=64170 RES=0x00 SYN URGP=0 > May 12 11:26:32 i3 kernel: FECESBOOK:IN= OUT=eth0 SRC=192.168.1.249 DST=31.13.80.12 LEN=60 TOS=0x00 PREC=0x00 TTL=64 ID=60152 DF PROTO=TCP SPT=50568 DPT=443 WINDOW=64170 RES=0x00 SYN URGP=0 > May 12 11:34:04 i3 syslog-ng[1384]: syslog-ng starting up; version='3.30.1' > May 12 11:34:04 i3 crond[1414]: /usr/sbin/crond 4.5 dillon's cron daemon, started with loglevel notice > May 12 11:34:04 i3 /usr/sbin/gpm[1443]: *** info [daemon/startup.c(136)]: > May 12 11:34:04 i3 /usr/sbin/gpm[1443]: Started gpm successfully. Entered daemon mode. > > When doing a regular shutdown I get stuff like... > > May 12 11:42:31 i3 shutdown[1974]: shutting down for system reboot > May 12 11:42:31 i3 init[1]: Switching to runlevel: 6 > May 12 11:42:32 i3 init[1]: Trying to re-exec init > May 12 11:42:32 i3 start-stop-daemon[2053]: Will stop /usr/sbin/sshd > May 12 11:42:32 i3 start-stop-daemon[2053]: Will stop PID 1764 > May 12 11:42:32 i3 start-stop-daemon[2053]: Sending signal 15 to PID 1764 > May 12 11:42:32 i3 sshd[1764]: Received signal 15; terminating. > OK, but those messages are yesterday, May 12th, at 11:42 in the morning. Being that you started this thread on May 12th at 9:35 (when I received the first email anyway) the actual lockup presumably occurred earlier. I believe you're in the correct file, or set of files. Depending on how your logging works (what logger, what options) generally the logs roll over into successive files. (*.1, *.2, *.3, etc) Assuming you have some idea when the lockup occurred you want to look for what was going on just before that. Do the logs hit a date/time and stop? Or do they go on logging with lots of similar or repeating messages for hours or days? Do double check that you log into a directory that doesn't run out of space once a month. That could cause problems also. HTH, MArk --00000000000061699e05c235d934 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable


On Thu, May 13, 2021 at 12:20 AM Walter Dnes <<= a href=3D"mailto:waltdnes@waltdnes.org">waltdnes@waltdnes.org> wrote= :
>
> On Wed, May 12, 2021 at 09:50:29PM -0500, Dale wrote
&= gt; > Walter Dnes wrote:
> > > On Wed, May 12, 2021 at 11:16= :56AM -0700, Mark Knecht wrote
> > >> Is there nothing in th= e system logs? What's near the end before whatever
> > >>= ; you get from booting after this issue?
> > > =C2=A0 Where wou= ld those logfiles be? =C2=A0"dmesg" starts off with...
> &g= t; >
> >
> >
> > Most likely in /var/log/mess= ages. The bad thing about dmesg, I think it
> > resets when reboot= ing which erases previous info.
> >
> > As Mark pointed o= ut, you need to go back to the time before it starts
> > adding th= e boot up process.
>
> =C2=A0 No help. =C2=A0/var/log/messages = goes straight from iptables messages to
> bootup with no indication o= f shutdown...
>
> May 12 11:26:31 i3 kernel: FECESBOOK:IN=3D OU= T=3Deth0 SRC=3D192.168.1.249 DST=3D31.13.80.12 LEN=3D60 TOS=3D0x00 PREC=3D0= x00 TTL=3D64 ID=3D5824 DF PROTO=3DTCP SPT=3D50566 DPT=3D443 WINDOW=3D64170 = RES=3D0x00 SYN URGP=3D0
> May 12 11:26:32 i3 kernel: FECESBOOK:IN=3D = OUT=3Deth0 SRC=3D192.168.1.249 DST=3D31.13.80.12 LEN=3D60 TOS=3D0x00 PREC= =3D0x00 TTL=3D64 ID=3D60152 DF PROTO=3DTCP SPT=3D50568 DPT=3D443 WINDOW=3D6= 4170 RES=3D0x00 SYN URGP=3D0
> May 12 11:34:04 i3 syslog-ng[1384]: sy= slog-ng starting up; version=3D'3.30.1'
> May 12 11:34:04 i3 = crond[1414]: /usr/sbin/crond 4.5 dillon's cron daemon, started with log= level notice
> May 12 11:34:04 i3 /usr/sbin/gpm[1443]: *** info [daem= on/startup.c(136)]:
> May 12 11:34:04 i3 /usr/sbin/gpm[1443]: Started= gpm successfully. Entered daemon mode.
>
> =C2=A0 When doing a= regular shutdown I get stuff like...
>
> May 12 11:42:31 i3 sh= utdown[1974]: shutting down for system reboot
> May 12 11:42:31 i3 in= it[1]: Switching to runlevel: 6
> May 12 11:42:32 i3 init[1]: Trying = to re-exec init
> May 12 11:42:32 i3 start-stop-daemon[2053]: Will st= op /usr/sbin/sshd
> May 12 11:42:32 i3 start-stop-daemon[2053]: Will = stop PID 1764
> May 12 11:42:32 i3 start-stop-daemon[2053]: Sending s= ignal 15 to PID 1764
> May 12 11:42:32 i3 sshd[1764]: Received signal= 15; terminating.
>

OK, but those messages are= yesterday, May 12th, at 11:42 in the morning. Being that you started this = thread on May 12th at 9:35 (when I received the first email anyway) the act= ual lockup presumably occurred earlier.

I believe = you're in the correct file, or set of files. Depending on how your logg= ing works (what logger, what options) generally the logs roll over into suc= cessive files. (*.1, *.2, *.3, etc) Assuming you have some idea when the lo= ckup occurred you want to look for what was going on just before that. Do t= he logs hit a date/time and stop? Or do they go on logging with lots of sim= ilar or repeating messages for hours or days?

Do d= ouble check that you log into a directory that doesn't run out of space= once a month. That could cause problems also.

HTH= ,
MArk
--00000000000061699e05c235d934--