[gentoo-user] /var/log/messages is huge

[gentoo-user] /var/log/messages is huge

[dhk]

Last week I got a message the my /var partition was almost full.  Since
it was 10G and I am using lvm2 I added 5G more to it.  This evening I
got the same message, now I had to investigate.  When I did I found that
/var/log/messages is huge.

# ls -lh /var/log/messages
-rw------- 1 root root 12G Aug 16 20:15 /var/log/messages

It seem to be constantly written to and growing.

# ls -l /var/log/messages
-rw------- 1 root root 11986603021 Aug 16 20:20 /var/log/messages
# ls -l /var/log/messages
-rw------- 1 root root 11986687279 Aug 16 20:20 /var/log/messages
# ls -l /var/log/messages
-rw------- 1 root root 11986759997 Aug 16 20:21 /var/log/messages
# ls -l /var/log/messages
-rw------- 1 root root 11986783403 Aug 16 20:21 /var/log/messages
# ls -l /var/log/messages
-rw------- 1 root root 11986830215 Aug 16 20:21 /var/log/messages

When I tail it I get the following.
# tail /var/log/messages
Aug 16 20:22:59 host kernel: usb-storage: -- transfer complete
Aug 16 20:22:59 host kernel: usb-storage: Bulk command transfer result=0
Aug 16 20:22:59 host kernel: usb-storage: Attempting to get CSW...
Aug 16 20:22:59 host kernel: usb-storage: usb_stor_bulk_transfer_buf:
xfer 13 bytes
Aug 16 20:22:59 host kernel: usb-storage: Status code 0; transferred 13/13
Aug 16 20:22:59 host kernel: usb-storage: -- transfer complete
Aug 16 20:22:59 host kernel: usb-storage: Bulk status result = 0
Aug 16 20:22:59 host kernel: usb-storage: Bulk Status S 0x53425355 T
0x5c33 R 0 Stat 0x0
Aug 16 20:22:59 host kernel: usb-storage: scsi cmd done, result=0x0
Aug 16 20:22:59 host kernel: usb-storage: *** thread sleeping.

What's going on here?  Why all these usb-storage messages?

I don't know exactly when this started, but I'm guessing it has to do
with a recent upgrade.

I also thought there's suppose to be some sort of log rotation going on
to prevent this.

Any ideas?

Right now I'm going to "cat /dev/null > /var/log/messages" so I can boot
up in the morning.

Now my /var dropped to 16%.

Thanks,

dhk

Re: [gentoo-user] /var/log/messages is huge

[Adam Carter]

[-- Attachment #1: Type: text/plain, Size: 882 bytes --]

> Aug 16 20:22:59 host kernel: usb-storage: -- transfer complete
> Aug 16 20:22:59 host kernel: usb-storage: Bulk command transfer result=0
> Aug 16 20:22:59 host kernel: usb-storage: Attempting to get CSW...
> Aug 16 20:22:59 host kernel: usb-storage: usb_stor_bulk_transfer_buf:
> xfer 13 bytes
> Aug 16 20:22:59 host kernel: usb-storage: Status code 0; transferred 13/13
> Aug 16 20:22:59 host kernel: usb-storage: -- transfer complete
> Aug 16 20:22:59 host kernel: usb-storage: Bulk status result = 0
> Aug 16 20:22:59 host kernel: usb-storage: Bulk Status S 0x53425355 T
> 0x5c33 R 0 Stat 0x0
> Aug 16 20:22:59 host kernel: usb-storage: scsi cmd done, result=0x0
> Aug 16 20:22:59 host kernel: usb-storage: *** thread sleeping.
>
> What's going on here?  Why all these usb-storage messages?
>

You've probably turned on CONFIG_USB_STORAGE_DEBUG when you configured the
kernel

[-- Attachment #2: Type: text/html, Size: 1139 bytes --]

Re: [gentoo-user] /var/log/messages is huge

[Florian CROUZAT]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1


On 17 août 2010, at 02:58, Adam Carter wrote:

>> Aug 16 20:22:59 host kernel: usb-storage: -- transfer complete
>> Aug 16 20:22:59 host kernel: usb-storage: Bulk command transfer result=0
>> Aug 16 20:22:59 host kernel: usb-storage: Attempting to get CSW...
>> Aug 16 20:22:59 host kernel: usb-storage: usb_stor_bulk_transfer_buf:
>> xfer 13 bytes
>> Aug 16 20:22:59 host kernel: usb-storage: Status code 0; transferred 13/13
>> Aug 16 20:22:59 host kernel: usb-storage: -- transfer complete
>> Aug 16 20:22:59 host kernel: usb-storage: Bulk status result = 0
>> Aug 16 20:22:59 host kernel: usb-storage: Bulk Status S 0x53425355 T
>> 0x5c33 R 0 Stat 0x0
>> Aug 16 20:22:59 host kernel: usb-storage: scsi cmd done, result=0x0
>> Aug 16 20:22:59 host kernel: usb-storage: *** thread sleeping.
>> 
>> What's going on here?  Why all these usb-storage messages?
> 
> You've probably turned on CONFIG_USB_STORAGE_DEBUG when you configured the kernel

Yeah and not only it makes your logs huge, but it also slows down your box using your CPU to flood while doing USB transfer.
On my old box, with CONFIG_USB_STORAGE_DEBUG set, syslog-ng eats 25% of my CPU when doing ntfs3g.
Totally something you want to turn off, unless you have serious issues with USB.

- ----
Florian
/ For security reasons, all text in this mail 
  is double-rot13 encrypted. /

-----BEGIN PGP SIGNATURE-----

iQEcBAEBAgAGBQJMai/3AAoJEI54KfPdrpQaiwIH/RC+6gjvSLzTbZE42A+GZWIb
jHljx/e/d9vCmV/65QxrhCUaFfUR4SlTCBvnmxBQxohDDtSzE6S6JbTxzk22obn+
tqqTOwb62HFBOs4TVNyPfNsjlhR+7BAcmRcff7d6XOJiiiak3gTXe8cnu2hnaAma
3Mq4GSE8LvokTaFvxCx0R9/NBkm7olZAl8UT73ibbIwxSfW+nkl2qlTv2xIVLMd2
u7nLycPd26cUHXFTH/FYFERg9tfXQ+VeKc3e5IoQIS7gkIQGLdUFOrzI2UkzhwrV
RbHBFn2BM7o3i40ImzIqHTEBdDT4AmZ2skKbhQy/U6W1xhczqW6XBQ5SLm4buIY=
=wVf4
-----END PGP SIGNATURE-----

Re: [gentoo-user] /var/log/messages is huge

[dhk]

On 08/17/2010 02:45 AM, Florian CROUZAT wrote:
> 
> On 17 août 2010, at 02:58, Adam Carter wrote:
> 
>>> Aug 16 20:22:59 host kernel: usb-storage: -- transfer complete
>>> Aug 16 20:22:59 host kernel: usb-storage: Bulk command transfer result=0
>>> Aug 16 20:22:59 host kernel: usb-storage: Attempting to get CSW...
>>> Aug 16 20:22:59 host kernel: usb-storage: usb_stor_bulk_transfer_buf:
>>> xfer 13 bytes
>>> Aug 16 20:22:59 host kernel: usb-storage: Status code 0; transferred 13/13
>>> Aug 16 20:22:59 host kernel: usb-storage: -- transfer complete
>>> Aug 16 20:22:59 host kernel: usb-storage: Bulk status result = 0
>>> Aug 16 20:22:59 host kernel: usb-storage: Bulk Status S 0x53425355 T
>>> 0x5c33 R 0 Stat 0x0
>>> Aug 16 20:22:59 host kernel: usb-storage: scsi cmd done, result=0x0
>>> Aug 16 20:22:59 host kernel: usb-storage: *** thread sleeping.
>>>
>>> What's going on here?  Why all these usb-storage messages?
> 
>> You've probably turned on CONFIG_USB_STORAGE_DEBUG when you configured the kernel
> 
> Yeah and not only it makes your logs huge, but it also slows down your box using your CPU to flood while doing USB transfer.
> On my old box, with CONFIG_USB_STORAGE_DEBUG set, syslog-ng eats 25% of my CPU when doing ntfs3g.
> Totally something you want to turn off, unless you have serious issues with USB.
> 
> ----
> Florian
> / For security reasons, all text in this mail 
>   is double-rot13 encrypted. /
> 

Yes, I have it turned on in the kernel.
# grep CONFIG_USB_STORAGE_DEBUG
/etc/kernels/kernel-config-x86_64-2.6.32-gentoo-r7
CONFIG_USB_STORAGE_DEBUG=y

I don't understand why it just started to be a problem.  I've had this
kernel for almost four months.  Shouldn't the log's be rotated and purged?

I've been using the default kernel config from the install.  I'll make a
new kernel with it turned off.

Thanks,

dhk

Re: [gentoo-user] /var/log/messages is huge

[Alex Schuster]

dhk writes:

> I don't understand why it just started to be a problem.  I've had this
> kernel for almost four months.  Shouldn't the log's be rotated and
> purged?

Depends on your system logger. Metalog does this automatically, but for 
others you probably have to install logrotate.

	Wonko

Re: [gentoo-user] /var/log/messages is huge

[dhk]

On 08/17/2010 07:17 AM, Alex Schuster wrote:
> dhk writes:
> 
>> I don't understand why it just started to be a problem.  I've had this
>> kernel for almost four months.  Shouldn't the log's be rotated and
>> purged?
> 
> Depends on your system logger. Metalog does this automatically, but for 
> others you probably have to install logrotate.
> 
> 	Wonko
> 
> 
> 
Right, I need logrotate.  I though it was installed.  Must have been on
the old system.

Thanks,

dhk

Re: [gentoo-user] /var/log/messages is huge

[Peter Humphrey]

On Tuesday 17 August 2010 11:57:13 dhk wrote:

> I don't understand why it just started to be a problem.  I've had
> this kernel for almost four months.

That's how long it's taken to fill your disk up.

-- 
Rgds
Peter.          Linux Counter 5290, 1994-04-23.