From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from lists.gentoo.org (pigeon.gentoo.org [208.92.234.80]) by finch.gentoo.org (Postfix) with ESMTP id 7659F138A87 for ; Mon, 23 Feb 2015 19:36:18 +0000 (UTC) Received: from pigeon.gentoo.org (localhost [127.0.0.1]) by pigeon.gentoo.org (Postfix) with SMTP id 3973EE08C8; Mon, 23 Feb 2015 19:36:13 +0000 (UTC) Received: from mail-ie0-f171.google.com (mail-ie0-f171.google.com [209.85.223.171]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by pigeon.gentoo.org (Postfix) with ESMTPS id B3568E079B for ; Mon, 23 Feb 2015 19:36:11 +0000 (UTC) Received: by iecrp18 with SMTP id rp18so26266575iec.1 for ; Mon, 23 Feb 2015 11:36:11 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :content-type; bh=OeMrA/K/gZjlbDPWZIebvmQT4di/LWavDsBciSRdYBw=; b=UV0krxHzGBSyqiVPOpq1Q42orMwb/JQANhbnrS+pqFaGLYYAIk7ePAari8Gg3P4QpF sBoiIkV9xuqVLfb0mXFFtt8qddshJOUL2OfN5AuM6ISFWH5LxXOE44LTDvmOwfhgiy4r 5oilbrRpYXfPJvM0EHCTtfkyYIr3dC7j6yN/dS9yojvShiHnMdlevBGtF43ShOTFNssk mll97nOUWCTxqLURgs3gRDzFBS+s1DyJFZv27qoaz16+kU23NUj7ixPhbC1wXoePrMu2 sHQSMCUadGz1wXdv1fQZwpN6R0taDG6jx3wCMa4+Rslu8i7yUyS5SFFz0XGrhf829sT9 YcEA== X-Received: by 10.42.188.133 with SMTP id da5mr13371034icb.37.1424720171055; Mon, 23 Feb 2015 11:36:11 -0800 (PST) Precedence: bulk List-Post: List-Help: List-Unsubscribe: List-Subscribe: List-Id: Gentoo Linux mail X-BeenThere: gentoo-user@lists.gentoo.org Reply-to: gentoo-user@lists.gentoo.org MIME-Version: 1.0 Received: by 10.107.149.149 with HTTP; Mon, 23 Feb 2015 11:35:50 -0800 (PST) In-Reply-To: <18633.1424719880@ccs.covici.com> References: <87lhjws8ci.fsf@heimdali.yagibdah.de> <28267.1424201355@ccs.covici.com> <87d257q7en.fsf@heimdali.yagibdah.de> <20150218223115.7fb56f66@digimed.co.uk> <87vbitldj5.fsf@heimdali.yagibdah.de> <20150223091529.656c0008@marcec.fritz.box> <16447.1424680874@ccs.covici.com> <4133.1424713749@ccs.covici.com> <20150223201946.36e90fed@marcec.fritz.box> <18633.1424719880@ccs.covici.com> From: =?UTF-8?B?Q2FuZWsgUGVsw6FleiBWYWxkw6lz?= Date: Mon, 23 Feb 2015 13:35:50 -0600 Message-ID: Subject: Re: [gentoo-user] syslog-ng: how to read the log files To: gentoo-user@lists.gentoo.org Content-Type: multipart/alternative; boundary=20cf3020769ab34025050fc683fd X-Archives-Salt: a75a1162-070b-41b2-b0ea-23c43c176690 X-Archives-Hash: adcad0a282a0d8fb99767c6c04e748f7 --20cf3020769ab34025050fc683fd Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable On Mon, Feb 23, 2015 at 1:31 PM, wrote: > > Marc Joliet wrote: > > > Am Mon, 23 Feb 2015 12:10:18 -0600 > > schrieb Canek Pel=C3=A1ez Vald=C3=A9s : > > > > > On Mon, Feb 23, 2015 at 11:49 AM, wrote: > > > > > > > > Canek Pel=C3=A1ez Vald=C3=A9s wrote: > > > > > > > > > On Mon, Feb 23, 2015 at 3:41 AM, wrote: > > > > > > > > > > > > Marc Joliet wrote: > > > > > > > > > > > > > Am Mon, 23 Feb 2015 00:41:50 +0100 > > > > > > > schrieb lee : > > > > > > > > > > > > > > > Neil Bothwick writes: > > > > > > > > > > > > > > > > > On Wed, 18 Feb 2015 21:49:54 +0100, lee wrote: > > > > > > > > > > > > > > > > > >> > I wonder if the OP is using systemd and trying to read the > > > > > journal > > > > > > > > >> > files? > > > > > > > > >> > > > > > > > > >> Nooo, I hate systemd ... > > > > > > > > >> > > > > > > > > >> What good are log files you can't read? > > > > > > > > > > > > > > > > > > You can't read syslog-ng log files without some reading > > > software, > > > > > usually > > > > > > > > > a combination of cat, grep and less. systemd does it all with > > > > > journalctl. > > > > > > > > > > > > > > > > > > There are good reasons to not use systemd, this isn't one of > > > them. > > > > > > > > > > > > > > > > To me it is one of the good reasons, and an important one. Plain > > > text > > > > > > > > can usually always be read without further ado, be it from rescue > > > > > > > > systems you booted or with software available on different > > > operating > > > > > > > > systems. It can be also be processed with scripts and sent as > > > email. > > > > > > > > You can probably even read it on your cell phone. You can still > > > read > > > > > > > > log files that were created 20 years ago when they are plain text. > > > > > > > > > > > > > > > > Can you do all that with the binary files created by systemd? I > > > can't > > > > > > > > even read them on a working system. > > > > > > > > > > > > > > What Canek and Rich already said is good, but I'll just add this: > > > it's > > > > > not like > > > > > > > you can't run a classic syslog implementation alongside the systemd > > > > > journal. > > > > > > > On my systems, by *default*, syslog-ng kept working as usual, > > > getting > > > > > the logs > > > > > > > from the systemd journal. If you want to go further, you can even > > > > > configure > > > > > > > the journal to not store logs permanently, so that you *only* end up > > > > > with > > > > > > > plain-text logs on your system (Duncan on gentoo-amd64 went this > > > way). > > > > > > > > > > > > > > So no, the format that the systemd journal uses is most decidedly > > > *not* > > > > > a reason > > > > > > > against using systemd. > > > > > > > > > > > > > > Personally, I'm probably going to uninstall syslog-ng, becaus= e > > > > > journalctl is > > > > > > > *such* a nice way to read logs, so why run something whose output > > > I'll > > > > > never > > > > > > > read again? I recommend reading > > > > > > > http://0pointer.net/blog/projects/journalctl.html for examples of > > > the > > > > > kind of > > > > > > > stuff you can do that would be cumbersome, if not *impossible* with > > > > > regular > > > > > > > syslog. > > > > > > > > > > > > Except that I get lots of messages about the system journal missing > > > > > > messages when forwarding to syslog, so how can I make sure this does > > > not > > > > > > happening? > > > > > > > > > > Could you please show those messages? systemd sends *everything* to the > > > > > journal, and then the journal (optionally) can send it too to a regular > > > > > syslog. In that sense, it's impossible for the journal to miss an= y > > > message. > > > > > > > > > > The only way in which the journal could miss messages is at very early > > > boot > > > > > stages; but with a proper initramfs (like the ones generated with > > > dracut), > > > > > even those get caught. You get to put an instance of systemd and the > > > > > journal inside the initramfs, and so it's available almost from the > > > > > beginning. > > > > > > > > > > And if you use gummiboot, then you can even log from the moment the UEFI > > > > > firmware comes to life. > > > > > > > > So, I get lots of messages in my regular syslog-ng /var/log/message= s > > > > like the following: > > > > Feb 23 12:47:52 ccs.covici.com systemd-journal[715]: Forwarding to > > > > syslog missed 15 messages. > > > > > > > > So, I saw a post on Google to up the queue length, and I uped it to 200, > > > > but no joy, still get the messages like the one above. > > > > > > Are you using the unit file provided by syslog-ng (systemd-delta doesn't > > > mention syslog)? Also, is /etc/systemd/system/syslog.service is a lin= k > > > to /usr/lib/systemd/system/syslog-ng.service? > > > > > > I do, and I don't get any of those messages. I use the default journa= l > > > configuration. According to [1], this should be fixed. > > > > I remember getting a small number of messages like that, too, on my laptop. > > However, it's at the university, so I can't check now to see what types of > > messages were missed (if any; if I understand [1] correctly, those messages are > > most likely bogus?). > > > > But yeah, that's any idea, Covici: see what's in /var/log/messages, compare that > > to the journalctl output, and check if any messages were actually missed ("diff > > -U" might be of help here). And if/once you did that, what kinds of messages > > were missed, if any? If those messages really are bogus, you shouldn't see any > > differences between the two. > > > > > Regards. > > > > > > https://github.com/balabit/syslog-ng/issues/314 > > > > Note that that fix would only be in the ~arch version of syslog-ng, the current > > stable version (3.4.8) is a few months too old. > > I am up to 3.6 something, so the fix should be there. But my unit file > is different, so that remains to check. I would try the provided unit file. It seems that the only difference with yours is that it doesn't comment the Restart=3Don-failure line, and that it has StandardOutput=3Dnull. I think the general idea is always to use upstream's unit files. They write the software, supposedly they should know better. Regards. -- Canek Pel=C3=A1ez Vald=C3=A9s Profesor de asignatura, Facultad de Ciencias Universidad Nacional Aut=C3=B3noma de M=C3=A9xico --20cf3020769ab34025050fc683fd Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable
On Mon, Feb 23, 2015 at 1:31 PM, <covici@ccs.covici.com> wrote:
>
> Ma= rc Joliet <marcec@gmx.de> wrote:=
>
> > Am Mon, 23 Feb 2015 12:10:18 -0600
> > schri= eb Canek Pel=C3=A1ez Vald=C3=A9s <ca= neko@gmail.com>:
> >
> > > On Mon, Feb 23, 2015= at 11:49 AM, <covici@ccs.covic= i.com> wrote:
> > > >
> > > > Canek Pe= l=C3=A1ez Vald=C3=A9s <caneko@gmail.= com> wrote:
> > > >
> > > > > On Mo= n, Feb 23, 2015 at 3:41 AM, <co= vici@ccs.covici.com> wrote:
> > > > > >
>= > > > > > Marc Joliet <= marcec@gmx.de> wrote:
> > > > > >
> > = > > > > > Am Mon, 23 Feb 2015 00:41:50 +0100
> > &g= t; > > > > schrieb lee <l= ee@yagibdah.de>:
> > > > > > >
> > = > > > > > > Neil Bothwick <neil@digimed.co.uk> writes:
> > > > > &= gt; > >
> > > > > > > > > On Wed, 18 Fe= b 2015 21:49:54 +0100, lee wrote:
> > > > > > > >= ; >
> > > > > > > > >> > I wonder if= the OP is using systemd and trying to read the
> > > > >= journal
> > > > > > > > >> > files?> > > > > > > > >>
> > > > &g= t; > > > >> Nooo, I hate systemd ...
> > > > = > > > > >>
> > > > > > > > >= ;> What good are log files you can't read?
> > > > &g= t; > > > >
> > > > > > > > > You = can't read syslog-ng log files without some reading
> > > s= oftware,
> > > > > usually
> > > > > &g= t; > > > a combination of cat, grep and less. systemd does it all = with
> > > > > journalctl.
> > > > > &g= t; > > >
> > > > > > > > > There are= good reasons to not use systemd, this isn't one of
> > > t= hem.
> > > > > > > >
> > > > >= > > > To me it is one of the good reasons, and an important one.= =C2=A0 Plain
> > > text
> > > > > > > &= gt; can usually always be read without further ado, be it from rescue
&g= t; > > > > > > > systems you booted or with software a= vailable on different
> > > operating
> > > > &g= t; > > > systems.=C2=A0 It can be also be processed with scripts a= nd sent as
> > > email.
> > > > > > > &= gt; You can probably even read it on your cell phone.=C2=A0 You can still> > > read
> > > > > > > > log files = that were created 20 years ago when they are plain text.
> > > = > > > > >
> > > > > > > > Can you= do all that with the binary files created by systemd? =C2=A0I
> >= > can't
> > > > > > > > even read them o= n a working system.
> > > > > > >
> > >= > > > > What Canek and Rich already said is good, but I'll= just add this:
> > > it's
> > > > > not = like
> > > > > > > you can't run a classic sysl= og implementation alongside the systemd
> > > > > journal= .
> > > > > > > On my systems, by *default*, syslog= -ng kept working as usual,
> > > getting
> > > >= > the logs
> > > > > > > from the systemd journ= al.=C2=A0 If you want to go further, you can even
> > > > &g= t; configure
> > > > > > > the journal to not store= logs permanently, so that you *only* end up
> > > > > wi= th
> > > > > > > plain-text logs on your system (Du= ncan on gentoo-amd64 went this
> > > way).
> > > &g= t; > > >
> > > > > > > So no, the format t= hat the systemd journal uses is most decidedly
> > > *not*
&= gt; > > > > a reason
> > > > > > > agai= nst using systemd.
> > > > > > >
> > > = > > > > Personally, I'm probably going to uninstall syslog-= ng, because
> > > > > journalctl is
> > > >= ; > > > *such* a nice way to read logs, so why run something whose= output
> > > I'll
> > > > > never
>= ; > > > > > > read again?=C2=A0 I recommend reading
&g= t; > > > > > > http://0pointer.net/blog/projects/journalctl.html f= or examples of
> > > the
> > > > > kind of> > > > > > > stuff you can do that would be cumberso= me, if not *impossible* with
> > > > > regular
> &g= t; > > > > > syslog.
> > > > > >
>= ; > > > > > Except that I get lots of messages about the sys= tem journal missing
> > > > > > messages when forwardi= ng to syslog, so how can I make sure this does
> > > not
>= ; > > > > > happening?
> > > > >
> &= gt; > > > Could you please show those messages? systemd sends *eve= rything* to the
> > > > > journal, and then the journal (= optionally) can send it too to a regular
> > > > > syslog= . In that sense, it's impossible for the journal to miss any
> &g= t; > message.
> > > > >
> > > > > Th= e only way in which the journal could miss messages is at very early
>= ; > > boot
> > > > > stages; but with a proper init= ramfs (like the ones generated with
> > > dracut),
> >= > > > even those get caught. You get to put an instance of system= d and the
> > > > > journal inside the initramfs, and so = it's available almost from the
> > > > > beginning.> > > > >
> > > > > And if you use gummi= boot, then you can even log from the moment the UEFI
> > > >= > firmware comes to life.
> > > >
> > > >= So, I get lots of messages in my regular syslog-ng /var/log/messages
&g= t; > > > like the following:
> > > > Feb 23 12:47:5= 2 ccs.covici.com systemd-journal[715]= : Forwarding to
> > > > syslog missed 15 messages.
> &= gt; > >
> > > > So, I saw a post on Google to up the q= ueue length, and I uped it to 200,
> > > > but no joy, still= get the messages like the one above.
> > >
> > > A= re you using the unit file provided by syslog-ng (systemd-delta doesn't=
> > > mention syslog)? Also, is /etc/systemd/system/syslog.ser= vice is a link
> > > to /usr/lib/systemd/system/syslog-ng.servi= ce?
> > >
> > > I do, and I don't get any of th= ose messages. I use the default journal
> > > configuration. Ac= cording to [1], this should be fixed.
> >
> > I remember = getting a small number of messages like that, too, on my laptop.
> &g= t; However, it's at the university, so I can't check now to see wha= t types of
> > messages were missed (if any; if I understand [1] c= orrectly, those messages are
> > most likely bogus?).
> >=
> > But yeah, that's any idea, Covici: see what's in /var= /log/messages, compare that
> > to the journalctl output, and chec= k if any messages were actually missed ("diff
> > -U" mi= ght be of help here).=C2=A0 And if/once you did that, what kinds of message= s
> > were missed, if any?=C2=A0 If those messages really are bogu= s, you shouldn't see any
> > differences between the two.
&= gt; >
> > > Regards.
> > >
> > > https://github.com= /balabit/syslog-ng/issues/314
> >
> > Note that that = fix would only be in the ~arch version of syslog-ng, the current
> &g= t; stable version (3.4.8) is a few months too old.
>
> I am up = to 3.6 something, so the fix should be there.=C2=A0 But my unit file
>= ; is different, so that remains to check.

I would try the provided u= nit file. It seems that the only difference with yours is that it doesn'= ;t comment the=C2=A0Restart=3Don-failure line, and that it has=C2=A0Standar= dOutput=3Dnull.

I think the general idea is always to us= e upstream's unit files. They write the software, supposedly they shoul= d know better.

Regards.
--
Canek Pel=C3=A1ez= Vald=C3=A9s
Profesor de asignatura, Facultad de Ciencias
Universidad= Nacional Aut=C3=B3noma de M=C3=A9xico

--20cf3020769ab34025050fc683fd--