From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from pigeon.gentoo.org ([208.92.234.80] helo=lists.gentoo.org) by finch.gentoo.org with esmtp (Exim 4.60) (envelope-from ) id 1NrzgR-0005Ti-HZ for garchives@archives.gentoo.org; Wed, 17 Mar 2010 20:17:54 +0000 Received: from pigeon.gentoo.org (localhost [127.0.0.1]) by pigeon.gentoo.org (Postfix) with SMTP id E26E7E0974; Wed, 17 Mar 2010 20:16:21 +0000 (UTC) Received: from mail-pw0-f53.google.com (mail-pw0-f53.google.com [209.85.160.53]) by pigeon.gentoo.org (Postfix) with ESMTP id 9A181E0974 for ; Wed, 17 Mar 2010 20:16:21 +0000 (UTC) Received: by pwj10 with SMTP id 10so1072712pwj.40 for ; Wed, 17 Mar 2010 13:16:21 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:in-reply-to:references :date:message-id:subject:from:to:content-type; bh=ppZGfZnIxdQJAfRFDicf2o5IbX9HUSOMAjOKKHQie6s=; b=Pvq+m8BMRl9F75flI9C5I+eKorb6u8Ywq5CZYx+g5ni0lHQD5UGq9BQ0BZdFcKygjo RBAD6oMftEEd07Q4qHM2OaMft2i596scY9Xo1WdA0lR+HqTx6or0stTqaHO6Om5ie8CI kBmcoX55+6WhamUVQSyU4TyYYnUL/rYBGO2xI= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; b=bZYDqt+5REagNLmVoGdsG5eHLMuaUYC7FVE/1s5qTcVxSBjW0QifKEbjYje/U1c3UL 8wOS6eIcAWwmZl6KfAUT6nXLbu8xKJTzgzWPP3N7gckIWHXL6zCMu6hrb+dflWqeHr6L HMDxzOTHs/ny9rRzUdZk4LII6fYlbYkSF0+Rs= Precedence: bulk List-Post: List-Help: List-Unsubscribe: List-Subscribe: List-Id: Gentoo Linux mail X-BeenThere: gentoo-user@lists.gentoo.org Reply-to: gentoo-user@lists.gentoo.org MIME-Version: 1.0 Received: by 10.141.13.7 with SMTP id q7mr50724rvi.257.1268856980987; Wed, 17 Mar 2010 13:16:20 -0700 (PDT) In-Reply-To: <201003171239.47431.alan.mckinnon@gmail.com> References: <17bd4e851003161622x21b7e78chc228017250c7ff0f@mail.gmail.com> <201003171239.47431.alan.mckinnon@gmail.com> Date: Thu, 18 Mar 2010 09:16:20 +1300 Message-ID: <17bd4e851003171316y6785714brd7327983d5da02cf@mail.gmail.com> Subject: Re: [gentoo-user] syslog-ng filtering From: Ralph Slooten To: gentoo-user@lists.gentoo.org Content-Type: multipart/alternative; boundary=000e0cd11ae09f5e63048204c985 X-Archives-Salt: fb933395-6790-4c6a-bd76-98901ecbdd3d X-Archives-Hash: 8b899eaaa940f16627dc178d0a08a2c3 --000e0cd11ae09f5e63048204c985 Content-Type: text/plain; charset=ISO-8859-1 Fantastic, you hit the nail right on the head! Works like a charm now. Now I'm wondering how it is you found out that it was this way and not the other? Robert maintains the documentation for rsync which I did look at, but with 225 pages I wasn't able to find this useful piece of information. Man syslog-ng.conf does not explain it either, in fact I searched Google and found several "tutorials", none mentioning this ;-) Maybe I'm the idiot here, however I thought that this was a common way of getting rid of unwanted crud from the syslog? Also, I just read the gentoo-wiki site page again and it says : filter f_shorewall { not match("regex" value("Shorewall")); }; # Filter everything except regex keyword Shorewall Surely this is the exact same mistake I made? Either that or I'm reading it wrong.... On 17 March 2010 23:39, Alan McKinnon wrote: > On Wednesday 17 March 2010 01:22:59 Ralph Slooten wrote: > > Hi all, > > > > Has anyone here worked out how to filter out syslog messages using > > syslog-ng v3? The old syntax doesn't work (well complains bitterly about > > performance and says to use regex), and no matter what I try I cannot get > > the new syntax to work :-/ I have a syslog-ng server which logs to MySQL > > for multiple clients in a network, however the database just keeps > growing > > with irrelevant data I'd prefer to just quietly ignore on the server > side. > > > > I'm trying to filter out (exclude) messages such as: > > (root) CMD (/root/bin/vmware-checker) > > and > > (root) CMD (test -x /usr/sbin/run-crons && /usr/sbin/run-crons ) > > > > ============== > > filter myfilter { > > not match("regex" value("\/usr\/sbin\/run-crons")) > > and not match("regex" value("vmware-checker")); > > } > > Hah! this caught me out too. > > The value of "value" cannot be anything arbitrary - syslog-ng has no clue > what > you mean. The value is a field name, either a pre-defined one, or something > you defined using a parser. The docs are ambiguous on this, it's not clear > that the supplied values are abstracts. You are truing to search for the > string "regex" in a field called /usr/bin/vmware-checker. > > Which obviously will not work. > > I think you want: > > match("\/usr\/sbin\/run-crons" value "MESSAGE") > > Note that it is MESSAGE. You want the field name, not it's dereferenced > value. > > > > > log { > > source(src); > > source(remote); > > filter(myfilter); > > destination(d_mysql); > > }; > > =============== > > > > However they just keep coming through the filter (ie: not matching the > "not > > match" filter). I've tried escaping the slashes, not escaping them ... > even > > partial words, but I obviously am missing something somewhere. > > > > Anyone have any ideas? > > > > Thanks in advance, > > Ralph > > -- > alan dot mckinnon at gmail dot com > --000e0cd11ae09f5e63048204c985 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Fantastic, you hit the nail right on the head! Works like a charm now.
=
Now I'm wondering how it is you found out that it was th= is way and not the other? Robert maintains the documentation for rsync whic= h I did look at, but with 225 pages I wasn't able to find this useful p= iece of information. Man syslog-ng.conf does not explain it either, in fact= I searched Google and found several "tutorials", none mentioning= this ;-)

Maybe I'm the idiot here, however I thought that th= is was a common way of getting rid of unwanted crud from the syslog?=A0

Also, I just read the gentoo-wiki site page again and= it says :

filter f_shorewall { not match("regex" value("Shorewall"=
;)); };  # Filter everything except regex keyword Shorewall
Surely this is the exact same mistake I made? Either that or I'= m reading it wrong....





On 17 March 2010 23:39, Alan McKinnon <alan.mckinnon@gmail.com> wrote:
On Wednesday 17 March 201= 0 01:22:59 Ralph Slooten wrote:
> Hi all,
>
> Has anyone here worked out how to filter out syslog messages using
> syslog-ng v3? The old syntax doesn't work (well complains bitterly= about
> performance and says to use regex), and no matter what I try I cannot = get
> the new syntax to work :-/ I have a syslog-ng server which logs to MyS= QL
> for multiple clients in a network, however the database just keeps gro= wing
> with irrelevant data I'd prefer to just quietly ignore on the serv= er side.
>
> I'm trying to filter out (exclude) messages such as:
> =A0 (root) CMD (/root/bin/vmware-checker)
> and
> =A0 (root) CMD (test -x /usr/sbin/run-crons && /usr/sbin/run-c= rons )
>
> =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
> filter myfilter {
> =A0 =A0 =A0 =A0 not match("regex" value("\/usr\/sbin\/r= un-crons"))
> =A0 =A0 =A0 =A0 and not match("regex" value("vmware-che= cker"));
> }

Hah! this caught me out too.

The value of "value" cannot be anything arbitrary - syslog-ng has= no clue what
you mean. The value is a field name, either a pre-defined one, or something=
you defined using a parser. The docs are ambiguous on this, it's not cl= ear
that the supplied values are abstracts. You are truing to search for the string "regex" in a field called /usr/bin/vmware-checker.

Which obviously will not work.

I think you want:

match("\/usr\/sbin\/run-crons" value "MESSAGE")

Note that it is MESSAGE. You want the field name, not it's dereferenced= value.



> log {
> =A0 =A0 =A0 =A0 source(src);
> =A0 =A0 =A0 =A0 source(remote);
> =A0 =A0 =A0 =A0 filter(myfilter);
> =A0 =A0 =A0 =A0 destination(d_mysql);
> };
> =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
>
> However they just keep coming through the filter (ie: not matching the= "not
> match" filter). I've tried escaping the slashes, not escaping= them ... even
> partial words, but I obviously am missing something somewhere.
>
> Anyone have any ideas?
>
> Thanks in advance,
> Ralph

--
alan dot mckinnon at gmail dot com

--000e0cd11ae09f5e63048204c985--