[gentoo-user] Re: syslog-ng: v2->v3 config issue...

[gentoo-user] Re: syslog-ng: v2->v3 config issue...

[Fekete Robert]

Hi Jarry,

I work for BalaBit, the developer of syslog-ng, and am the maintainer
of the syslog-ng docs.
You are right, the program-override option is missing from the
documentation of the file source, but it should work anyway.
We did a quick test and it was working on our Ubuntu machines (tested
with syslog-ng 3.02a), both on kernel messages and also on custom
files containing log messages.
Which version of syslog-ng are you running? Are the messages in the file in 
correct syslog format, or do they have some custom format?

If the problem persists, could you open a ticket in the syslog-ng bugzilla at 
https://bugzilla.balabit.com/?

Regards,

Robert Fekete


Hi,
as syslog-ng 3.0.x became stable, all my servers updated
to it from 2.1.4, but I have a problem with configuration:

In 2.x I used "log_prefix()" option for "file()" source.
When I tried to start syslog-ng 3.x it complained about
"log_prefix()" being deprecated, and said I have to use
"program_override()" instead.

I modified syslog-ng.conf, but it does not work at all.
It simply acts as if there was no "program_override()"
option in "file()" source.

I checked syslog-ng-v3.0-guide-admin-en.pdf and found this:
"log_prefix()" really *is* deprecated, but it seems to me
that "program_override()" was not implemented in "file()"
source driver at all! At least, I did not find it as valid
option for "file()" source driver in the chapter 8 Reference
(in syslog-ng admin guide)...

How can I fix this? I definitelly need that "log_prefix()"
(or "program_override()") option as I use it later for
filtering of non-standard log messages on my log-server...

Jarry

Re: [gentoo-user] Re: syslog-ng: v2->v3 config issue...

[Jarry]

Fekete Robert wrote:
> You are right, the program-override option is missing from the
> documentation of the file source, but it should work anyway.
> We did a quick test and it was working on our Ubuntu machines (tested
> with syslog-ng 3.02a), both on kernel messages and also on custom
> files containing log messages.

Well, I'm not sure where is the problem. I'm using syslog-ng-3.0.4
(the last stable version in portage). This is relevant part of my
"new" /etc/syslog-ng.conf:
====================
options { chain_hostnames(no);
           stats_freq(3600);
           ts_format(iso);
           flush_lines(1);
           log_fifo_size(250); };

source s_teamspeak { file("/var/log/teamspeak2-server/server.log"
                         flags(store-legacy-msghdr)
                         program_override("teamspeak: ")
                         log_fetch_limit(100)
                         flags(no-parse)); };

destination d_teamspeak { file("/var/log/ts2.log"); };
log { source(s_teamspeak); destination(d_teamspeak); };
==========================

One line in source (/var/log/teamspeak-server/server.log):
04-11-09 16:52:54,ALL,Info... (etc)

Corresponding line in /var/log/ts2.log (that program_override()
is simply missing):
2009-11-04T16:52:54+00:00 talk 04-11-09 16:52:54,ALL,Info...

For comparison, the same part of my syslog-ng v2.x config:
==========================
options { chain_hostnames(off);
           sync(0);
           stats(43200);
           ts_format(iso); };

source s_teamspeak2 { file("/var/log/teamspeak2-server/server.log"
                         log_prefix("teamspeak2: ")
                         follow_freq(1)
                         flags(no-parse)); };

destination d_teamspeak { file("/var/log/ts2.log"); };
log { source(s_teamspeak); destination(d_teamspeak); };
===========================

And this is what I got in ts2.log with syslog-ng v2.x:

2009-09-25T18:17:41+00:00 talk teamspeak2: 28-07-09 18:49:39,ALL,Info...

You see the difference?
syslog-ng 2.x: "iso-time hostname *log_prefix* message"
syslog-ng 3.x: "iso-time hostname message"
Where is program_override?

v2/v3 config-files are now not absolutely the same but even when
I made them identical (removed fifo_size, fetch_limit, flags, etc)
I still had this problem. And I observed this strange behavior
not only with this particular file() source, but with all file()
sources. So what could be the reason?

Jarry

-- 
_______________________________________________________________
This mailbox accepts e-mails only from selected mailing-lists!
Everything else is considered to be spam and therefore deleted.

Re: [gentoo-user] Re: syslog-ng: v2->v3 config issue...

[frobert]

Hi Jarry,
thanks for the detailed info. I have discussed the issue with my  
colleagues, and it seems that the error is on our side: there was a  
performance-related change in the program-override option in 3.0.4,  
which broke the function.

So you can either downgrade to an older version (3.0.3 should work),  
or if you want to stick to 3.0.4, you can try to add a rewrite rule to  
set the PROGRAM field to teamspeak (which may or may not work in this  
case, since the program field seems to be empty in the message -  
sorry, I haven't had the time to test it).

Alternatively, you can create a template for this destination and  
rebuild the message from macros and add a default value for program  
($ISODATE $HOST ${PROGRAM:-teamspeak2} $MESSAGE)

I hope one of these will work for you.

Regards,

Robert


Quoting Jarry <mr.jarry@gmail.com>:

> Fekete Robert wrote:
>> You are right, the program-override option is missing from the
>> documentation of the file source, but it should work anyway.
>> We did a quick test and it was working on our Ubuntu machines (tested
>> with syslog-ng 3.02a), both on kernel messages and also on custom
>> files containing log messages.
>
> Well, I'm not sure where is the problem. I'm using syslog-ng-3.0.4
> (the last stable version in portage). This is relevant part of my
> "new" /etc/syslog-ng.conf:
> ====================
> options { chain_hostnames(no);
>           stats_freq(3600);
>           ts_format(iso);
>           flush_lines(1);
>           log_fifo_size(250); };
>
> source s_teamspeak { file("/var/log/teamspeak2-server/server.log"
>                         flags(store-legacy-msghdr)
>                         program_override("teamspeak: ")
>                         log_fetch_limit(100)
>                         flags(no-parse)); };
>
> destination d_teamspeak { file("/var/log/ts2.log"); };
> log { source(s_teamspeak); destination(d_teamspeak); };
> ==========================
>
> One line in source (/var/log/teamspeak-server/server.log):
> 04-11-09 16:52:54,ALL,Info... (etc)
>
> Corresponding line in /var/log/ts2.log (that program_override()
> is simply missing):
> 2009-11-04T16:52:54+00:00 talk 04-11-09 16:52:54,ALL,Info...
>
> For comparison, the same part of my syslog-ng v2.x config:
> ==========================
> options { chain_hostnames(off);
>           sync(0);
>           stats(43200);
>           ts_format(iso); };
>
> source s_teamspeak2 { file("/var/log/teamspeak2-server/server.log"
>                         log_prefix("teamspeak2: ")
>                         follow_freq(1)
>                         flags(no-parse)); };
>
> destination d_teamspeak { file("/var/log/ts2.log"); };
> log { source(s_teamspeak); destination(d_teamspeak); };
> ===========================
>
> And this is what I got in ts2.log with syslog-ng v2.x:
>
> 2009-09-25T18:17:41+00:00 talk teamspeak2: 28-07-09 18:49:39,ALL,Info...
>
> You see the difference?
> syslog-ng 2.x: "iso-time hostname *log_prefix* message"
> syslog-ng 3.x: "iso-time hostname message"
> Where is program_override?
>
> v2/v3 config-files are now not absolutely the same but even when
> I made them identical (removed fifo_size, fetch_limit, flags, etc)
> I still had this problem. And I observed this strange behavior
> not only with this particular file() source, but with all file()
> sources. So what could be the reason?
>
> Jarry
>
> -- 
> _______________________________________________________________
> This mailbox accepts e-mails only from selected mailing-lists!
> Everything else is considered to be spam and therefore deleted.



----------------------------------------------------------------
This message was sent using IMP, the Internet Messaging Program.

Re: [gentoo-user] Re: syslog-ng: v2->v3 config issue...

[Jarry]

frobert@balabit.hu wrote:

> thanks for the detailed info. I have discussed the issue with my 
> colleagues, and it seems that the error is on our side: there was a 
> performance-related change in the program-override option in 3.0.4, 
> which broke the function.

Hi Robert, thanks for reply. I will notify gentoo syslog-ng package
maintaner and ask him to include 3.0.3 so that I could downgrade,
because right now 3.0.4 is the only 3.x in portage...

BR,
Jarry

-- 
_______________________________________________________________
This mailbox accepts e-mails only from selected mailing-lists!
Everything else is considered to be spam and therefore deleted.