[gentoo-user] I don't seem to have a system log. Help, please!

[gentoo-user] I don't seem to have a system log. Help, please!

[Alan Mackenzie]

Hello, Gentoo!

I've pretty much got my new system up and running.  It took me less than
a week (compared with the month it took me when I first installed Gentoo
a few years ago).  The most time consuming bit was getting my email
server (qmail) going.  I've still got to go through my old
/var/lib/portage/world file, and see which packages I had I still want
installed.

However, I don't seem to have a system log.  There is no file named
/var/log/syslog, or anything like it.

I've got syslog-ng installed, and "rc-update show" shows that it is
in runlevel default.  Indeed, there exists /var/run/syslog-ng.pid and
/var/run/syslog-ng.ctl.  But no /var/log/syslog, if that's what the
logfile is indeed called.  (The syslog-ng manpages don't make this
clear.)

Do I actually need to configure the name of a log file in
/etc/conf.d/syslog-ng?  The Gentoo installation guide didn't mention, or
even hint at, such being necessary.

Clearly, I'm missing something obvious here.  What is it?

Thanks in advance for the help.

-- 
Alan Mackenzie (Nuremberg, Germany).

Re: [gentoo-user] I don't seem to have a system log. Help, please!

[Matthias Hanft]

Alan Mackenzie wrote:
> 
> Do I actually need to configure the name of a log file in
> /etc/conf.d/syslog-ng?  The Gentoo installation guide didn't mention, or
> even hint at, such being necessary.

The names of the log files (and much more) are configured in
/etc/syslog-ng/syslog-ng.conf - since I have some special
configuration there, I don't know if /var/log/syslog is the
default, but /var/log/messages is a good guess, too.

And (from what I have heard) if you use systemd instead of
openrc, there are no syslog files at all - you have to export
them (from some binary database) manually to some human-
readable format. But I don't know much about that - never
used systemd on any Gentoo Linux yet.

-Matt

Re: [gentoo-user] I don't seem to have a system log. Help, please!

[Neil Bothwick]

[-- Attachment #1: Type: text/plain, Size: 1021 bytes --]

On Mon, 09 Feb 2015 11:06:42 +0100, Matthias Hanft wrote:

> > Do I actually need to configure the name of a log file in
> > /etc/conf.d/syslog-ng?  The Gentoo installation guide didn't mention,
> > or even hint at, such being necessary.  
> 
> The names of the log files (and much more) are configured in
> /etc/syslog-ng/syslog-ng.conf - since I have some special
> configuration there, I don't know if /var/log/syslog is the
> default, but /var/log/messages is a good guess, too.

The default is /var/log/messages.

> And (from what I have heard) if you use systemd instead of
> openrc, there are no syslog files at all - you have to export
> them (from some binary database) manually to some human-
> readable format. But I don't know much about that - never
> used systemd on any Gentoo Linux yet.

Or just install syslog-ng. systemd's journal doesn't preclude the use of
a traditional logger too.


-- 
Neil Bothwick

WinErr 003: Dynamic linking error - Your mistake is now in every file

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 181 bytes --]

Re: [gentoo-user] I don't seem to have a system log. Help, please!

[Rich Freeman]

On Mon, Feb 9, 2015 at 5:06 AM, Matthias Hanft <mh@hanft.de> wrote:
>
> And (from what I have heard) if you use systemd instead of
> openrc, there are no syslog files at all - you have to export
> them (from some binary database) manually to some human-
> readable format. But I don't know much about that - never
> used systemd on any Gentoo Linux yet.

You don't have to export them from anything unless you need their
content in a text file.  If you just run "journalctl" that is the
equivalent of typing cat /var/log/messages.  If you do want to parse
them with an external tool then you get your choice of several text
formats and json.

And yes, you can also run syslog, though I never really got the point
of that.  The value of the journal is that you capture full metadata
for your log entries and you can just query it vs having to parse
undelimited text files.  Heck, it seems like half the enterprise
monitoring tools start out by grabbing that log file that has
discarded most of the context and then putting it in a database and
attempting to re-create it all.

-- 
Rich

Re: [gentoo-user] I don't seem to have a system log. Help, please!

[Mick]

[-- Attachment #1: Type: Text/Plain, Size: 586 bytes --]

On Monday 09 Feb 2015 11:23:15 Rich Freeman wrote:
> You don't have to export them from anything unless you need their
> content in a text file.  If you just run "journalctl" that is the
> equivalent of typing cat /var/log/messages.  If you do want to parse
> them with an external tool then you get your choice of several text
> formats and json.

The thing is I never use cat.  I invariably use less, rview, or grep, to 
browse or search the log files.

How will this work with journalctl, will I have to export them first into a 
different format?

-- 
Regards,
Mick

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 473 bytes --]

Re: [gentoo-user] I don't seem to have a system log. Help, please!

[Alec Ten Harmsel]

On 02/09/2015 06:49 AM, Mick wrote:
> On Monday 09 Feb 2015 11:23:15 Rich Freeman wrote:
>> You don't have to export them from anything unless you need their
>> content in a text file.  If you just run "journalctl" that is the
>> equivalent of typing cat /var/log/messages.  If you do want to parse
>> them with an external tool then you get your choice of several text
>> formats and json.
> The thing is I never use cat.  I invariably use less, rview, or grep, to 
> browse or search the log files.
>
> How will this work with journalctl, will I have to export them first into a 
> different format?
>

You can run `journalctl | grep whatever`. I don't know what rview is,
but as long as whatever you're using supports pipes you should be fine.

Re: [gentoo-user] I don't seem to have a system log. Help, please!

[Rich Freeman]

On Mon, Feb 9, 2015 at 6:52 AM, Alec Ten Harmsel
<alec@alectenharmsel.com> wrote:
>
> On 02/09/2015 06:49 AM, Mick wrote:
>> On Monday 09 Feb 2015 11:23:15 Rich Freeman wrote:
>>> You don't have to export them from anything unless you need their
>>> content in a text file.  If you just run "journalctl" that is the
>>> equivalent of typing cat /var/log/messages.  If you do want to parse
>>> them with an external tool then you get your choice of several text
>>> formats and json.
>> The thing is I never use cat.  I invariably use less, rview, or grep, to
>> browse or search the log files.
>>
>> How will this work with journalctl, will I have to export them first into a
>> different format?
>>
>
> You can run `journalctl | grep whatever`. I don't know what rview is,
> but as long as whatever you're using supports pipes you should be fine.
>

Keep in mind that if you're grepping logs, there is probably a better
way to accomplish what you want to do with journalctl's options.
Finding all output from a particular daemon is going to be more
reliable if you filter by unit, versus getting verbose log output from
your mail server that has "mysql" somewhere in it or whatever.  That
is the main reason for using a binary log format.

But, yes, you can just pipe the output into the tool of your choice.
If you keep a lot of logs like I do it might be wiser to prefilter it
a bit, such as by adding -b to the options to limit it to entries
since the last reboot.

I also tend to keep a journalctl -f running in a screen session, which
is the equivalent of a tail -f.

If you're using an automated tool you can also use cursors to bookmark
the last entry you read and then ask journalctl for entries since that
one.  Of course, an automated tool would probably just read the logs
via dbus or whatever (I haven't taken the time to look into the APIs).

-- 
Rich

Re: [gentoo-user] I don't seem to have a system log. Help, please!

[Alec Ten Harmsel]

On 02/09/2015 08:02 AM, Rich Freeman wrote:
> Keep in mind that if you're grepping logs, there is probably a better
> way to accomplish what you want to do with journalctl's options.
> Finding all output from a particular daemon is going to be more
> reliable if you filter by unit, versus getting verbose log output from
> your mail server that has "mysql" somewhere in it or whatever.  That
> is the main reason for using a binary log format.

Of course, of course. I should have expanded a little more, but I was on
my way to work out. I don't use systemd at work, but for my server I
generally find the most helpful command to check a particular service is
`systemctl status <service>` to see how it got screwed up. I imagine it
would be pretty useful to grab the JSON output from every host and put
it in elastic search or mongodb or something, but I don't have any
experience doing that. At the same time, though, the message would still
have to be parsed by something and a lot of the metadata looks to be not
extremely useful (atm anyways).

> If you're using an automated tool you can also use cursors to bookmark
> the last entry you read and then ask journalctl for entries since that
> one.  Of course, an automated tool would probably just read the logs
> via dbus or whatever (I haven't taken the time to look into the APIs).
>

Using the low-level DBus C API makes me cry just a little bit; I've been
doing a ton of DBus stuff to add good systemd support to bossman. That
said, the API systemctl exposes over DBus is pretty common-sensical, so
I'm sure the journalctl one is straightforward as well.

Alec

Re: [gentoo-user] I don't seem to have a system log. Help, please!

[covici]

Rich Freeman <rich0@gentoo.org> wrote:

> On Mon, Feb 9, 2015 at 6:52 AM, Alec Ten Harmsel
> <alec@alectenharmsel.com> wrote:
> >
> > On 02/09/2015 06:49 AM, Mick wrote:
> >> On Monday 09 Feb 2015 11:23:15 Rich Freeman wrote:
> >>> You don't have to export them from anything unless you need their
> >>> content in a text file.  If you just run "journalctl" that is the
> >>> equivalent of typing cat /var/log/messages.  If you do want to parse
> >>> them with an external tool then you get your choice of several text
> >>> formats and json.
> >> The thing is I never use cat.  I invariably use less, rview, or grep, to
> >> browse or search the log files.
> >>
> >> How will this work with journalctl, will I have to export them first into a
> >> different format?
> >>
> >
> > You can run `journalctl | grep whatever`. I don't know what rview is,
> > but as long as whatever you're using supports pipes you should be fine.
> >
> 
> Keep in mind that if you're grepping logs, there is probably a better
> way to accomplish what you want to do with journalctl's options.
> Finding all output from a particular daemon is going to be more
> reliable if you filter by unit, versus getting verbose log output from
> your mail server that has "mysql" somewhere in it or whatever.  That
> is the main reason for using a binary log format.
> 
> But, yes, you can just pipe the output into the tool of your choice.
> If you keep a lot of logs like I do it might be wiser to prefilter it
> a bit, such as by adding -b to the options to limit it to entries
> since the last reboot.
> 
> I also tend to keep a journalctl -f running in a screen session, which
> is the equivalent of a tail -f.
> 
> If you're using an automated tool you can also use cursors to bookmark
> the last entry you read and then ask journalctl for entries since that
> one.  Of course, an automated tool would probably just read the logs
> via dbus or whatever (I haven't taken the time to look into the APIs).

I wonder if the original poster is using systemd?  Also, I find
journalctl very clumsy to find things about a specific program, such as
mail logs or whatever -- unless I am missing something.  I use
syslog-ng, although I get a lot of messages which say forwarding to
syslog missed n messages from system journal, so maybe its a problem,
but how would you use logwatch without something like syslog-ng?
-- 
Your life is like a penny.  You're going to lose it.  The question is:
How do
you spend it?

         John Covici
         covici@ccs.covici.com

Re: [gentoo-user] I don't seem to have a system log. Help, please!

[Rich Freeman]

On Mon, Feb 9, 2015 at 9:26 AM,  <covici@ccs.covici.com> wrote:
>
> I wonder if the original poster is using systemd?

He already said he isn't.  He just was looking for the wrong filename.

> Also, I find journalctl very clumsy to find things about a specific
> program, such as mail logs or whatever -- unless I am missing
> something.

Well, the journal only contains stuff sent to it.  So, if apache dumps
some stuff to stdout or to /dev/log or whatever then it will be in the
journal.  If apache dumps its logs directly to a file in
/var/log/apache then the journal won't contain it.  Many files in
/var/log were not created by syslog-ng, and this would not show up in
the journal.

>  I use syslog-ng, although I get a lot of messages which say
> forwarding to syslog missed n messages from system journal, so maybe
> its a problem, but how would you use logwatch without something like
> syslog-ng?

You'd need to use a systemd-aware log watcher.

-- 
Rich

Re: [gentoo-user] I don't seem to have a system log. Help, please!

[covici]

Rich Freeman <rich0@gentoo.org> wrote:

> On Mon, Feb 9, 2015 at 9:26 AM,  <covici@ccs.covici.com> wrote:
> >
> > I wonder if the original poster is using systemd?
> 
> He already said he isn't.  He just was looking for the wrong filename.
> 
> > Also, I find journalctl very clumsy to find things about a specific
> > program, such as mail logs or whatever -- unless I am missing
> > something.
> 
> Well, the journal only contains stuff sent to it.  So, if apache dumps
> some stuff to stdout or to /dev/log or whatever then it will be in the
> journal.  If apache dumps its logs directly to a file in
> /var/log/apache then the journal won't contain it.  Many files in
> /var/log were not created by syslog-ng, and this would not show up in
> the journal.
> 
> >  I use syslog-ng, although I get a lot of messages which say
> > forwarding to syslog missed n messages from system journal, so maybe
> > its a problem, but how would you use logwatch without something like
> > syslog-ng?
> 
> You'd need to use a systemd-aware log watcher.

Is there such a thing?

-- 
Your life is like a penny.  You're going to lose it.  The question is:
How do
you spend it?

         John Covici
         covici@ccs.covici.com

[gentoo-user]

[alexandra vargas]

[-- Attachment #1: Type: text/plain, Size: 15 bytes --]

 
 		 	   		  

[-- Attachment #2: Type: text/html, Size: 239 bytes --]

Re: [gentoo-user] I don't seem to have a system log. Help, please!

[Alan Mackenzie]

Hello, Matthias.

On Mon, Feb 09, 2015 at 11:06:42AM +0100, Matthias Hanft wrote:
> Alan Mackenzie wrote:

> > Do I actually need to configure the name of a log file in
> > /etc/conf.d/syslog-ng?  The Gentoo installation guide didn't mention, or
> > even hint at, such being necessary.

> The names of the log files (and much more) are configured in
> /etc/syslog-ng/syslog-ng.conf - since I have some special
> configuration there, I don't know if /var/log/syslog is the
> default, but /var/log/messages is a good guess, too.

Yes, I've got a /var/log/messages.  I've even looked at it many times in
the past.  But I didn't know that it was THE system log.  Thanks!

> And (from what I have heard) if you use systemd instead of
> openrc, there are no syslog files at all - you have to export
> them (from some binary database) manually to some human-
> readable format. But I don't know much about that - never
> used systemd on any Gentoo Linux yet.

No, I've never used systemd either.  It's useful to be able to read
/var/log/messages with less, probe it with grep/awk/perl, etc., without
having to learn some special purpose script language.

> -Matt

-- 
Alan Mackenzie (Nuremberg, Germany).

Re: [gentoo-user] I don't seem to have a system log. Help, please!

[Neil Bothwick]

[-- Attachment #1: Type: text/plain, Size: 675 bytes --]

On Mon, 9 Feb 2015 11:23:52 +0000, Alan Mackenzie wrote:

> No, I've never used systemd either.  It's useful to be able to read
> /var/log/messages with less, probe it with grep/awk/perl, etc., without
> having to learn some special purpose script language.

journalctl outputs to less (or whatever $PAGER contains) by default or
you can pipe its output to anything else. Running journalctl with no
arguments is the equivalent of cat /var/log/messages.

I do have syslog installed, but that's a holdover from when I was
experimenting with systemd, I really could get rid of it now.


-- 
Neil Bothwick

How do you know when it's time to tune your bagpipes?

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 181 bytes --]

Re: [gentoo-user] I don't seem to have a system log. Help, please!

[Alan McKinnon]

On 09/02/2015 11:48, Alan Mackenzie wrote:
> Hello, Gentoo!
> 
> I've pretty much got my new system up and running.  It took me less than
> a week (compared with the month it took me when I first installed Gentoo
> a few years ago).  The most time consuming bit was getting my email
> server (qmail) going.  I've still got to go through my old
> /var/lib/portage/world file, and see which packages I had I still want
> installed.
> 
> However, I don't seem to have a system log.  There is no file named
> /var/log/syslog, or anything like it.
> 
> I've got syslog-ng installed, and "rc-update show" shows that it is
> in runlevel default.  Indeed, there exists /var/run/syslog-ng.pid and
> /var/run/syslog-ng.ctl.  But no /var/log/syslog, if that's what the
> logfile is indeed called.  (The syslog-ng manpages don't make this
> clear.)
> 
> Do I actually need to configure the name of a log file in
> /etc/conf.d/syslog-ng?  The Gentoo installation guide didn't mention, or
> even hint at, such being necessary.
> 
> Clearly, I'm missing something obvious here.  What is it?
> 
> Thanks in advance for the help.
> 


Gentoo defaults to calling it /var/log/messages

(it's also constantly tailed on vt12, just in case you need to see
what's going on it right now)



-- 
Alan McKinnon
alan.mckinnon@gmail.com

Re: [gentoo-user] I don't seem to have a system log. Help, please!

[Alan Mackenzie]

Hi, Alan.

On Mon, Feb 09, 2015 at 12:19:20PM +0200, Alan McKinnon wrote:
> On 09/02/2015 11:48, Alan Mackenzie wrote:
> > Hello, Gentoo!

> > I've pretty much got my new system up and running.  It took me less than
> > a week (compared with the month it took me when I first installed Gentoo
> > a few years ago).  The most time consuming bit was getting my email
> > server (qmail) going.  I've still got to go through my old
> > /var/lib/portage/world file, and see which packages I had I still want
> > installed.

> > However, I don't seem to have a system log.  There is no file named
> > /var/log/syslog, or anything like it.

> > I've got syslog-ng installed, and "rc-update show" shows that it is
> > in runlevel default.  Indeed, there exists /var/run/syslog-ng.pid and
> > /var/run/syslog-ng.ctl.  But no /var/log/syslog, if that's what the
> > logfile is indeed called.  (The syslog-ng manpages don't make this
> > clear.)

> > Do I actually need to configure the name of a log file in
> > /etc/conf.d/syslog-ng?  The Gentoo installation guide didn't mention, or
> > even hint at, such being necessary.

> > Clearly, I'm missing something obvious here.  What is it?

> > Thanks in advance for the help.



> Gentoo defaults to calling it /var/log/messages

Yes.  :-)

> (it's also constantly tailed on vt12, just in case you need to see
> what's going on it right now)

I didn't know that.  Wow!  Is this something relatively new, or has it
always been there?

> -- 
> Alan McKinnon
> alan.mckinnon@gmail.com

-- 
Alan Mackenzie (Nuremberg, Germany).

Re: [gentoo-user] I don't seem to have a system log. Help, please!

[Matthias Hanft]

Alan Mackenzie wrote:
> 
> On Mon, Feb 09, 2015 at 12:19:20PM +0200, Alan McKinnon wrote:
>> (it's also constantly tailed on vt12, just in case you need to see
>> what's going on it right now)
> 
> I didn't know that.  Wow!  Is this something relatively new, or has it
> always been there?

I installed my first Gentoo server (which is still operating)
on 4/13/2006, and I'm pretty sure I could press ALT+F12 at the
console for reading syslog (=messages) even then.

-Matt

Re: [gentoo-user] I don't seem to have a system log. Help, please!

[Mick]

[-- Attachment #1: Type: Text/Plain, Size: 2106 bytes --]

On Monday 09 Feb 2015 10:19:20 Alan McKinnon wrote:
> On 09/02/2015 11:48, Alan Mackenzie wrote:
> > Hello, Gentoo!
> > 
> > I've pretty much got my new system up and running.  It took me less than
> > a week (compared with the month it took me when I first installed Gentoo
> > a few years ago).  The most time consuming bit was getting my email
> > server (qmail) going.  I've still got to go through my old
> > /var/lib/portage/world file, and see which packages I had I still want
> > installed.
> > 
> > However, I don't seem to have a system log.  There is no file named
> > /var/log/syslog, or anything like it.
> > 
> > I've got syslog-ng installed, and "rc-update show" shows that it is
> > in runlevel default.  Indeed, there exists /var/run/syslog-ng.pid and
> > /var/run/syslog-ng.ctl.  But no /var/log/syslog, if that's what the
> > logfile is indeed called.  (The syslog-ng manpages don't make this
> > clear.)
> > 
> > Do I actually need to configure the name of a log file in
> > /etc/conf.d/syslog-ng?  The Gentoo installation guide didn't mention, or
> > even hint at, such being necessary.
> > 
> > Clearly, I'm missing something obvious here.  What is it?
> > 
> > Thanks in advance for the help.
> 
> Gentoo defaults to calling it /var/log/messages
> 
> (it's also constantly tailed on vt12, just in case you need to see
> what's going on it right now)

I noticed the same on a recent installation.  /var/log/syslog is not created 
by default any more, when installing syslog-ng.  I haven't looked in the 
/etc/syslog-ng/syslog-ng.conf file of the new install to see what's different, 
but it used to be that something like this would do the trick:
=============================================================
destination d_syslog { file("/var/log/syslog"); };

filter f_syslog { not facility(authpriv, mail); }

log { source(src); filter(f_syslog); destination(d_syslog); };
=============================================================

I am not sure if the format has changed since the last time I looked at it.

-- 
Regards,
Mick

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 473 bytes --]

Re: [gentoo-user] I don't seem to have a system log. Help, please!

[Neil Bothwick]

[-- Attachment #1: Type: text/plain, Size: 428 bytes --]

On Mon, 9 Feb 2015 11:29:24 +0000, Mick wrote:

> I noticed the same on a recent installation.  /var/log/syslog is not
> created by default any more, when installing syslog-ng. 

I've using syslog-ng on Gentoo for well over ten years and it's always
defaulted to /var/log/messages in that time. Other loggers may use a
different file by default.


-- 
Neil Bothwick

0 and 1. Now what could be so hard about that?

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 181 bytes --]

Re: [gentoo-user] I don't seem to have a system log. Help, please!

[Adam Carter]

[-- Attachment #1: Type: text/plain, Size: 253 bytes --]

One little corner case; if you're running systemd 216 and syslog-ng <3.6,
you need to add ForwardToSyslog=yes to /etc/systemd/journald.conf. With
systemd 215 and earlier, messages are forwarded to syslog by default, and
syslog-ng 3.6 is journald aware.

[-- Attachment #2: Type: text/html, Size: 286 bytes --]