[gentoo-mirrors] Release: Carl v0.5 (RSync logfile analyzer)

[gentoo-mirrors] Release: Carl v0.5 (RSync logfile analyzer)

[Tobias Klausmann]

Hi!

I'm pleased to announce version 0.5 of my RSync logfile parsing
utility called Carl. As usual, it's available here:

http://www.schwarzvogel.de/software-misc.shtml

And here's the direct download for you impatient folk:

http://www.schwarzvogel.de/pkgs/carl-0.5.tar.gz

Here are the MD5- and SHA1-Sums:

d9d5f6095e10d3ee8d5527f5febcbd2b1bff05b6  carl-0.5.tar.gz
877af0f3da3782fc01c01d1e505cc52f  carl-0.5.tar.gz

Version 0.5 brings a more consistent output layout, better option 
parsing and IP obfuscation support. Also, I ask you to thoroughly
test it, bend it, break it (send me copies of the pieces if you like).

Carl is licensed under the GPL-2. Feel free to submit suggestions
or even patches to me. 

Here's an example report (with fancy obfuscation turned on):

Carl (Carl Analyzes Rsync Logfiles) 0.4
(C) Tobias Klausmann
Psyco found and enabled.
Reading from 'december.log'
Total traffic: 232.20 GBytes
Total number of sessions: 46702
Total number of unique IPs: 11755
Log seems to span 31.00 days.

 Top 10 Hosts by byte count
     bytes     ( Bytes )     IP-Address
-----------------------------------------
   508298839 (   484.75M) 9713287ec06f49e47294a3d4f8e5445a
   516272928 (   492.36M) e562b485ff3ba993ac047114f8982ae1
   555298985 (   529.57M) 431222bfb8e51a6944142a8e5c442ab1
   641000515 (   611.31M) 5efc541c1f8f19e5bc49a7c1ed86913c
   781678951 (   745.47M) d64583071d308ebc0ed1169e1871ed27
   806646848 (   769.28M) a5ad98c2dff07bdb74812d7e0fe289d6
  1960339410 (     1.83G) e9de0ddcdb2331f556987e49fc903b4c
  2187803840 (     2.04G) 7c0a7271ec8e8bc5db4dc080f9862a5f
  5543744306 (     5.16G) 1227425e97fa2c40274a7f9064b24481
 36163830527 (    33.68G) 2d468d4a6967209792b37b58105e90d1
-----------------------------------------
Average traffic per day: 8042902235.13 bytes (7.49GB)
Top 5% of IPs (587) account for 122048652073 bytes (113.67GB) of traffic,
which is 48.95% of the total traffic.

 Top 10 Hosts by session count
Sessions   per day    IP-Address
----------------------------------
   139      4.48   0b7e18a5f1319b28994d3c3f6b1c0a14
   145      4.68   431222bfb8e51a6944142a8e5c442ab1
   153      4.94   1455996155afd674b51c17baf8a486ab
   164      5.29   5efc541c1f8f19e5bc49a7c1ed86913c
   189      6.10   1855a5c4c1badb3c144f51820c56e23f
   210      6.77   a5ad98c2dff07bdb74812d7e0fe289d6
   468     15.10   e9de0ddcdb2331f556987e49fc903b4c
   862     27.81   7c0a7271ec8e8bc5db4dc080f9862a5f
  1477     47.65   1227425e97fa2c40274a7f9064b24481
  9469    305.46   2d468d4a6967209792b37b58105e90d1
----------------------------------
Average number of sessions per day: 1506.54
Top 5% of IPs (621) account for 28391 sessions.
which is 60.79% of the total number of sessions.

Analyzed 386253 lines in 2.72 seconds, 141923.07 lines per second

Regards,
Tobias

-- 
Never touch a burning system.
-- 
gentoo-mirrors@gentoo.org mailing list

Re: [gentoo-mirrors] Release: Carl v0.5 (RSync logfile analyzer)

[Tobias Klausmann]

[-- Attachment #1: Type: text/plain, Size: 1356 bytes --]

Hi! 

I'm pleased to announce version 0.7 of my RSync logfile parsing
utility called Carl. As usual, it's available here:

http://www.schwarzvogel.de/software-misc.shtml

And here's the direct download for you impatient folk:

http://www.schwarzvogel.de/pkgs/carl-0.7.tar.gz

Here are the MD5- and SHA1-Sums:

2e61fdecea7427aa0c888201c983dae9  carl-0.7.tar.gz
0e9beea03a5ccd1bc4576a3894b010cfa6b11dc0  carl-0.7.tar.gz

This version has two new options:

  -r, --reverse         set reverse (classic) display order
  -s, --short           display only the two top-10 lists

-r restores the old output order which has the address with the
largest number of sessions/traffic at the bottom - those are now
at the top by default.

-s is the short mode which results in *only* the top10-lists and
their captions being printed.

Also, the lists now have a rank displayed in their first column.
Additionally, the hostname as reported by rsync is included if
available. Note that this is the reverse lookup at that point in
time, which means it a) is easily forged and b) may have changed.
If you contact an organization about abusive RSync behaviour, be
sure to contact the IP owner as reported by the whois tool.

Regards,
Tobias

PS: Thanks goes out to Peltzi for all the suggestions.

-- 
Never touch a burning system.

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]