[nylug-workshop] Log analyzer (Re: Regular meetings of the Python workshop)

[Peter C. Norton]

On Tue, Apr 10, 2007 at 03:37:50AM -0400, Yusuke Shinyama wrote:
> Hi,
> 
> I put up a newer version of logweeder.
> http://www.unixuser.org/~euske/python/logweeder/
> 
> This version is significantly improved in its speed.
> Now it can process thousands of lines per second, and
> it can also load a previously constructed pattern file for further speedup.
> 
> The major drawback is that since the algorithm is now a greedy
> way, it might not return optimal results.  For example, if the log
> file is like as follows, line 1 and line 100000 are recognized as
> different clusters:
> 
>   line 1: ypbind[1234]: aaaa
>   line 2: ypbind[1234]: bbbb
> 
>     (cluster for "ypbind[1234]: ..." is formed)
> 
>     (then ypbind restarted for some reason)
> 
>   line 100000: ypbind[7890]: aaaa ...
>   line 100001: ypbind[7890]: bbbb ...
> 
>     (another cluster for "ypbind[7890]: ..." is formed)
> 
> I haven't yet incorporated Peter's patch because the program
> structure is rather different, althoug I mostly incorporated the
> general idea. (and personally I'm not fond of using configParser.)

Oh, well, I'm not appreciated, fine :)

Seriously, It seems like the problem with this is that there's no
extra weight being given to the logging tag, which could probably be
fixed, I think.

-Peter

-- 
The 5 year plan:
In five years we'll make up another plan.
Or just re-use this one.