nixspamsum
author Pascal Volk <user@localhost.localdomain.org>
Fri, 19 Feb 2010 11:40:39 +0000
changeset 8 7f131cf431bc
parent 7 83c4798e291d
child 9 3a392067015c
permissions -rwxr-xr-x
Switched to fileinput module. This allows to read also gzip/bzip2 compressed mail logs, w/o additional command-line options.
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
0
2d97e75f16cf initial commit: »don't fear the nervous delete finger«
Pascal Volk <user@localhost.localdomain.org>
parents:
diff changeset
     1
#!/usr/bin/env python
2d97e75f16cf initial commit: »don't fear the nervous delete finger«
Pascal Volk <user@localhost.localdomain.org>
parents:
diff changeset
     2
# -*- coding: utf-8 -*-
7
83c4798e291d Small code cleanups. Added current year to the copyright notices.
Pascal Volk <user@localhost.localdomain.org>
parents: 6
diff changeset
     3
# Copyright 2009 - 2010 Pascal Volk
5
65590f05bb97 Added COPYING to the repository
Pascal Volk <user@localhost.localdomain.org>
parents: 4
diff changeset
     4
# See COPYING for distribution information.
0
2d97e75f16cf initial commit: »don't fear the nervous delete finger«
Pascal Volk <user@localhost.localdomain.org>
parents:
diff changeset
     5
2d97e75f16cf initial commit: »don't fear the nervous delete finger«
Pascal Volk <user@localhost.localdomain.org>
parents:
diff changeset
     6
__author__ = 'Pascal Volk'
4
ae58d9881be5 showResult(): count matches; break if no matches were found
Pascal Volk <user@localhost.localdomain.org>
parents: 3
diff changeset
     7
__version__ = '0.1.2'
ae58d9881be5 showResult(): count matches; break if no matches were found
Pascal Volk <user@localhost.localdomain.org>
parents: 3
diff changeset
     8
__date__ = '2009-07-03'
0
2d97e75f16cf initial commit: »don't fear the nervous delete finger«
Pascal Volk <user@localhost.localdomain.org>
parents:
diff changeset
     9
2d97e75f16cf initial commit: »don't fear the nervous delete finger«
Pascal Volk <user@localhost.localdomain.org>
parents:
diff changeset
    10
import os
2d97e75f16cf initial commit: »don't fear the nervous delete finger«
Pascal Volk <user@localhost.localdomain.org>
parents:
diff changeset
    11
import re
8
7f131cf431bc Switched to fileinput module. This allows to read also gzip/bzip2
Pascal Volk <user@localhost.localdomain.org>
parents: 7
diff changeset
    12
import fileinput
0
2d97e75f16cf initial commit: »don't fear the nervous delete finger«
Pascal Volk <user@localhost.localdomain.org>
parents:
diff changeset
    13
6
723b4c4e15cb Set/updated descriptions and error messages.
Pascal Volk <user@localhost.localdomain.org>
parents: 5
diff changeset
    14
class NiXSapmSum(object):
723b4c4e15cb Set/updated descriptions and error messages.
Pascal Volk <user@localhost.localdomain.org>
parents: 5
diff changeset
    15
    """
723b4c4e15cb Set/updated descriptions and error messages.
Pascal Volk <user@localhost.localdomain.org>
parents: 5
diff changeset
    16
    Small log parser class to parse and summarize NiX Spam DNSBL lookup
723b4c4e15cb Set/updated descriptions and error messages.
Pascal Volk <user@localhost.localdomain.org>
parents: 5
diff changeset
    17
    based rejects from a mail log file.
723b4c4e15cb Set/updated descriptions and error messages.
Pascal Volk <user@localhost.localdomain.org>
parents: 5
diff changeset
    18
    """
723b4c4e15cb Set/updated descriptions and error messages.
Pascal Volk <user@localhost.localdomain.org>
parents: 5
diff changeset
    19
    __slots__ = ('_doms', '_mxs', '_repo')
0
2d97e75f16cf initial commit: »don't fear the nervous delete finger«
Pascal Volk <user@localhost.localdomain.org>
parents:
diff changeset
    20
2d97e75f16cf initial commit: »don't fear the nervous delete finger«
Pascal Volk <user@localhost.localdomain.org>
parents:
diff changeset
    21
    """Regular expression pattern for mail logs from Postfix"""
2d97e75f16cf initial commit: »don't fear the nervous delete finger«
Pascal Volk <user@localhost.localdomain.org>
parents:
diff changeset
    22
    RE_PF = '''^[\w\s:-]{17,80}\spostfix\/smtpd\[[\d]{3,5}\]: NOQUEUE: reject:.*blocked using ix.dnsbl.manitu.net; Spam sent to the mailhost ((?:[a-z0-9-]{1,63}\.){1,}[a-z]{2,6}) was detected by NiX Spam.*$'''
2d97e75f16cf initial commit: »don't fear the nervous delete finger«
Pascal Volk <user@localhost.localdomain.org>
parents:
diff changeset
    23
2d97e75f16cf initial commit: »don't fear the nervous delete finger«
Pascal Volk <user@localhost.localdomain.org>
parents:
diff changeset
    24
    def __init__(self):
2d97e75f16cf initial commit: »don't fear the nervous delete finger«
Pascal Volk <user@localhost.localdomain.org>
parents:
diff changeset
    25
        self._doms = {}
2d97e75f16cf initial commit: »don't fear the nervous delete finger«
Pascal Volk <user@localhost.localdomain.org>
parents:
diff changeset
    26
        self._mxs  = {}
2d97e75f16cf initial commit: »don't fear the nervous delete finger«
Pascal Volk <user@localhost.localdomain.org>
parents:
diff changeset
    27
        self._repo = None
2d97e75f16cf initial commit: »don't fear the nervous delete finger«
Pascal Volk <user@localhost.localdomain.org>
parents:
diff changeset
    28
2d97e75f16cf initial commit: »don't fear the nervous delete finger«
Pascal Volk <user@localhost.localdomain.org>
parents:
diff changeset
    29
    def setLogFormat(self, format='postfix'):
2d97e75f16cf initial commit: »don't fear the nervous delete finger«
Pascal Volk <user@localhost.localdomain.org>
parents:
diff changeset
    30
        if format == 'postfix':
1
7d5cee19c20a Renamed class NiXSpamPlot to NiXSapmSum. Added getDomLen(). Rewrote showResult()
Pascal Volk <user@localhost.localdomain.org>
parents: 0
diff changeset
    31
            self._repo = re.compile(NiXSapmSum.RE_PF)
0
2d97e75f16cf initial commit: »don't fear the nervous delete finger«
Pascal Volk <user@localhost.localdomain.org>
parents:
diff changeset
    32
        else:
2d97e75f16cf initial commit: »don't fear the nervous delete finger«
Pascal Volk <user@localhost.localdomain.org>
parents:
diff changeset
    33
            raise Exception('MTA/Logformat not supported yet.')
2d97e75f16cf initial commit: »don't fear the nervous delete finger«
Pascal Volk <user@localhost.localdomain.org>
parents:
diff changeset
    34
2d97e75f16cf initial commit: »don't fear the nervous delete finger«
Pascal Volk <user@localhost.localdomain.org>
parents:
diff changeset
    35
    def parseLog(self, filehandle):
2d97e75f16cf initial commit: »don't fear the nervous delete finger«
Pascal Volk <user@localhost.localdomain.org>
parents:
diff changeset
    36
        for l in filehandle:
2d97e75f16cf initial commit: »don't fear the nervous delete finger«
Pascal Volk <user@localhost.localdomain.org>
parents:
diff changeset
    37
            mo = self._repo.match(l)
2d97e75f16cf initial commit: »don't fear the nervous delete finger«
Pascal Volk <user@localhost.localdomain.org>
parents:
diff changeset
    38
            if mo:
2d97e75f16cf initial commit: »don't fear the nervous delete finger«
Pascal Volk <user@localhost.localdomain.org>
parents:
diff changeset
    39
                mx = mo.group(1)
2d97e75f16cf initial commit: »don't fear the nervous delete finger«
Pascal Volk <user@localhost.localdomain.org>
parents:
diff changeset
    40
                try:
2d97e75f16cf initial commit: »don't fear the nervous delete finger«
Pascal Volk <user@localhost.localdomain.org>
parents:
diff changeset
    41
                    self._mxs[mx] += 1
2d97e75f16cf initial commit: »don't fear the nervous delete finger«
Pascal Volk <user@localhost.localdomain.org>
parents:
diff changeset
    42
                except KeyError:
2d97e75f16cf initial commit: »don't fear the nervous delete finger«
Pascal Volk <user@localhost.localdomain.org>
parents:
diff changeset
    43
                    self._mxs[mx]  = 1
2d97e75f16cf initial commit: »don't fear the nervous delete finger«
Pascal Volk <user@localhost.localdomain.org>
parents:
diff changeset
    44
2d97e75f16cf initial commit: »don't fear the nervous delete finger«
Pascal Volk <user@localhost.localdomain.org>
parents:
diff changeset
    45
    def countByDom(self):
2d97e75f16cf initial commit: »don't fear the nervous delete finger«
Pascal Volk <user@localhost.localdomain.org>
parents:
diff changeset
    46
        for mx in self._mxs.keys():
2d97e75f16cf initial commit: »don't fear the nervous delete finger«
Pascal Volk <user@localhost.localdomain.org>
parents:
diff changeset
    47
            dom = '.'.join(mx.split('.')[-2:])
2d97e75f16cf initial commit: »don't fear the nervous delete finger«
Pascal Volk <user@localhost.localdomain.org>
parents:
diff changeset
    48
            try:
2d97e75f16cf initial commit: »don't fear the nervous delete finger«
Pascal Volk <user@localhost.localdomain.org>
parents:
diff changeset
    49
                self._doms[dom] += self._mxs[mx]
2d97e75f16cf initial commit: »don't fear the nervous delete finger«
Pascal Volk <user@localhost.localdomain.org>
parents:
diff changeset
    50
            except KeyError:
2d97e75f16cf initial commit: »don't fear the nervous delete finger«
Pascal Volk <user@localhost.localdomain.org>
parents:
diff changeset
    51
                self._doms[dom]  = self._mxs[mx]
2d97e75f16cf initial commit: »don't fear the nervous delete finger«
Pascal Volk <user@localhost.localdomain.org>
parents:
diff changeset
    52
2d97e75f16cf initial commit: »don't fear the nervous delete finger«
Pascal Volk <user@localhost.localdomain.org>
parents:
diff changeset
    53
    def getDomains(self):
2d97e75f16cf initial commit: »don't fear the nervous delete finger«
Pascal Volk <user@localhost.localdomain.org>
parents:
diff changeset
    54
        return self._doms
2d97e75f16cf initial commit: »don't fear the nervous delete finger«
Pascal Volk <user@localhost.localdomain.org>
parents:
diff changeset
    55
2d97e75f16cf initial commit: »don't fear the nervous delete finger«
Pascal Volk <user@localhost.localdomain.org>
parents:
diff changeset
    56
    def getMXs(self):
2d97e75f16cf initial commit: »don't fear the nervous delete finger«
Pascal Volk <user@localhost.localdomain.org>
parents:
diff changeset
    57
        return self._mxs
2d97e75f16cf initial commit: »don't fear the nervous delete finger«
Pascal Volk <user@localhost.localdomain.org>
parents:
diff changeset
    58
8
7f131cf431bc Switched to fileinput module. This allows to read also gzip/bzip2
Pascal Volk <user@localhost.localdomain.org>
parents: 7
diff changeset
    59
0
2d97e75f16cf initial commit: »don't fear the nervous delete finger«
Pascal Volk <user@localhost.localdomain.org>
parents:
diff changeset
    60
def getOptionParser():
2d97e75f16cf initial commit: »don't fear the nervous delete finger«
Pascal Volk <user@localhost.localdomain.org>
parents:
diff changeset
    61
    from optparse import OptionParser
6
723b4c4e15cb Set/updated descriptions and error messages.
Pascal Volk <user@localhost.localdomain.org>
parents: 5
diff changeset
    62
    description = 'NiX Spam DNSBL lookup based rejects summarizer'
0
2d97e75f16cf initial commit: »don't fear the nervous delete finger«
Pascal Volk <user@localhost.localdomain.org>
parents:
diff changeset
    63
    usage  = 'usage: %prog [options] maillog [maillog [...]]'
1
7d5cee19c20a Renamed class NiXSpamPlot to NiXSapmSum. Added getDomLen(). Rewrote showResult()
Pascal Volk <user@localhost.localdomain.org>
parents: 0
diff changeset
    64
    version = '%prog '+__version__
7d5cee19c20a Renamed class NiXSpamPlot to NiXSapmSum. Added getDomLen(). Rewrote showResult()
Pascal Volk <user@localhost.localdomain.org>
parents: 0
diff changeset
    65
    parser = OptionParser(description=description,usage=usage,version=version)
0
2d97e75f16cf initial commit: »don't fear the nervous delete finger«
Pascal Volk <user@localhost.localdomain.org>
parents:
diff changeset
    66
    parser.add_option('-d', action='store_true', dest='countByDom',
2d97e75f16cf initial commit: »don't fear the nervous delete finger«
Pascal Volk <user@localhost.localdomain.org>
parents:
diff changeset
    67
            default=False, help='summarize all MX by domain')
2d97e75f16cf initial commit: »don't fear the nervous delete finger«
Pascal Volk <user@localhost.localdomain.org>
parents:
diff changeset
    68
    parser.add_option('-m', action='store_false', dest='countByDom',
2d97e75f16cf initial commit: »don't fear the nervous delete finger«
Pascal Volk <user@localhost.localdomain.org>
parents:
diff changeset
    69
            help='count per MX host [default]')
2d97e75f16cf initial commit: »don't fear the nervous delete finger«
Pascal Volk <user@localhost.localdomain.org>
parents:
diff changeset
    70
    parser.add_option('-o', dest='oFormat', default='table',metavar='FORMAT',
7
83c4798e291d Small code cleanups. Added current year to the copyright notices.
Pascal Volk <user@localhost.localdomain.org>
parents: 6
diff changeset
    71
            choices=('csv', 'table'),
0
2d97e75f16cf initial commit: »don't fear the nervous delete finger«
Pascal Volk <user@localhost.localdomain.org>
parents:
diff changeset
    72
            help='the output format: table or csv [default: %default]')
2d97e75f16cf initial commit: »don't fear the nervous delete finger«
Pascal Volk <user@localhost.localdomain.org>
parents:
diff changeset
    73
    parser.add_option('-p', action='store_true', dest='percent', default=False,
2d97e75f16cf initial commit: »don't fear the nervous delete finger«
Pascal Volk <user@localhost.localdomain.org>
parents:
diff changeset
    74
            help='show also percentages in table output [default: %default]')
2d97e75f16cf initial commit: »don't fear the nervous delete finger«
Pascal Volk <user@localhost.localdomain.org>
parents:
diff changeset
    75
    parser.add_option('-s', dest='order', default='name', metavar='SORTBY',
2d97e75f16cf initial commit: »don't fear the nervous delete finger«
Pascal Volk <user@localhost.localdomain.org>
parents:
diff changeset
    76
            help='arrange output by: name or count [default: %default]')
2d97e75f16cf initial commit: »don't fear the nervous delete finger«
Pascal Volk <user@localhost.localdomain.org>
parents:
diff changeset
    77
    parser.add_option('-t', dest='format', default='postfix',metavar='MTA',
7
83c4798e291d Small code cleanups. Added current year to the copyright notices.
Pascal Volk <user@localhost.localdomain.org>
parents: 6
diff changeset
    78
            choices=('postfix',),
0
2d97e75f16cf initial commit: »don't fear the nervous delete finger«
Pascal Volk <user@localhost.localdomain.org>
parents:
diff changeset
    79
            help='MTA that generated the maillog [default: %default]')
2d97e75f16cf initial commit: »don't fear the nervous delete finger«
Pascal Volk <user@localhost.localdomain.org>
parents:
diff changeset
    80
    return parser
2d97e75f16cf initial commit: »don't fear the nervous delete finger«
Pascal Volk <user@localhost.localdomain.org>
parents:
diff changeset
    81
7
83c4798e291d Small code cleanups. Added current year to the copyright notices.
Pascal Volk <user@localhost.localdomain.org>
parents: 6
diff changeset
    82
8
7f131cf431bc Switched to fileinput module. This allows to read also gzip/bzip2
Pascal Volk <user@localhost.localdomain.org>
parents: 7
diff changeset
    83
def check_files(log_files):
7f131cf431bc Switched to fileinput module. This allows to read also gzip/bzip2
Pascal Volk <user@localhost.localdomain.org>
parents: 7
diff changeset
    84
    """Checks that all files from *log_files* exist and all of them are
7f131cf431bc Switched to fileinput module. This allows to read also gzip/bzip2
Pascal Volk <user@localhost.localdomain.org>
parents: 7
diff changeset
    85
    readable.
7f131cf431bc Switched to fileinput module. This allows to read also gzip/bzip2
Pascal Volk <user@localhost.localdomain.org>
parents: 7
diff changeset
    86
7f131cf431bc Switched to fileinput module. This allows to read also gzip/bzip2
Pascal Volk <user@localhost.localdomain.org>
parents: 7
diff changeset
    87
    If a file doesn't exist or is not readable, it will be removed from
7f131cf431bc Switched to fileinput module. This allows to read also gzip/bzip2
Pascal Volk <user@localhost.localdomain.org>
parents: 7
diff changeset
    88
    *log_files* set.
7f131cf431bc Switched to fileinput module. This allows to read also gzip/bzip2
Pascal Volk <user@localhost.localdomain.org>
parents: 7
diff changeset
    89
7f131cf431bc Switched to fileinput module. This allows to read also gzip/bzip2
Pascal Volk <user@localhost.localdomain.org>
parents: 7
diff changeset
    90
    This function will return *True*, if at least one file has passed the
7f131cf431bc Switched to fileinput module. This allows to read also gzip/bzip2
Pascal Volk <user@localhost.localdomain.org>
parents: 7
diff changeset
    91
    checks. Otherwise *False* will be returned. And the *log_files* set
7f131cf431bc Switched to fileinput module. This allows to read also gzip/bzip2
Pascal Volk <user@localhost.localdomain.org>
parents: 7
diff changeset
    92
    will be emptied.
7f131cf431bc Switched to fileinput module. This allows to read also gzip/bzip2
Pascal Volk <user@localhost.localdomain.org>
parents: 7
diff changeset
    93
7f131cf431bc Switched to fileinput module. This allows to read also gzip/bzip2
Pascal Volk <user@localhost.localdomain.org>
parents: 7
diff changeset
    94
    :param log_files: set of file names
7f131cf431bc Switched to fileinput module. This allows to read also gzip/bzip2
Pascal Volk <user@localhost.localdomain.org>
parents: 7
diff changeset
    95
    :type log_files: set
7f131cf431bc Switched to fileinput module. This allows to read also gzip/bzip2
Pascal Volk <user@localhost.localdomain.org>
parents: 7
diff changeset
    96
    :rtype: bool"""
7f131cf431bc Switched to fileinput module. This allows to read also gzip/bzip2
Pascal Volk <user@localhost.localdomain.org>
parents: 7
diff changeset
    97
    assert isinstance(log_files, set), 'log_files argument must be a set'
7f131cf431bc Switched to fileinput module. This allows to read also gzip/bzip2
Pascal Volk <user@localhost.localdomain.org>
parents: 7
diff changeset
    98
    lf_copy = log_files.copy()
7f131cf431bc Switched to fileinput module. This allows to read also gzip/bzip2
Pascal Volk <user@localhost.localdomain.org>
parents: 7
diff changeset
    99
    for lf in lf_copy:
7f131cf431bc Switched to fileinput module. This allows to read also gzip/bzip2
Pascal Volk <user@localhost.localdomain.org>
parents: 7
diff changeset
   100
        if not os.path.isfile(lf):
7f131cf431bc Switched to fileinput module. This allows to read also gzip/bzip2
Pascal Volk <user@localhost.localdomain.org>
parents: 7
diff changeset
   101
            os.sys.stderr.write('Warning: No such file: %r\n' % lf)
7f131cf431bc Switched to fileinput module. This allows to read also gzip/bzip2
Pascal Volk <user@localhost.localdomain.org>
parents: 7
diff changeset
   102
            log_files.remove(lf)
7f131cf431bc Switched to fileinput module. This allows to read also gzip/bzip2
Pascal Volk <user@localhost.localdomain.org>
parents: 7
diff changeset
   103
        elif not os.access(lf, os.R_OK):
7f131cf431bc Switched to fileinput module. This allows to read also gzip/bzip2
Pascal Volk <user@localhost.localdomain.org>
parents: 7
diff changeset
   104
            os.sys.stderr.write('Warning: Cannot read file: %r\n' % lf)
7f131cf431bc Switched to fileinput module. This allows to read also gzip/bzip2
Pascal Volk <user@localhost.localdomain.org>
parents: 7
diff changeset
   105
            log_files.remove(lf)
7f131cf431bc Switched to fileinput module. This allows to read also gzip/bzip2
Pascal Volk <user@localhost.localdomain.org>
parents: 7
diff changeset
   106
7f131cf431bc Switched to fileinput module. This allows to read also gzip/bzip2
Pascal Volk <user@localhost.localdomain.org>
parents: 7
diff changeset
   107
    if log_files:
7f131cf431bc Switched to fileinput module. This allows to read also gzip/bzip2
Pascal Volk <user@localhost.localdomain.org>
parents: 7
diff changeset
   108
        return True
7f131cf431bc Switched to fileinput module. This allows to read also gzip/bzip2
Pascal Volk <user@localhost.localdomain.org>
parents: 7
diff changeset
   109
    return False
0
2d97e75f16cf initial commit: »don't fear the nervous delete finger«
Pascal Volk <user@localhost.localdomain.org>
parents:
diff changeset
   110
1
7d5cee19c20a Renamed class NiXSpamPlot to NiXSapmSum. Added getDomLen(). Rewrote showResult()
Pascal Volk <user@localhost.localdomain.org>
parents: 0
diff changeset
   111
2
a72ea07394cc NiXSapmSum.getTotal(): deleted
Pascal Volk <user@localhost.localdomain.org>
parents: 1
diff changeset
   112
def buildTable(output, domains, percent, orderBy):
a72ea07394cc NiXSapmSum.getTotal(): deleted
Pascal Volk <user@localhost.localdomain.org>
parents: 1
diff changeset
   113
    k = 0 if orderBy == 'name' else 1
a72ea07394cc NiXSapmSum.getTotal(): deleted
Pascal Volk <user@localhost.localdomain.org>
parents: 1
diff changeset
   114
    doms = sorted(domains.items(), lambda d,c: cmp(d[k],c[k]), reverse=k)
7
83c4798e291d Small code cleanups. Added current year to the copyright notices.
Pascal Volk <user@localhost.localdomain.org>
parents: 6
diff changeset
   115
    dlen = len(max(domains.iterkeys(), key=len)) + 1
2
a72ea07394cc NiXSapmSum.getTotal(): deleted
Pascal Volk <user@localhost.localdomain.org>
parents: 1
diff changeset
   116
    clen = len(str(max(domains.values())))
a72ea07394cc NiXSapmSum.getTotal(): deleted
Pascal Volk <user@localhost.localdomain.org>
parents: 1
diff changeset
   117
    total = sum(domains.values())
a72ea07394cc NiXSapmSum.getTotal(): deleted
Pascal Volk <user@localhost.localdomain.org>
parents: 1
diff changeset
   118
    if percent:
3
6b0d09cdfbdb buildTable(): small cosmetics/enhancements
Pascal Volk <user@localhost.localdomain.org>
parents: 2
diff changeset
   119
        format = ' %%%ds  %%%dd  %%6.2f %%%%\n' % (-dlen, clen)
2
a72ea07394cc NiXSapmSum.getTotal(): deleted
Pascal Volk <user@localhost.localdomain.org>
parents: 1
diff changeset
   120
        for d, c in doms:
a72ea07394cc NiXSapmSum.getTotal(): deleted
Pascal Volk <user@localhost.localdomain.org>
parents: 1
diff changeset
   121
            dfrac = 100./total*c
a72ea07394cc NiXSapmSum.getTotal(): deleted
Pascal Volk <user@localhost.localdomain.org>
parents: 1
diff changeset
   122
            output.write(format % (d, c, dfrac))
3
6b0d09cdfbdb buildTable(): small cosmetics/enhancements
Pascal Volk <user@localhost.localdomain.org>
parents: 2
diff changeset
   123
        output.write('%s\n' % ((clen+dlen+14)*'-'))
6b0d09cdfbdb buildTable(): small cosmetics/enhancements
Pascal Volk <user@localhost.localdomain.org>
parents: 2
diff changeset
   124
        output.write(format % ('total', total, 100))
2
a72ea07394cc NiXSapmSum.getTotal(): deleted
Pascal Volk <user@localhost.localdomain.org>
parents: 1
diff changeset
   125
    else:
3
6b0d09cdfbdb buildTable(): small cosmetics/enhancements
Pascal Volk <user@localhost.localdomain.org>
parents: 2
diff changeset
   126
        format = ' %%%ds  %%%dd\n' % (-dlen, clen)
2
a72ea07394cc NiXSapmSum.getTotal(): deleted
Pascal Volk <user@localhost.localdomain.org>
parents: 1
diff changeset
   127
        for d in doms:
a72ea07394cc NiXSapmSum.getTotal(): deleted
Pascal Volk <user@localhost.localdomain.org>
parents: 1
diff changeset
   128
            output.write(format % d)
3
6b0d09cdfbdb buildTable(): small cosmetics/enhancements
Pascal Volk <user@localhost.localdomain.org>
parents: 2
diff changeset
   129
        output.write('%s\n' % ((clen+dlen+4)*'-'))
6b0d09cdfbdb buildTable(): small cosmetics/enhancements
Pascal Volk <user@localhost.localdomain.org>
parents: 2
diff changeset
   130
        output.write(format % ('total', total))
2
a72ea07394cc NiXSapmSum.getTotal(): deleted
Pascal Volk <user@localhost.localdomain.org>
parents: 1
diff changeset
   131
1
7d5cee19c20a Renamed class NiXSpamPlot to NiXSapmSum. Added getDomLen(). Rewrote showResult()
Pascal Volk <user@localhost.localdomain.org>
parents: 0
diff changeset
   132
def showResult(nixspamsum, options):
0
2d97e75f16cf initial commit: »don't fear the nervous delete finger«
Pascal Volk <user@localhost.localdomain.org>
parents:
diff changeset
   133
    if options.countByDom:
1
7d5cee19c20a Renamed class NiXSpamPlot to NiXSapmSum. Added getDomLen(). Rewrote showResult()
Pascal Volk <user@localhost.localdomain.org>
parents: 0
diff changeset
   134
        nixspamsum.countByDom()
7d5cee19c20a Renamed class NiXSpamPlot to NiXSapmSum. Added getDomLen(). Rewrote showResult()
Pascal Volk <user@localhost.localdomain.org>
parents: 0
diff changeset
   135
        domains = nixspamsum.getDomains()
0
2d97e75f16cf initial commit: »don't fear the nervous delete finger«
Pascal Volk <user@localhost.localdomain.org>
parents:
diff changeset
   136
    else:
1
7d5cee19c20a Renamed class NiXSpamPlot to NiXSapmSum. Added getDomLen(). Rewrote showResult()
Pascal Volk <user@localhost.localdomain.org>
parents: 0
diff changeset
   137
        domains = nixspamsum.getMXs()
7
83c4798e291d Small code cleanups. Added current year to the copyright notices.
Pascal Volk <user@localhost.localdomain.org>
parents: 6
diff changeset
   138
    if not domains:
6
723b4c4e15cb Set/updated descriptions and error messages.
Pascal Volk <user@localhost.localdomain.org>
parents: 5
diff changeset
   139
        print "No NiX Spam DNSBL rejects found"
4
ae58d9881be5 showResult(): count matches; break if no matches were found
Pascal Volk <user@localhost.localdomain.org>
parents: 3
diff changeset
   140
        return
ae58d9881be5 showResult(): count matches; break if no matches were found
Pascal Volk <user@localhost.localdomain.org>
parents: 3
diff changeset
   141
ae58d9881be5 showResult(): count matches; break if no matches were found
Pascal Volk <user@localhost.localdomain.org>
parents: 3
diff changeset
   142
    from cStringIO import StringIO
ae58d9881be5 showResult(): count matches; break if no matches were found
Pascal Volk <user@localhost.localdomain.org>
parents: 3
diff changeset
   143
    output = StringIO()
2
a72ea07394cc NiXSapmSum.getTotal(): deleted
Pascal Volk <user@localhost.localdomain.org>
parents: 1
diff changeset
   144
    # build the table
1
7d5cee19c20a Renamed class NiXSpamPlot to NiXSapmSum. Added getDomLen(). Rewrote showResult()
Pascal Volk <user@localhost.localdomain.org>
parents: 0
diff changeset
   145
    if options.oFormat == 'table':
2
a72ea07394cc NiXSapmSum.getTotal(): deleted
Pascal Volk <user@localhost.localdomain.org>
parents: 1
diff changeset
   146
        buildTable(output, domains, options.percent, options.order)
a72ea07394cc NiXSapmSum.getTotal(): deleted
Pascal Volk <user@localhost.localdomain.org>
parents: 1
diff changeset
   147
    # generate comma separated values
1
7d5cee19c20a Renamed class NiXSpamPlot to NiXSapmSum. Added getDomLen(). Rewrote showResult()
Pascal Volk <user@localhost.localdomain.org>
parents: 0
diff changeset
   148
    elif options.oFormat == 'csv':
4
ae58d9881be5 showResult(): count matches; break if no matches were found
Pascal Volk <user@localhost.localdomain.org>
parents: 3
diff changeset
   149
        order = domains.keys()
ae58d9881be5 showResult(): count matches; break if no matches were found
Pascal Volk <user@localhost.localdomain.org>
parents: 3
diff changeset
   150
        order.sort()
ae58d9881be5 showResult(): count matches; break if no matches were found
Pascal Volk <user@localhost.localdomain.org>
parents: 3
diff changeset
   151
        for d in order:
ae58d9881be5 showResult(): count matches; break if no matches were found
Pascal Volk <user@localhost.localdomain.org>
parents: 3
diff changeset
   152
            output.write("'%s',%d\n" % (d, domains[d]))
2
a72ea07394cc NiXSapmSum.getTotal(): deleted
Pascal Volk <user@localhost.localdomain.org>
parents: 1
diff changeset
   153
    # should never be reached
1
7d5cee19c20a Renamed class NiXSpamPlot to NiXSapmSum. Added getDomLen(). Rewrote showResult()
Pascal Volk <user@localhost.localdomain.org>
parents: 0
diff changeset
   154
    else:
2
a72ea07394cc NiXSapmSum.getTotal(): deleted
Pascal Volk <user@localhost.localdomain.org>
parents: 1
diff changeset
   155
        print "Oops, error in function showResult() happend"
a72ea07394cc NiXSapmSum.getTotal(): deleted
Pascal Volk <user@localhost.localdomain.org>
parents: 1
diff changeset
   156
    # show the result
a72ea07394cc NiXSapmSum.getTotal(): deleted
Pascal Volk <user@localhost.localdomain.org>
parents: 1
diff changeset
   157
    print output.getvalue()
0
2d97e75f16cf initial commit: »don't fear the nervous delete finger«
Pascal Volk <user@localhost.localdomain.org>
parents:
diff changeset
   158
2d97e75f16cf initial commit: »don't fear the nervous delete finger«
Pascal Volk <user@localhost.localdomain.org>
parents:
diff changeset
   159
def main():
2d97e75f16cf initial commit: »don't fear the nervous delete finger«
Pascal Volk <user@localhost.localdomain.org>
parents:
diff changeset
   160
    parser = getOptionParser()
2d97e75f16cf initial commit: »don't fear the nervous delete finger«
Pascal Volk <user@localhost.localdomain.org>
parents:
diff changeset
   161
    opts, args = parser.parse_args()
8
7f131cf431bc Switched to fileinput module. This allows to read also gzip/bzip2
Pascal Volk <user@localhost.localdomain.org>
parents: 7
diff changeset
   162
    if not args:
6
723b4c4e15cb Set/updated descriptions and error messages.
Pascal Volk <user@localhost.localdomain.org>
parents: 5
diff changeset
   163
        parser.error('No log file specified')
8
7f131cf431bc Switched to fileinput module. This allows to read also gzip/bzip2
Pascal Volk <user@localhost.localdomain.org>
parents: 7
diff changeset
   164
    # drop duplicates
7f131cf431bc Switched to fileinput module. This allows to read also gzip/bzip2
Pascal Volk <user@localhost.localdomain.org>
parents: 7
diff changeset
   165
    log_files = set(args)
7f131cf431bc Switched to fileinput module. This allows to read also gzip/bzip2
Pascal Volk <user@localhost.localdomain.org>
parents: 7
diff changeset
   166
    # remove inexistent/unreadable files
7f131cf431bc Switched to fileinput module. This allows to read also gzip/bzip2
Pascal Volk <user@localhost.localdomain.org>
parents: 7
diff changeset
   167
    if not check_files(log_files):
7f131cf431bc Switched to fileinput module. This allows to read also gzip/bzip2
Pascal Volk <user@localhost.localdomain.org>
parents: 7
diff changeset
   168
        os.sys.stderr.write('No readable log files found\n')
7f131cf431bc Switched to fileinput module. This allows to read also gzip/bzip2
Pascal Volk <user@localhost.localdomain.org>
parents: 7
diff changeset
   169
        return 1
7f131cf431bc Switched to fileinput module. This allows to read also gzip/bzip2
Pascal Volk <user@localhost.localdomain.org>
parents: 7
diff changeset
   170
1
7d5cee19c20a Renamed class NiXSpamPlot to NiXSapmSum. Added getDomLen(). Rewrote showResult()
Pascal Volk <user@localhost.localdomain.org>
parents: 0
diff changeset
   171
    nixss = NiXSapmSum()
7d5cee19c20a Renamed class NiXSpamPlot to NiXSapmSum. Added getDomLen(). Rewrote showResult()
Pascal Volk <user@localhost.localdomain.org>
parents: 0
diff changeset
   172
    nixss.setLogFormat(opts.format)
8
7f131cf431bc Switched to fileinput module. This allows to read also gzip/bzip2
Pascal Volk <user@localhost.localdomain.org>
parents: 7
diff changeset
   173
7f131cf431bc Switched to fileinput module. This allows to read also gzip/bzip2
Pascal Volk <user@localhost.localdomain.org>
parents: 7
diff changeset
   174
    fi = fileinput.FileInput(log_files, openhook=fileinput.hook_compressed)
7f131cf431bc Switched to fileinput module. This allows to read also gzip/bzip2
Pascal Volk <user@localhost.localdomain.org>
parents: 7
diff changeset
   175
    nixss.parseLog(fi)
7f131cf431bc Switched to fileinput module. This allows to read also gzip/bzip2
Pascal Volk <user@localhost.localdomain.org>
parents: 7
diff changeset
   176
    fi.close()
1
7d5cee19c20a Renamed class NiXSpamPlot to NiXSapmSum. Added getDomLen(). Rewrote showResult()
Pascal Volk <user@localhost.localdomain.org>
parents: 0
diff changeset
   177
    showResult(nixss, opts)
8
7f131cf431bc Switched to fileinput module. This allows to read also gzip/bzip2
Pascal Volk <user@localhost.localdomain.org>
parents: 7
diff changeset
   178
    return 0
0
2d97e75f16cf initial commit: »don't fear the nervous delete finger«
Pascal Volk <user@localhost.localdomain.org>
parents:
diff changeset
   179
2d97e75f16cf initial commit: »don't fear the nervous delete finger«
Pascal Volk <user@localhost.localdomain.org>
parents:
diff changeset
   180
if __name__ == '__main__':
8
7f131cf431bc Switched to fileinput module. This allows to read also gzip/bzip2
Pascal Volk <user@localhost.localdomain.org>
parents: 7
diff changeset
   181
    os.sys.exit(main())