nixspamsum
author Pascal Volk <user@localhost.localdomain.org>
Sun, 06 Feb 2011 12:47:36 +0000
changeset 13 b0c05ce0f44c
parent 11 a5f5a8f288da
child 14 1fc9b5ed89be
permissions -rwxr-xr-x
Adjusted NiXSapmSum.RE_PF to the new TXT RR format.
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
0
2d97e75f16cf initial commit: »don't fear the nervous delete finger«
Pascal Volk <user@localhost.localdomain.org>
parents:
diff changeset
     1
#!/usr/bin/env python
2d97e75f16cf initial commit: »don't fear the nervous delete finger«
Pascal Volk <user@localhost.localdomain.org>
parents:
diff changeset
     2
# -*- coding: utf-8 -*-
7
83c4798e291d Small code cleanups. Added current year to the copyright notices.
Pascal Volk <user@localhost.localdomain.org>
parents: 6
diff changeset
     3
# Copyright 2009 - 2010 Pascal Volk
5
65590f05bb97 Added COPYING to the repository
Pascal Volk <user@localhost.localdomain.org>
parents: 4
diff changeset
     4
# See COPYING for distribution information.
0
2d97e75f16cf initial commit: »don't fear the nervous delete finger«
Pascal Volk <user@localhost.localdomain.org>
parents:
diff changeset
     5
2d97e75f16cf initial commit: »don't fear the nervous delete finger«
Pascal Volk <user@localhost.localdomain.org>
parents:
diff changeset
     6
__author__ = 'Pascal Volk'
11
a5f5a8f288da PEP-8-ified the code. Version 0.1.3
Pascal Volk <user@localhost.localdomain.org>
parents: 10
diff changeset
     7
__version__ = '0.1.3'
a5f5a8f288da PEP-8-ified the code. Version 0.1.3
Pascal Volk <user@localhost.localdomain.org>
parents: 10
diff changeset
     8
__date__ = '2010-04-11'
0
2d97e75f16cf initial commit: »don't fear the nervous delete finger«
Pascal Volk <user@localhost.localdomain.org>
parents:
diff changeset
     9
2d97e75f16cf initial commit: »don't fear the nervous delete finger«
Pascal Volk <user@localhost.localdomain.org>
parents:
diff changeset
    10
import os
2d97e75f16cf initial commit: »don't fear the nervous delete finger«
Pascal Volk <user@localhost.localdomain.org>
parents:
diff changeset
    11
import re
8
7f131cf431bc Switched to fileinput module. This allows to read also gzip/bzip2
Pascal Volk <user@localhost.localdomain.org>
parents: 7
diff changeset
    12
import fileinput
0
2d97e75f16cf initial commit: »don't fear the nervous delete finger«
Pascal Volk <user@localhost.localdomain.org>
parents:
diff changeset
    13
11
a5f5a8f288da PEP-8-ified the code. Version 0.1.3
Pascal Volk <user@localhost.localdomain.org>
parents: 10
diff changeset
    14
6
723b4c4e15cb Set/updated descriptions and error messages.
Pascal Volk <user@localhost.localdomain.org>
parents: 5
diff changeset
    15
class NiXSapmSum(object):
723b4c4e15cb Set/updated descriptions and error messages.
Pascal Volk <user@localhost.localdomain.org>
parents: 5
diff changeset
    16
    """
723b4c4e15cb Set/updated descriptions and error messages.
Pascal Volk <user@localhost.localdomain.org>
parents: 5
diff changeset
    17
    Small log parser class to parse and summarize NiX Spam DNSBL lookup
723b4c4e15cb Set/updated descriptions and error messages.
Pascal Volk <user@localhost.localdomain.org>
parents: 5
diff changeset
    18
    based rejects from a mail log file.
723b4c4e15cb Set/updated descriptions and error messages.
Pascal Volk <user@localhost.localdomain.org>
parents: 5
diff changeset
    19
    """
723b4c4e15cb Set/updated descriptions and error messages.
Pascal Volk <user@localhost.localdomain.org>
parents: 5
diff changeset
    20
    __slots__ = ('_doms', '_mxs', '_repo')
0
2d97e75f16cf initial commit: »don't fear the nervous delete finger«
Pascal Volk <user@localhost.localdomain.org>
parents:
diff changeset
    21
9
3a392067015c Count also naked IPv4 addresses. Beautified NiXSapmSum:RE_PF.
Pascal Volk <user@localhost.localdomain.org>
parents: 8
diff changeset
    22
    RE_FQDN = '(?:[a-z0-9-]{1,63}\.){1,}[a-z]{2,6}'
3a392067015c Count also naked IPv4 addresses. Beautified NiXSapmSum:RE_PF.
Pascal Volk <user@localhost.localdomain.org>
parents: 8
diff changeset
    23
    RE_IPv4 = '(?:[\d]{1,3}\.){3}[\d]{1,3}'
0
2d97e75f16cf initial commit: »don't fear the nervous delete finger«
Pascal Volk <user@localhost.localdomain.org>
parents:
diff changeset
    24
    """Regular expression pattern for mail logs from Postfix"""
9
3a392067015c Count also naked IPv4 addresses. Beautified NiXSapmSum:RE_PF.
Pascal Volk <user@localhost.localdomain.org>
parents: 8
diff changeset
    25
    RE_PF = r'''^[\w\s:-]{17,80}\spostfix\/smtpd\[[\d]{1,5}\]:\sNOQUEUE:
3a392067015c Count also naked IPv4 addresses. Beautified NiXSapmSum:RE_PF.
Pascal Volk <user@localhost.localdomain.org>
parents: 8
diff changeset
    26
                \sreject:.*blocked\susing\six.dnsbl.manitu.net;
13
b0c05ce0f44c Adjusted NiXSapmSum.RE_PF to the new TXT RR format.
Pascal Volk <user@localhost.localdomain.org>
parents: 11
diff changeset
    27
                \sYour\se\-mail\sservice\swas\sdetected\sby\s(%s|%s)
b0c05ce0f44c Adjusted NiXSapmSum.RE_PF to the new TXT RR format.
Pascal Volk <user@localhost.localdomain.org>
parents: 11
diff changeset
    28
                \s\(NiX\sSpam\)\sas\sspamming\sat.*$''' % (RE_FQDN, RE_IPv4)
9
3a392067015c Count also naked IPv4 addresses. Beautified NiXSapmSum:RE_PF.
Pascal Volk <user@localhost.localdomain.org>
parents: 8
diff changeset
    29
0
2d97e75f16cf initial commit: »don't fear the nervous delete finger«
Pascal Volk <user@localhost.localdomain.org>
parents:
diff changeset
    30
    def __init__(self):
2d97e75f16cf initial commit: »don't fear the nervous delete finger«
Pascal Volk <user@localhost.localdomain.org>
parents:
diff changeset
    31
        self._doms = {}
11
a5f5a8f288da PEP-8-ified the code. Version 0.1.3
Pascal Volk <user@localhost.localdomain.org>
parents: 10
diff changeset
    32
        self._mxs = {}
0
2d97e75f16cf initial commit: »don't fear the nervous delete finger«
Pascal Volk <user@localhost.localdomain.org>
parents:
diff changeset
    33
        self._repo = None
2d97e75f16cf initial commit: »don't fear the nervous delete finger«
Pascal Volk <user@localhost.localdomain.org>
parents:
diff changeset
    34
2d97e75f16cf initial commit: »don't fear the nervous delete finger«
Pascal Volk <user@localhost.localdomain.org>
parents:
diff changeset
    35
    def setLogFormat(self, format='postfix'):
2d97e75f16cf initial commit: »don't fear the nervous delete finger«
Pascal Volk <user@localhost.localdomain.org>
parents:
diff changeset
    36
        if format == 'postfix':
9
3a392067015c Count also naked IPv4 addresses. Beautified NiXSapmSum:RE_PF.
Pascal Volk <user@localhost.localdomain.org>
parents: 8
diff changeset
    37
            self._repo = re.compile(NiXSapmSum.RE_PF, re.VERBOSE)
0
2d97e75f16cf initial commit: »don't fear the nervous delete finger«
Pascal Volk <user@localhost.localdomain.org>
parents:
diff changeset
    38
        else:
2d97e75f16cf initial commit: »don't fear the nervous delete finger«
Pascal Volk <user@localhost.localdomain.org>
parents:
diff changeset
    39
            raise Exception('MTA/Logformat not supported yet.')
2d97e75f16cf initial commit: »don't fear the nervous delete finger«
Pascal Volk <user@localhost.localdomain.org>
parents:
diff changeset
    40
2d97e75f16cf initial commit: »don't fear the nervous delete finger«
Pascal Volk <user@localhost.localdomain.org>
parents:
diff changeset
    41
    def parseLog(self, filehandle):
2d97e75f16cf initial commit: »don't fear the nervous delete finger«
Pascal Volk <user@localhost.localdomain.org>
parents:
diff changeset
    42
        for l in filehandle:
2d97e75f16cf initial commit: »don't fear the nervous delete finger«
Pascal Volk <user@localhost.localdomain.org>
parents:
diff changeset
    43
            mo = self._repo.match(l)
2d97e75f16cf initial commit: »don't fear the nervous delete finger«
Pascal Volk <user@localhost.localdomain.org>
parents:
diff changeset
    44
            if mo:
2d97e75f16cf initial commit: »don't fear the nervous delete finger«
Pascal Volk <user@localhost.localdomain.org>
parents:
diff changeset
    45
                mx = mo.group(1)
2d97e75f16cf initial commit: »don't fear the nervous delete finger«
Pascal Volk <user@localhost.localdomain.org>
parents:
diff changeset
    46
                try:
2d97e75f16cf initial commit: »don't fear the nervous delete finger«
Pascal Volk <user@localhost.localdomain.org>
parents:
diff changeset
    47
                    self._mxs[mx] += 1
2d97e75f16cf initial commit: »don't fear the nervous delete finger«
Pascal Volk <user@localhost.localdomain.org>
parents:
diff changeset
    48
                except KeyError:
11
a5f5a8f288da PEP-8-ified the code. Version 0.1.3
Pascal Volk <user@localhost.localdomain.org>
parents: 10
diff changeset
    49
                    self._mxs[mx] = 1
0
2d97e75f16cf initial commit: »don't fear the nervous delete finger«
Pascal Volk <user@localhost.localdomain.org>
parents:
diff changeset
    50
2d97e75f16cf initial commit: »don't fear the nervous delete finger«
Pascal Volk <user@localhost.localdomain.org>
parents:
diff changeset
    51
    def countByDom(self):
9
3a392067015c Count also naked IPv4 addresses. Beautified NiXSapmSum:RE_PF.
Pascal Volk <user@localhost.localdomain.org>
parents: 8
diff changeset
    52
        ipv4po = re.compile(NiXSapmSum.RE_IPv4)
0
2d97e75f16cf initial commit: »don't fear the nervous delete finger«
Pascal Volk <user@localhost.localdomain.org>
parents:
diff changeset
    53
        for mx in self._mxs.keys():
9
3a392067015c Count also naked IPv4 addresses. Beautified NiXSapmSum:RE_PF.
Pascal Volk <user@localhost.localdomain.org>
parents: 8
diff changeset
    54
            mo = ipv4po.match(mx)
3a392067015c Count also naked IPv4 addresses. Beautified NiXSapmSum:RE_PF.
Pascal Volk <user@localhost.localdomain.org>
parents: 8
diff changeset
    55
            if mo:
3a392067015c Count also naked IPv4 addresses. Beautified NiXSapmSum:RE_PF.
Pascal Volk <user@localhost.localdomain.org>
parents: 8
diff changeset
    56
                dom = mo.group(0)
3a392067015c Count also naked IPv4 addresses. Beautified NiXSapmSum:RE_PF.
Pascal Volk <user@localhost.localdomain.org>
parents: 8
diff changeset
    57
            else:
3a392067015c Count also naked IPv4 addresses. Beautified NiXSapmSum:RE_PF.
Pascal Volk <user@localhost.localdomain.org>
parents: 8
diff changeset
    58
                dom = '.'.join(mx.split('.')[-2:])
0
2d97e75f16cf initial commit: »don't fear the nervous delete finger«
Pascal Volk <user@localhost.localdomain.org>
parents:
diff changeset
    59
            try:
2d97e75f16cf initial commit: »don't fear the nervous delete finger«
Pascal Volk <user@localhost.localdomain.org>
parents:
diff changeset
    60
                self._doms[dom] += self._mxs[mx]
2d97e75f16cf initial commit: »don't fear the nervous delete finger«
Pascal Volk <user@localhost.localdomain.org>
parents:
diff changeset
    61
            except KeyError:
11
a5f5a8f288da PEP-8-ified the code. Version 0.1.3
Pascal Volk <user@localhost.localdomain.org>
parents: 10
diff changeset
    62
                self._doms[dom] = self._mxs[mx]
0
2d97e75f16cf initial commit: »don't fear the nervous delete finger«
Pascal Volk <user@localhost.localdomain.org>
parents:
diff changeset
    63
2d97e75f16cf initial commit: »don't fear the nervous delete finger«
Pascal Volk <user@localhost.localdomain.org>
parents:
diff changeset
    64
    def getDomains(self):
2d97e75f16cf initial commit: »don't fear the nervous delete finger«
Pascal Volk <user@localhost.localdomain.org>
parents:
diff changeset
    65
        return self._doms
2d97e75f16cf initial commit: »don't fear the nervous delete finger«
Pascal Volk <user@localhost.localdomain.org>
parents:
diff changeset
    66
2d97e75f16cf initial commit: »don't fear the nervous delete finger«
Pascal Volk <user@localhost.localdomain.org>
parents:
diff changeset
    67
    def getMXs(self):
2d97e75f16cf initial commit: »don't fear the nervous delete finger«
Pascal Volk <user@localhost.localdomain.org>
parents:
diff changeset
    68
        return self._mxs
2d97e75f16cf initial commit: »don't fear the nervous delete finger«
Pascal Volk <user@localhost.localdomain.org>
parents:
diff changeset
    69
8
7f131cf431bc Switched to fileinput module. This allows to read also gzip/bzip2
Pascal Volk <user@localhost.localdomain.org>
parents: 7
diff changeset
    70
0
2d97e75f16cf initial commit: »don't fear the nervous delete finger«
Pascal Volk <user@localhost.localdomain.org>
parents:
diff changeset
    71
def getOptionParser():
2d97e75f16cf initial commit: »don't fear the nervous delete finger«
Pascal Volk <user@localhost.localdomain.org>
parents:
diff changeset
    72
    from optparse import OptionParser
6
723b4c4e15cb Set/updated descriptions and error messages.
Pascal Volk <user@localhost.localdomain.org>
parents: 5
diff changeset
    73
    description = 'NiX Spam DNSBL lookup based rejects summarizer'
11
a5f5a8f288da PEP-8-ified the code. Version 0.1.3
Pascal Volk <user@localhost.localdomain.org>
parents: 10
diff changeset
    74
    usage = 'usage: %prog [options] maillog [maillog [...]]'
a5f5a8f288da PEP-8-ified the code. Version 0.1.3
Pascal Volk <user@localhost.localdomain.org>
parents: 10
diff changeset
    75
    version = '%prog ' + __version__
a5f5a8f288da PEP-8-ified the code. Version 0.1.3
Pascal Volk <user@localhost.localdomain.org>
parents: 10
diff changeset
    76
    parser = OptionParser(description=description, usage=usage,
a5f5a8f288da PEP-8-ified the code. Version 0.1.3
Pascal Volk <user@localhost.localdomain.org>
parents: 10
diff changeset
    77
                          version=version)
0
2d97e75f16cf initial commit: »don't fear the nervous delete finger«
Pascal Volk <user@localhost.localdomain.org>
parents:
diff changeset
    78
    parser.add_option('-d', action='store_true', dest='countByDom',
2d97e75f16cf initial commit: »don't fear the nervous delete finger«
Pascal Volk <user@localhost.localdomain.org>
parents:
diff changeset
    79
            default=False, help='summarize all MX by domain')
2d97e75f16cf initial commit: »don't fear the nervous delete finger«
Pascal Volk <user@localhost.localdomain.org>
parents:
diff changeset
    80
    parser.add_option('-m', action='store_false', dest='countByDom',
2d97e75f16cf initial commit: »don't fear the nervous delete finger«
Pascal Volk <user@localhost.localdomain.org>
parents:
diff changeset
    81
            help='count per MX host [default]')
11
a5f5a8f288da PEP-8-ified the code. Version 0.1.3
Pascal Volk <user@localhost.localdomain.org>
parents: 10
diff changeset
    82
    parser.add_option('-o', dest='oFormat', default='table', metavar='FORMAT',
7
83c4798e291d Small code cleanups. Added current year to the copyright notices.
Pascal Volk <user@localhost.localdomain.org>
parents: 6
diff changeset
    83
            choices=('csv', 'table'),
0
2d97e75f16cf initial commit: »don't fear the nervous delete finger«
Pascal Volk <user@localhost.localdomain.org>
parents:
diff changeset
    84
            help='the output format: table or csv [default: %default]')
2d97e75f16cf initial commit: »don't fear the nervous delete finger«
Pascal Volk <user@localhost.localdomain.org>
parents:
diff changeset
    85
    parser.add_option('-p', action='store_true', dest='percent', default=False,
2d97e75f16cf initial commit: »don't fear the nervous delete finger«
Pascal Volk <user@localhost.localdomain.org>
parents:
diff changeset
    86
            help='show also percentages in table output [default: %default]')
2d97e75f16cf initial commit: »don't fear the nervous delete finger«
Pascal Volk <user@localhost.localdomain.org>
parents:
diff changeset
    87
    parser.add_option('-s', dest='order', default='name', metavar='SORTBY',
10
07b9fe5c6fcf getOptionParser(): Check -s option's value -> reasonable error message.
Pascal Volk <user@localhost.localdomain.org>
parents: 9
diff changeset
    88
            choices=('count', 'name'),
0
2d97e75f16cf initial commit: »don't fear the nervous delete finger«
Pascal Volk <user@localhost.localdomain.org>
parents:
diff changeset
    89
            help='arrange output by: name or count [default: %default]')
11
a5f5a8f288da PEP-8-ified the code. Version 0.1.3
Pascal Volk <user@localhost.localdomain.org>
parents: 10
diff changeset
    90
    parser.add_option('-t', dest='format', default='postfix', metavar='MTA',
7
83c4798e291d Small code cleanups. Added current year to the copyright notices.
Pascal Volk <user@localhost.localdomain.org>
parents: 6
diff changeset
    91
            choices=('postfix',),
0
2d97e75f16cf initial commit: »don't fear the nervous delete finger«
Pascal Volk <user@localhost.localdomain.org>
parents:
diff changeset
    92
            help='MTA that generated the maillog [default: %default]')
2d97e75f16cf initial commit: »don't fear the nervous delete finger«
Pascal Volk <user@localhost.localdomain.org>
parents:
diff changeset
    93
    return parser
2d97e75f16cf initial commit: »don't fear the nervous delete finger«
Pascal Volk <user@localhost.localdomain.org>
parents:
diff changeset
    94
7
83c4798e291d Small code cleanups. Added current year to the copyright notices.
Pascal Volk <user@localhost.localdomain.org>
parents: 6
diff changeset
    95
8
7f131cf431bc Switched to fileinput module. This allows to read also gzip/bzip2
Pascal Volk <user@localhost.localdomain.org>
parents: 7
diff changeset
    96
def check_files(log_files):
7f131cf431bc Switched to fileinput module. This allows to read also gzip/bzip2
Pascal Volk <user@localhost.localdomain.org>
parents: 7
diff changeset
    97
    """Checks that all files from *log_files* exist and all of them are
7f131cf431bc Switched to fileinput module. This allows to read also gzip/bzip2
Pascal Volk <user@localhost.localdomain.org>
parents: 7
diff changeset
    98
    readable.
7f131cf431bc Switched to fileinput module. This allows to read also gzip/bzip2
Pascal Volk <user@localhost.localdomain.org>
parents: 7
diff changeset
    99
7f131cf431bc Switched to fileinput module. This allows to read also gzip/bzip2
Pascal Volk <user@localhost.localdomain.org>
parents: 7
diff changeset
   100
    If a file doesn't exist or is not readable, it will be removed from
7f131cf431bc Switched to fileinput module. This allows to read also gzip/bzip2
Pascal Volk <user@localhost.localdomain.org>
parents: 7
diff changeset
   101
    *log_files* set.
7f131cf431bc Switched to fileinput module. This allows to read also gzip/bzip2
Pascal Volk <user@localhost.localdomain.org>
parents: 7
diff changeset
   102
7f131cf431bc Switched to fileinput module. This allows to read also gzip/bzip2
Pascal Volk <user@localhost.localdomain.org>
parents: 7
diff changeset
   103
    This function will return *True*, if at least one file has passed the
7f131cf431bc Switched to fileinput module. This allows to read also gzip/bzip2
Pascal Volk <user@localhost.localdomain.org>
parents: 7
diff changeset
   104
    checks. Otherwise *False* will be returned. And the *log_files* set
7f131cf431bc Switched to fileinput module. This allows to read also gzip/bzip2
Pascal Volk <user@localhost.localdomain.org>
parents: 7
diff changeset
   105
    will be emptied.
7f131cf431bc Switched to fileinput module. This allows to read also gzip/bzip2
Pascal Volk <user@localhost.localdomain.org>
parents: 7
diff changeset
   106
7f131cf431bc Switched to fileinput module. This allows to read also gzip/bzip2
Pascal Volk <user@localhost.localdomain.org>
parents: 7
diff changeset
   107
    :param log_files: set of file names
7f131cf431bc Switched to fileinput module. This allows to read also gzip/bzip2
Pascal Volk <user@localhost.localdomain.org>
parents: 7
diff changeset
   108
    :type log_files: set
7f131cf431bc Switched to fileinput module. This allows to read also gzip/bzip2
Pascal Volk <user@localhost.localdomain.org>
parents: 7
diff changeset
   109
    :rtype: bool"""
7f131cf431bc Switched to fileinput module. This allows to read also gzip/bzip2
Pascal Volk <user@localhost.localdomain.org>
parents: 7
diff changeset
   110
    assert isinstance(log_files, set), 'log_files argument must be a set'
7f131cf431bc Switched to fileinput module. This allows to read also gzip/bzip2
Pascal Volk <user@localhost.localdomain.org>
parents: 7
diff changeset
   111
    lf_copy = log_files.copy()
7f131cf431bc Switched to fileinput module. This allows to read also gzip/bzip2
Pascal Volk <user@localhost.localdomain.org>
parents: 7
diff changeset
   112
    for lf in lf_copy:
7f131cf431bc Switched to fileinput module. This allows to read also gzip/bzip2
Pascal Volk <user@localhost.localdomain.org>
parents: 7
diff changeset
   113
        if not os.path.isfile(lf):
7f131cf431bc Switched to fileinput module. This allows to read also gzip/bzip2
Pascal Volk <user@localhost.localdomain.org>
parents: 7
diff changeset
   114
            os.sys.stderr.write('Warning: No such file: %r\n' % lf)
7f131cf431bc Switched to fileinput module. This allows to read also gzip/bzip2
Pascal Volk <user@localhost.localdomain.org>
parents: 7
diff changeset
   115
            log_files.remove(lf)
7f131cf431bc Switched to fileinput module. This allows to read also gzip/bzip2
Pascal Volk <user@localhost.localdomain.org>
parents: 7
diff changeset
   116
        elif not os.access(lf, os.R_OK):
7f131cf431bc Switched to fileinput module. This allows to read also gzip/bzip2
Pascal Volk <user@localhost.localdomain.org>
parents: 7
diff changeset
   117
            os.sys.stderr.write('Warning: Cannot read file: %r\n' % lf)
7f131cf431bc Switched to fileinput module. This allows to read also gzip/bzip2
Pascal Volk <user@localhost.localdomain.org>
parents: 7
diff changeset
   118
            log_files.remove(lf)
7f131cf431bc Switched to fileinput module. This allows to read also gzip/bzip2
Pascal Volk <user@localhost.localdomain.org>
parents: 7
diff changeset
   119
7f131cf431bc Switched to fileinput module. This allows to read also gzip/bzip2
Pascal Volk <user@localhost.localdomain.org>
parents: 7
diff changeset
   120
    if log_files:
7f131cf431bc Switched to fileinput module. This allows to read also gzip/bzip2
Pascal Volk <user@localhost.localdomain.org>
parents: 7
diff changeset
   121
        return True
7f131cf431bc Switched to fileinput module. This allows to read also gzip/bzip2
Pascal Volk <user@localhost.localdomain.org>
parents: 7
diff changeset
   122
    return False
0
2d97e75f16cf initial commit: »don't fear the nervous delete finger«
Pascal Volk <user@localhost.localdomain.org>
parents:
diff changeset
   123
1
7d5cee19c20a Renamed class NiXSpamPlot to NiXSapmSum. Added getDomLen(). Rewrote showResult()
Pascal Volk <user@localhost.localdomain.org>
parents: 0
diff changeset
   124
2
a72ea07394cc NiXSapmSum.getTotal(): deleted
Pascal Volk <user@localhost.localdomain.org>
parents: 1
diff changeset
   125
def buildTable(output, domains, percent, orderBy):
a72ea07394cc NiXSapmSum.getTotal(): deleted
Pascal Volk <user@localhost.localdomain.org>
parents: 1
diff changeset
   126
    k = 0 if orderBy == 'name' else 1
11
a5f5a8f288da PEP-8-ified the code. Version 0.1.3
Pascal Volk <user@localhost.localdomain.org>
parents: 10
diff changeset
   127
    doms = sorted(domains.items(), lambda d, c: cmp(d[k], c[k]), reverse=k)
7
83c4798e291d Small code cleanups. Added current year to the copyright notices.
Pascal Volk <user@localhost.localdomain.org>
parents: 6
diff changeset
   128
    dlen = len(max(domains.iterkeys(), key=len)) + 1
2
a72ea07394cc NiXSapmSum.getTotal(): deleted
Pascal Volk <user@localhost.localdomain.org>
parents: 1
diff changeset
   129
    clen = len(str(max(domains.values())))
a72ea07394cc NiXSapmSum.getTotal(): deleted
Pascal Volk <user@localhost.localdomain.org>
parents: 1
diff changeset
   130
    total = sum(domains.values())
a72ea07394cc NiXSapmSum.getTotal(): deleted
Pascal Volk <user@localhost.localdomain.org>
parents: 1
diff changeset
   131
    if percent:
3
6b0d09cdfbdb buildTable(): small cosmetics/enhancements
Pascal Volk <user@localhost.localdomain.org>
parents: 2
diff changeset
   132
        format = ' %%%ds  %%%dd  %%6.2f %%%%\n' % (-dlen, clen)
2
a72ea07394cc NiXSapmSum.getTotal(): deleted
Pascal Volk <user@localhost.localdomain.org>
parents: 1
diff changeset
   133
        for d, c in doms:
11
a5f5a8f288da PEP-8-ified the code. Version 0.1.3
Pascal Volk <user@localhost.localdomain.org>
parents: 10
diff changeset
   134
            dfrac = 100. / total * c
2
a72ea07394cc NiXSapmSum.getTotal(): deleted
Pascal Volk <user@localhost.localdomain.org>
parents: 1
diff changeset
   135
            output.write(format % (d, c, dfrac))
11
a5f5a8f288da PEP-8-ified the code. Version 0.1.3
Pascal Volk <user@localhost.localdomain.org>
parents: 10
diff changeset
   136
        output.write('%s\n' % ((clen + dlen + 14) * '-'))
3
6b0d09cdfbdb buildTable(): small cosmetics/enhancements
Pascal Volk <user@localhost.localdomain.org>
parents: 2
diff changeset
   137
        output.write(format % ('total', total, 100))
2
a72ea07394cc NiXSapmSum.getTotal(): deleted
Pascal Volk <user@localhost.localdomain.org>
parents: 1
diff changeset
   138
    else:
3
6b0d09cdfbdb buildTable(): small cosmetics/enhancements
Pascal Volk <user@localhost.localdomain.org>
parents: 2
diff changeset
   139
        format = ' %%%ds  %%%dd\n' % (-dlen, clen)
2
a72ea07394cc NiXSapmSum.getTotal(): deleted
Pascal Volk <user@localhost.localdomain.org>
parents: 1
diff changeset
   140
        for d in doms:
a72ea07394cc NiXSapmSum.getTotal(): deleted
Pascal Volk <user@localhost.localdomain.org>
parents: 1
diff changeset
   141
            output.write(format % d)
11
a5f5a8f288da PEP-8-ified the code. Version 0.1.3
Pascal Volk <user@localhost.localdomain.org>
parents: 10
diff changeset
   142
        output.write('%s\n' % ((clen + dlen + 4) * '-'))
3
6b0d09cdfbdb buildTable(): small cosmetics/enhancements
Pascal Volk <user@localhost.localdomain.org>
parents: 2
diff changeset
   143
        output.write(format % ('total', total))
2
a72ea07394cc NiXSapmSum.getTotal(): deleted
Pascal Volk <user@localhost.localdomain.org>
parents: 1
diff changeset
   144
11
a5f5a8f288da PEP-8-ified the code. Version 0.1.3
Pascal Volk <user@localhost.localdomain.org>
parents: 10
diff changeset
   145
1
7d5cee19c20a Renamed class NiXSpamPlot to NiXSapmSum. Added getDomLen(). Rewrote showResult()
Pascal Volk <user@localhost.localdomain.org>
parents: 0
diff changeset
   146
def showResult(nixspamsum, options):
0
2d97e75f16cf initial commit: »don't fear the nervous delete finger«
Pascal Volk <user@localhost.localdomain.org>
parents:
diff changeset
   147
    if options.countByDom:
1
7d5cee19c20a Renamed class NiXSpamPlot to NiXSapmSum. Added getDomLen(). Rewrote showResult()
Pascal Volk <user@localhost.localdomain.org>
parents: 0
diff changeset
   148
        nixspamsum.countByDom()
7d5cee19c20a Renamed class NiXSpamPlot to NiXSapmSum. Added getDomLen(). Rewrote showResult()
Pascal Volk <user@localhost.localdomain.org>
parents: 0
diff changeset
   149
        domains = nixspamsum.getDomains()
0
2d97e75f16cf initial commit: »don't fear the nervous delete finger«
Pascal Volk <user@localhost.localdomain.org>
parents:
diff changeset
   150
    else:
1
7d5cee19c20a Renamed class NiXSpamPlot to NiXSapmSum. Added getDomLen(). Rewrote showResult()
Pascal Volk <user@localhost.localdomain.org>
parents: 0
diff changeset
   151
        domains = nixspamsum.getMXs()
7
83c4798e291d Small code cleanups. Added current year to the copyright notices.
Pascal Volk <user@localhost.localdomain.org>
parents: 6
diff changeset
   152
    if not domains:
6
723b4c4e15cb Set/updated descriptions and error messages.
Pascal Volk <user@localhost.localdomain.org>
parents: 5
diff changeset
   153
        print "No NiX Spam DNSBL rejects found"
4
ae58d9881be5 showResult(): count matches; break if no matches were found
Pascal Volk <user@localhost.localdomain.org>
parents: 3
diff changeset
   154
        return
ae58d9881be5 showResult(): count matches; break if no matches were found
Pascal Volk <user@localhost.localdomain.org>
parents: 3
diff changeset
   155
ae58d9881be5 showResult(): count matches; break if no matches were found
Pascal Volk <user@localhost.localdomain.org>
parents: 3
diff changeset
   156
    from cStringIO import StringIO
ae58d9881be5 showResult(): count matches; break if no matches were found
Pascal Volk <user@localhost.localdomain.org>
parents: 3
diff changeset
   157
    output = StringIO()
2
a72ea07394cc NiXSapmSum.getTotal(): deleted
Pascal Volk <user@localhost.localdomain.org>
parents: 1
diff changeset
   158
    # build the table
1
7d5cee19c20a Renamed class NiXSpamPlot to NiXSapmSum. Added getDomLen(). Rewrote showResult()
Pascal Volk <user@localhost.localdomain.org>
parents: 0
diff changeset
   159
    if options.oFormat == 'table':
2
a72ea07394cc NiXSapmSum.getTotal(): deleted
Pascal Volk <user@localhost.localdomain.org>
parents: 1
diff changeset
   160
        buildTable(output, domains, options.percent, options.order)
a72ea07394cc NiXSapmSum.getTotal(): deleted
Pascal Volk <user@localhost.localdomain.org>
parents: 1
diff changeset
   161
    # generate comma separated values
1
7d5cee19c20a Renamed class NiXSpamPlot to NiXSapmSum. Added getDomLen(). Rewrote showResult()
Pascal Volk <user@localhost.localdomain.org>
parents: 0
diff changeset
   162
    elif options.oFormat == 'csv':
4
ae58d9881be5 showResult(): count matches; break if no matches were found
Pascal Volk <user@localhost.localdomain.org>
parents: 3
diff changeset
   163
        order = domains.keys()
ae58d9881be5 showResult(): count matches; break if no matches were found
Pascal Volk <user@localhost.localdomain.org>
parents: 3
diff changeset
   164
        order.sort()
ae58d9881be5 showResult(): count matches; break if no matches were found
Pascal Volk <user@localhost.localdomain.org>
parents: 3
diff changeset
   165
        for d in order:
ae58d9881be5 showResult(): count matches; break if no matches were found
Pascal Volk <user@localhost.localdomain.org>
parents: 3
diff changeset
   166
            output.write("'%s',%d\n" % (d, domains[d]))
2
a72ea07394cc NiXSapmSum.getTotal(): deleted
Pascal Volk <user@localhost.localdomain.org>
parents: 1
diff changeset
   167
    # should never be reached
1
7d5cee19c20a Renamed class NiXSpamPlot to NiXSapmSum. Added getDomLen(). Rewrote showResult()
Pascal Volk <user@localhost.localdomain.org>
parents: 0
diff changeset
   168
    else:
2
a72ea07394cc NiXSapmSum.getTotal(): deleted
Pascal Volk <user@localhost.localdomain.org>
parents: 1
diff changeset
   169
        print "Oops, error in function showResult() happend"
a72ea07394cc NiXSapmSum.getTotal(): deleted
Pascal Volk <user@localhost.localdomain.org>
parents: 1
diff changeset
   170
    # show the result
a72ea07394cc NiXSapmSum.getTotal(): deleted
Pascal Volk <user@localhost.localdomain.org>
parents: 1
diff changeset
   171
    print output.getvalue()
0
2d97e75f16cf initial commit: »don't fear the nervous delete finger«
Pascal Volk <user@localhost.localdomain.org>
parents:
diff changeset
   172
11
a5f5a8f288da PEP-8-ified the code. Version 0.1.3
Pascal Volk <user@localhost.localdomain.org>
parents: 10
diff changeset
   173
0
2d97e75f16cf initial commit: »don't fear the nervous delete finger«
Pascal Volk <user@localhost.localdomain.org>
parents:
diff changeset
   174
def main():
2d97e75f16cf initial commit: »don't fear the nervous delete finger«
Pascal Volk <user@localhost.localdomain.org>
parents:
diff changeset
   175
    parser = getOptionParser()
2d97e75f16cf initial commit: »don't fear the nervous delete finger«
Pascal Volk <user@localhost.localdomain.org>
parents:
diff changeset
   176
    opts, args = parser.parse_args()
8
7f131cf431bc Switched to fileinput module. This allows to read also gzip/bzip2
Pascal Volk <user@localhost.localdomain.org>
parents: 7
diff changeset
   177
    if not args:
6
723b4c4e15cb Set/updated descriptions and error messages.
Pascal Volk <user@localhost.localdomain.org>
parents: 5
diff changeset
   178
        parser.error('No log file specified')
8
7f131cf431bc Switched to fileinput module. This allows to read also gzip/bzip2
Pascal Volk <user@localhost.localdomain.org>
parents: 7
diff changeset
   179
    # drop duplicates
7f131cf431bc Switched to fileinput module. This allows to read also gzip/bzip2
Pascal Volk <user@localhost.localdomain.org>
parents: 7
diff changeset
   180
    log_files = set(args)
7f131cf431bc Switched to fileinput module. This allows to read also gzip/bzip2
Pascal Volk <user@localhost.localdomain.org>
parents: 7
diff changeset
   181
    # remove inexistent/unreadable files
7f131cf431bc Switched to fileinput module. This allows to read also gzip/bzip2
Pascal Volk <user@localhost.localdomain.org>
parents: 7
diff changeset
   182
    if not check_files(log_files):
7f131cf431bc Switched to fileinput module. This allows to read also gzip/bzip2
Pascal Volk <user@localhost.localdomain.org>
parents: 7
diff changeset
   183
        os.sys.stderr.write('No readable log files found\n')
7f131cf431bc Switched to fileinput module. This allows to read also gzip/bzip2
Pascal Volk <user@localhost.localdomain.org>
parents: 7
diff changeset
   184
        return 1
7f131cf431bc Switched to fileinput module. This allows to read also gzip/bzip2
Pascal Volk <user@localhost.localdomain.org>
parents: 7
diff changeset
   185
1
7d5cee19c20a Renamed class NiXSpamPlot to NiXSapmSum. Added getDomLen(). Rewrote showResult()
Pascal Volk <user@localhost.localdomain.org>
parents: 0
diff changeset
   186
    nixss = NiXSapmSum()
7d5cee19c20a Renamed class NiXSpamPlot to NiXSapmSum. Added getDomLen(). Rewrote showResult()
Pascal Volk <user@localhost.localdomain.org>
parents: 0
diff changeset
   187
    nixss.setLogFormat(opts.format)
8
7f131cf431bc Switched to fileinput module. This allows to read also gzip/bzip2
Pascal Volk <user@localhost.localdomain.org>
parents: 7
diff changeset
   188
7f131cf431bc Switched to fileinput module. This allows to read also gzip/bzip2
Pascal Volk <user@localhost.localdomain.org>
parents: 7
diff changeset
   189
    fi = fileinput.FileInput(log_files, openhook=fileinput.hook_compressed)
7f131cf431bc Switched to fileinput module. This allows to read also gzip/bzip2
Pascal Volk <user@localhost.localdomain.org>
parents: 7
diff changeset
   190
    nixss.parseLog(fi)
7f131cf431bc Switched to fileinput module. This allows to read also gzip/bzip2
Pascal Volk <user@localhost.localdomain.org>
parents: 7
diff changeset
   191
    fi.close()
1
7d5cee19c20a Renamed class NiXSpamPlot to NiXSapmSum. Added getDomLen(). Rewrote showResult()
Pascal Volk <user@localhost.localdomain.org>
parents: 0
diff changeset
   192
    showResult(nixss, opts)
8
7f131cf431bc Switched to fileinput module. This allows to read also gzip/bzip2
Pascal Volk <user@localhost.localdomain.org>
parents: 7
diff changeset
   193
    return 0
0
2d97e75f16cf initial commit: »don't fear the nervous delete finger«
Pascal Volk <user@localhost.localdomain.org>
parents:
diff changeset
   194
11
a5f5a8f288da PEP-8-ified the code. Version 0.1.3
Pascal Volk <user@localhost.localdomain.org>
parents: 10
diff changeset
   195
0
2d97e75f16cf initial commit: »don't fear the nervous delete finger«
Pascal Volk <user@localhost.localdomain.org>
parents:
diff changeset
   196
if __name__ == '__main__':
8
7f131cf431bc Switched to fileinput module. This allows to read also gzip/bzip2
Pascal Volk <user@localhost.localdomain.org>
parents: 7
diff changeset
   197
    os.sys.exit(main())