merged changes from default(29295afafa91)

VMM/account: Fixed a typo in Account.delete(). Due to a wrong placed comma `userdelete … force' ended in a TypeError.

VMM/mailbox: Decode error messages from doveadm. Those errors occur only with the dbox mailbox format when the mailbox.folders contains non-ASCII mailbox names.

VMM/cli/config: … encode/decode …

VMM{/cli}/config: Explicitly pass utf-8 as the encoding to open().

VMM/cli/subcommands: Fixed some encode/decode issues.

VMM/cli/subcommands: Print usable password schemes in sorted order.

VMM/password: Code cleanups: - Rely no longer on Crypto.Hash for md4 support. Python's hashlib on FreeBSD, Linux and OpenBSD provides the md4 hash. - Dropped DIGEST-MD5 password hashing for Dovecot <= v1.1.12/v1.2.beta2

VMM/cli: Avoid UnicodeDecodeErrors with some „obsolete“ encodings.

VMM: Post-2to3 fixes. Added, (re)moved some (en|de)code() calls.

VMM: Post-2to3 fix. Don't pass 'str=1' ('unicode=1' before 2to3) to gettext.install().

VMM/password: Post-2to3 fixes. Replaced ….encode('hex') and ….encode('base64') by b64encode(…) and b2a_hex(…). Added some necessary encode() and decode() calls. Fixed NTLM hash generating with non ASCII characters.

VMM/config: s/_boolean_states/BOOLEAN_STATES/g

VMM/handler: Dropped psycopg2 typecaster registration.

VMM/handler: Reworked config file permission check.

merged changes from default(86c2ed36aba8)

po: Updated Finnish translation (translated by Jorma Karvonen).

Updated shebang to use python3 instead of python.

pgsql/set-permissions.py: Ran 2to3 from Python 3.2.3.

Ran 2to3 from Python 3.2.3.

Use the with statement for file operations.

VMM/config: Undone changeset 90d69ae4f40d.

VMM/cli/subcommands: Code cleanup in list_pwschemes().

setup.py: PEP-314-ified. Distutils setup() function now has the 'requires` keyword parameter.

VMM: Partial PEP-308-ification.

VMM: Removed module pycompat.

VMM/pycompat: Removed module hashlib.

doc/web: Updated OpenBSD's package names.

doc/web: Fixed a typo.

VMM/password: Added scheme SCRAM-SHA-1.

Documentation: Fixed dictionary quota URI.

install.sh: Don't fail if $SYSCONFDIR doesn't exist.

doc/web/ext_references: Fixed URL of pg_hba.conf.

doc/web: Added links to the Debian package and the git repos. README: Added git-URL.

Added tag vmm-0.6.1 for changeset 682431c45b24

Released vmm-0.6.1

doc/web: Updated documentation for vmm-0.6.1.

Add list* commands to manpages

UPGRADE: Added procedure for updating from v0.6.0.

update_config: Updated to work with version 0.6.0. Fixed some PEP8 related issues.

doc/web: Updated and extended documentation. Updated description of subcommands aliasdelete and catchalldelete. Added description of subcommands listaddresses, listaliases, listrelocated and listusers.

VMM/cli/clihelp: Added missing help messages. Added descriptions of subcommands listaddresses, listaliases, listrelocated and listusers.

po: Updated Dutch translation (translated by Erwin Poeze).

VMM/alias: Load destination addresses sorted. This produces sorted output for the aliasinfo subcommand.

VMM: {alias,catchall}delete: Accept multiple destinations.

po: Updated Dutch translation (translated by Erwin Poeze).

VMM/cli/main: Check if we have warnings before exiting. If so, print them and exit afterwards.

VMM/domain: Enabled transport-validation. Validate the transport also in Domain.set_transport() and Domain.update_transport().

VMM: Moved transport-validation to common.

VMM/account: Fixed account's transport check. Make sure that 'virtual:' isn't used as transport when the mailbox-format is set to mdbox or sdbox.

po: Updated Dutch translation (translated by Erwin Poeze).

PEP8: Fixed all PEP8 related issues.

po: Regenerated vmm.pot

Added a few comments for the Translation Project.

VMM: Corrected/updated some msgids. Unified some messages and corrected a few typos.

VMM/cli/subcommands: Corrected username's usage string. username's name argument is optional.

VMM/cli: Activated help subcommand. Implemented Command.help_(), which reads the command's help message from cli.clihelp.

VMM/handler: Fixed user{quota,services,transport} subcommands. The subcommands user{quota,services,transport} actually failed with the arguments '<address> domain'. This is part II of changeset a75923ce2842

doc/web: Added missing userservices description. Fixed order of subcommand listing.

man: Typo-fix.

VMM/cli/subcommands: Corrected two usage strings. domainnote's and usernote's note argument is optional.

VMM: Decode localized numbers to the current encoding. That avoids UnicodeDecodeErrors since locale.format(%d) may return localized numbers, which contain NO-BREAK SPACE as the thousands separator.

VMM/cli/subcommands: Don't mark empty strings as translatable. gettext('') returns the header entry with meta information, not the empty string.

doc/web: Added description 'Upgrade to Dovecot ≧ v1.2.beta2'.

pgsql: Added dovecot_update_v1.2+.pgsql. Use this file when you are upgrading your Dovecot installation to a version >= 1.2.beta2.

VMM/cli/subcommands: Reverted accidentally committed help() code.

pgsql/create_tables.pgsql: Eliminated two unnecessary differences. Yeah, cosmetic only.

pgsql: Don't select service_set.ssid in dovecotpassword() unconditional. This may cause an empty result, which will make logins impossible.

VMM/serviceset: Fixed a conditional ProgrammingError. Affects only setups with Dovecot < v1.2.beta2. Actually the SQL query in method ServiceSet._load_by_services wasn't updated. Removed unnecessary code, which would have also produced a ProgrammingError, if it would have been correct.

VMM/cli/subcommands: Added 'missing' newline below domain notes. This patch contains also a small code cleanup and PEP8-fix.

doc/web: Added the transport filed to the usertransport example.

doc/web: Added the 'Howto'. The reStructuredText version of vmm.1, in multiple parts.

man/{de/}man1: A few more corrections.

man/{de/}man1: Small corrections for domaininfo's description.

man/{de/}man5: s/Wiki/Homepage/g

man/{de/}man1: s/Wiki/Homepage/g

VMM/common: Improve search_address complexity Checking the dictionary (a hash) for existence of a key is likely to be O(log(n)), while checking a list is O(n). Therefore, to increase performance, this patch changes the check accordingly.

doc/web: Refreshed vmm.cfg

vmm.cfg: Updated/improved comments.

If an alias has multiple destinations, multiple records exist, due to the nature of the database. address_list would then return the same alias multiple times, which does not add any information, eats screen space and is potentially confusing. Therefore, we SELECT DISTINCTly from the alias table. Signed-off-by: martin f. krafft <madduck@debian.org> --- VirtualMailManager/common.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)

Due to the UNION query in address_list, the assumption that the list of gids received from the database would be continuous does not hold. To prevent addresses for domains with multiple entry types from being listed, it is necessary to check the list of domain gids for every entry. Signed-off-by: martin f. krafft <madduck@debian.org> --- VirtualMailManager/common.py | 4 +--- 1 file changed, 1 insertion(+), 3 deletions(-)

doc/web: Emphasized a few lines of Dovecot/PostgreSQL configuration file excerpts.

.hgignore: "doc/build" is no longer required.

doc: Added source of http://vmm.localdomain.org/.

doc: Moved API documentation to doc/api.

INSTALL: Updated PostgreSQL configuration URL. Use ${sql} and ${proxysql} expressions in the main.cf description.

README: Substituted a few words.

README: Updated URL of the hg repository@sf.net.

man: Deleted erroneous placed backslash.

Added tag vmm-0.6.0 for changeset 3238c58d01ae

Released vmm-0.6.0

Merged changes from v0.6.x(28230a8230bf).

Close v0.6.x branch.

vmm: Don't insert the module directory into sys.path by default. Doesn't make much sense, if the code is installed inside sys.path.

Updated copyright notices to include the year 2012.

TODO: Removed "listpwschemes" entry. Was done with changeset a64c1b5e08b4

UPGRADE: Updated documentation.

VMM/cli/subcommands: Replaced the keyword `default' by `domain'. Patch by Martin F. Krafft

INSTALL: Added description how to create additional dove* user/group. `doveauth' user and group for the authentication process and the group `dovemail' for mail related Dovecot processes.

INSTALL: Updated documentation.

Configure.Dovecot_2: Added configuration documentation for Dovecot_2 >= 2.0.

VMM: The Python 2.4.x compatibility commit - part III.

VMM/account: Check the account's transport only when we have a transport.

VMM: The Python 2.4.x compatibility commit.

pgsql/set-permissions.py: Extended permissions. Postfix needs read access on the postfix_gid view. Dovecot needs read permissions on both mailboxformat columns.

VMM/handler: Restored method Handler.domain_transport(). It went somehow lost with changeset 5806fb74130b.

pgsql: Added helper script: set-permissions.py.

Provide list{addresses,aliases,users,relocated} subcommands The patch provides the list{addresses,aliases,users,relocated} subcommands to the UI. All commands actually take the same path in the code and only one query is run with different parameters for each case. There are still two shortcomings: 1. With alias domains, the output order is not as one might want it, e.g. foo@example.org bar@example.org foo@example.com bar@example.com when it should really be foo@ twice and then bar@ twice. I have not found a way to modify the SQL accordingly. 2. The SELECT queries for Accounts, Alias and Relocated are hard-coded in common.py.

man: escape hyphens Hyphens inside words must be spelt \- instead of just -. This patch fixes up the manpages. * * * .

man: add 'catchall' to domaininfo Since the addition of catchall, the domaininfo subcommand can also take 'catchall' as a [detail] to limit the output of details to the catch-all aliases. Also, the number of catch-all destinations is now included in the simple domaininfo output.

man: vmm1: Use `fqdn' instead of `domain' in catchall* descriptions. Fixed a typo s/cadd/cad/. Replaced double quotes by \(dq.

VMM/cli/subcommands: Use the `fqdn' argument placeholder also for catch-all subcommands.

Fix transport_maps function for non-existent domains The postfix_transport_maps function had a bug causing 2012-04-15 17:40:22 CEST LOG: statement: SELECT transport FROM postfix_transport_map('logcheck', 'domine.madduck.net'); 2012-04-15 17:40:22 CEST ERROR: query returned no rows when the domain was not in the database. This would make did be NULL and make the query fail. This patch moves the tid query until after a check for did. If the latter is NULL, the function RETURNs (rather than fails).

Revert caching of destination interpolation Since 'destination' comes from the table in the query, it cannot be cached across queries! Doh!

pgsql: update_tables_*: Added missing `NOT' to `… tid DROP NOT NULL;'.

pgsql: Create column `note' as text.

Add docstring for common.format_domain_default

Document domainnote in manpages 5806fb74130b did not contain this, so I must have forgotten it. Oops.

Remove domain/account notes from TODO

Include account note in getuser output Modify the getuser output to include an account note, if one is present

Display formatted notes in info output If the user/domain object has an attached note, this patch makes the userinfo/domaininfo output print the note, wrapped to the terminal size.

Ignore *.egg-status build directory created by setuptools

Add user/domainnote commands to manpages

Add note field to Account/Domain and CLI

Modify SQL to add note field to domain_data and users

man: vmm.1: subcommand's name argument is optional now. Based on martin f. krafft's v2 of changeset c340ba4bd544.

Allow username to take no argument to erase value Once a username has been set, it could not be removed via the UI. Now, if no argument is passed to username, the value stored is replaced with NULL.

Add inheritance as a feature to README

Change 'force' argument to reset user records for tid/ssid/qid If domain{quotalimit,transport,serviceset} are run with 'force' and settings are inheritable from the domain, then rather than to copy the ID to the user table, the affected field in the user records should be set to NULL so that the domain default is used.

Change UI to accept 'default' to restore inherited values Instead of explicit values for tid/ssid/qid, the UI now accepts 'default', which removed user-pecific settings and hence causes domain defaults to be used.

Modify userinfo output to indicate when domain defaults are displayed When Account instances reference NULL tid/qid/ssid, the data must come from the associated domain, and this should be indicated. For transport and services, this is easy to do as the string passed in the info dict can simply be modified. For quotalimit, however, another method must be used due to the CLI-side formatting. All approaches use a common formatter outsourced to the common.py file.

Modify Account class to handle NULL references This patch modifies the Account class to defer to using the associated domain's tid/ssid/qid fields if the per-instance fields are None/NULL.

Modify SQL update script to accept NULL fields for users The value NULL in the qid/ssid/tid fields of the user table means that the value from the associated domain record should be used instead. This patch modifies the PL/pgSQL functions used by Dovecot and Postfix accordingly.

man: Add examples for userservices

Reverted previous three changesets (e09139525580, 85517c8fde36, 3acbff727626)

Modify userinfo output to indicate when domain defaults are displayed When Account instances reference NULL tid/qid/ssid, the data must come from the associated domain, and this should be indicated. For transport and services, this is easy to do as the string passed in the info dict can simply be modified. For quotalimit, however, another method must be used due to the CLI-side formatting. All approaches use a common formatter outsourced to the common.py file.

Modify Account class to handle NULL references This patch modifies the Account class to defer to using the associated domain's tid/ssid/qid fields if the per-instance fields are None/NULL.

Modify SQL update script to accept NULL fields for users The value NULL in the qid/ssid/tid fields of the user table means that the value from the associated domain record should be used instead. This patch modifies the PL/pgSQL functions used by Dovecot and Postfix accordingly.

Add list* subcommands to TODO

add domain/account notes to TODO

Add my WIP items to the TODO list

Cache interpolated destination Instead of running the string modification function for destination interpolation twice, store the result once and use the stored result subsequently.

man: Use example.com in catchallinfo example output.

README: reStructuredText fix: ERROR/3 and WARNING/2.

Enable interpolation of alias destinations This patch modifies the virtual_alias_maps function so that the destination address is parsed for the place holders %n, %d and %=, which are replaced with the localpart, the domain or the full address with '@' replaced by '=' of the queried key. In combination with alias domains, this allows for domain-specific recipients. E.g. given example.org and its alias domain example.com, defining an alias postmaster@example.org → postmaster+%d@example.org will cause mail to postmaster@example.*com* to go to postmaster+example.*com*@example.org.

Add catch-all aliases to README/features

Add catchall subcommands to manpages

Include catch-all count in domaininfo output

Make PL/pgSQL function feed back identity for mailboxes/relocated when there are catchall destinations. Without catchall aliases, if no virtual_alias matches, the query can just return NULL and Postfix will later check mailboxes/relocated for the address to rewrite. However, since virtual aliases are handled long before mailboxes/relocated, a catchall alias would also catch mail to mailboxes and relocated addresses, which we do not want. The way to tell postfix to keep delivering is for the virtual alias map to return the search key itself (identity function). This patch changes the postfix_virtual_alias_maps Pl/pgSQL function to do exactly that, but only if there are catchall destinations defined for the domain in question — otherwise it returns NULL when no match is found.

Fix syntax errors

Modify virtual_alias_maps function to check mailboxes/relocated first

Make virtual_alias_maps function search catchall when no aliases are found

Add catchall info to domain_info output

Correct syntax errors

Fixup catchall_info to return iterator, not self

Correct syntax errors

Teach CLI about catchall* commands

Provide catchall_* methods to the Handler class

First version of a CatchallAlias class I based the CatchallAlias class in catchall.py heavily on the Alias class, but by copy, not deriving. The two are functionally related, but the implementations are too different because CatchallAliases have no localpart.

Add database table for catchall destinations Catch-all aliases are stored for a domain. Since there can be multiple destinations per domain, this is done using a 1:n relation on the gid. This commit extends the SQL schema definition and also provides appropriate upgrade DDL.

pgsql: Merged types and functions into the create_tables… files.

postfix: Removed VIEW-based comments from configuration files.

pgsql: Removed most of the VIEWs. There are only two views left for now: - postfix_gid - vmm_domain_info All other views have a faster replacement: plpgsql functions.

Ignore even more Debian-build-related files

Ignore more Debian-build-related files Debian produces log and debhelper files in debian/* that do not need to be known to mercurial. Hence, this patch adds the globs to .hgignore.

Add non-root TODO

VMM/config: Fixed a typo: s/DB_MUDULES/DB_MODULES/g.

Ignore build files related to Debian packaging During Debian packaging, the package is built in ./debian/vmm/, and quilt is used (./.pc/). Both directories contain only generated files and should hence be ignored from the perspective of Mercurial.

Instruct mercurial to ignore build directory Python's distutil builds the package into ./build/, which can and should be ignored with respect to the repository. This patch simply adds the appropriate line to .hgignore.

VMM/handler: Create domain directory with umask 0022.

Modify address check query to obtain well-defined result The way in which UNION does not yield the desired result, because (a) UNION merges results and (b) the result order is undefined. This patch changes the query to select the counts as columns and hence provides a well-defined order.

postfix: Uncommented the query in pgsql-smtpd_sender_login_maps.cf.

postfix: Renamed pgsql-transport.cf to pgsql-transport_maps.cf. Now it matches pgsql-${postfix-parameter}.cf.

po: Updated French translation (translated by Dimitri Duc).

po: Updated Finnish translation (translated by Jorma Karvonen).

VMM/cli/subcommands: Fixed a TypeError in user_info().

postfix: Use pgsql functions in query templates where possible.

pgsql: Updated required privileges for function dovecotpassword.

README: Added section `Installation Prerequisites'. Reworked some parts.

update_config: Write the old package install dir to a tmp file. upgrade: Remove old CamelCase files. Removed some unused variables.

VMM: The pyPgSQL compatibility commit.

VMM: The Python 2.4.x compatibility commit.

VMM/relocated: Don't accept something@localhost as destination.

man1: Updated configuration related descriptions/examples, accordingly to commit 4ff0fa3ba0fa.

Updated the author's e-mail address.

man: Updated the URL of the bug tracker.

man5: Updated the `usable password schemes' part.

man1: Copied new formatted/translated command output into the examples.

VMM/cli/subcommands: Small improvement of domaininfo's output. Pointed out that the displayed quota limit is valid per user.

man1: Removed obsolete commands and added new commands.

man/de/man1: Removed translated argument names.

VMM/cli/subcommands: Use the common output formatting in list_pwschemes().

VMM/handler: Added a comment for the Translation Project.

VMM/cli/subcommands: Subcommand argument placeholders are no longer translatable.

po: Updated German translation (translated by Mario Blättermann).

po: Updated address for message-id bug reports. Looks like as sf.net would have horrible configured mail servers. :-/

merged changes from default(c0e1fb1b0145)

po: Added Finnish translation to the repository. Many thanks to Jorma Karvonen from the Translation Project for the work.

VMM/password: list_schemes(): report suffixes with a leading dot.

po: Updated messages template and German translation.

VMM/cli/subcommands: Added new subcommand `listpwschemes' (lp). Marked version()'s ctx argument as unused.

VMM/password: Added function list_schemes().

man5: Moved some option descriptions to section doamin, accordingly to commit 4ff0fa3ba0fa. Added a brief explanation of which settings will be applied in which situations.

update_config: Updated, so that the previous config modifications will be applied to 0.5.2 configurations files.

VMM/config: Moved some options to section `domain': old new ------------------------------------------------------------ account.imap -> domain.imap account.pop3 -> domain.pop3 account.sieve -> domain.sieve account.smtp -> domain.account misc.quota_bytes -> domain.quota_bytes misc.quota_messages -> domain.quota_messages misc.transport -> domain.transport

VMM: Show a warning when a unsupported locale setting was detected.

VMM/cli/subcommands: Small improvement in domain_add(). Inform the admin, when the postmaster account will be auto-created.

VMM/cli/subcommands: Fixed a small failure in alias_info().

TODO: updated …

VMM/domain: Extended Domain._chk_state() to support both states.

VMM/constants: Cleared out obsolete error codes.

VMM/account: Removed unused import.

VMM/{maillocation,quotalimit}: Unified object initialization code. Simply raise a ValueError for wrong values (unknown IDs).

VMM/transport: Reworked Transport initialization. Set the transport id only if we could find a transport with that id. Raise a ValueError instead of a translatable TransportError(VMMError). VMM/errors: Deleted class TransportError, it is no longer needed.

VMM/cli/subcommands: domainservices: Don't fail when neither a service nor the keyword force was provided.

VMM/domain: Extended/updated Domain.get_info() in order to: - include active service(s) || None - use more understandable English terms VMM/cli/subcommands: Updated (order of) keys of `domaininfo' output. .`userinfo' will always show `sieve', no longer `managesieve'.

VMM/cli/subcommands: Small indent fix.

VMM/cli/subcommands: Added subcommand domainservices (ds). Replaced subcommands userdisable and userenable by userservices (us).

VMM/account: Integrated class ServiceSet in class Account. Replaced methods Account.enable() and Account.disable() by Account.update_serviceset(). VMM/handler: Replaced methods Handler.user_disable() and Handler.user_enable() by Handler.user_services()

VMM/serviceset: Added @property ServiceSet.services -> dict.

VMM/account: Sorted imports …

VMM/domain: Integrated class ServiceSet in class Domain. VMM/handler: Extended Handler.domain_add() and added Handler.domain_services(). Removed unused domain.get_gid import.

VMM/serviceset: Use SERVICES inside of ServiceSet - avoid slicing.

VMM/serviceset: Added module constant SERVICES.

VMM: Added new module serviceset - provides class ServiceSet.

pgsql: Added new table `service_set'. Moved columns `smtp', `pop3' `imap' and `sieve' from the `users' table to the new `service_set' table.

update_config: Set database.module to pyPgSQL only when psycopg2 is unavailable.

README: Added contet.

man: Removed reStructuredText files.

man: Reworked/updated German manual pages.

man: Added missing `.RE' macro to sub section userinfo (ui).

po: Fixed 2 fatal msgfmt errors inde.po.

po: Updated German translation.

Messages: Replaced some 'Available' by 'Existing'.

po: Regenerated vmm.pot

VMM/emailaddress: Class DestinationEmailAddress accepts also something@localhost addresses now.

VMM/handler: Handler.alias_delete() also use class DestinationEmailAddress for the destination address, instead of class EmailAddress.

VMM/alias: Alias.del_destination() corrected the value assignment in the error message's dictionary.

VMM/alias: Alias._load_dests() use class DestinationEmailAddress for destination addresses, instead of class EmailAddress.

VMM/cli/subcommands: user_add() show only generated passwords - not the given.

VMM: Deleted a few unused variables.

VMM/network: Added new module network.

VMM/cli/subcommands: Added function update_cmd_map(). So the items will be added to the cmd_map when gettext's _() has been installed -> translatable subcommand descriptions.

po: Regenerated vmm.pot

VMM/cli/subcommands: The keyword 'force' is no longer translatable.

VMM: Updated some messages. (doesn't, isn't) -> (does not, is not) Added missing dots at the end of some sentences.

man: Replaced unnecessary \(aq glyphs in man1/vmm.1. Updated COPYING section in man1/vmm.1 and man5/vmm.cfg.5.

VMM/mailbox: Use the correct mailbox hierarchy separator in Mailbox.add_boxes() for mdbox and sdbox mailbox format.

VMM: Report quota usage/limit/percentage values formatted according to the current LC_ALL setting.

VMM/cli/subcommands: Improved legibility of domaininfo's "quota limit" output a little bit.

man/man5: Reworked/updated vmm.cfg.5. Removed vmm.cfg.5.rst.

man/man1: Small syntax fixes in vmm.1.

man/man1: Reworked/updated vmm.1. Removed vmm.1.rst. reStructuredText was nice to edit but the generated output …

VMM/cli/subcommands: Update also the RunContext's subcommand if 'plan A' in {,alias}domain_info() fails.

VMM/cli/subcommands: Fixed a msgid issue (format string with unnamed arguments).

VMM: Solve the default "misc.quota_bytes = '0'" problem where it may occur.