Changes between Version 17 and Version 18 of AnnouncerPlugin/MessageEncryption


Ignore:
Timestamp:
May 26, 2010, 7:56:11 PM (14 years ago)
Author:
Steffen Hoffmann
Comment:

major rewrite to reflect develoment until now

Legend:

Unmodified
Added
Removed
Modified
  • AnnouncerPlugin/MessageEncryption

    v17 v18  
    1 = Messages encryption =
    2 I'll document the effort to add support for optionally message encryption using GnuPG. See #6773 for the corresponding ticket asking for this enhancement.
    3 [[TOC(inline,heading=page content)]]
     1= Announcer cryptographic capabilities =
     2[[TOC(heading=page content)]]
     3This page documents the effort to add optional encryption for AnnouncerPlugin email messages using GnuPG.[[BR]]
     4See #6773 for the corresponding ticket asking for this enhancement.
    45
    5 == Code structure ==
    6 === Where ===
    7 Where to kick in and mangle the message body is of course an essential decision.
    8 My initial assumption that I could add cryptographically functions right before inserting recipient addresses to the message was wrong.
     6== Why ==
     7Let's see how cryptography could help with Trac and especially with it's announcements.
    98
    10 It might help a lot to look at [t:wiki:TracDev/Announcer internal structure and event flow] of AnnouncerPlugin in detail.
    11 By now I add the following into code of ./announcerplugin_trunk/announcer/distributors/mail.py from trunk of AnnouncerPlugin:
     9Imagine, you're using Trac in a corporate environment, typically allowing external access to Trac, repositories etc. only after authorization or not at all.
     10Still, you may wish to keep business partners, support customers, etc. informed about certain or all developments, and that involves sending potentially sensitive information outside the tightly controlled corporate network. Co-workers in a home office setup may create a similar demand.
    1211
    13 {{{
    14          return msgid
    15  
    16      def _do_send(self, transport, event, format, recipients, formatter):
    17          output = formatter.format(transport, event.realm, format, event)
    18 +
    19 +        email_encrypt = True
    20 +        if email_encrypt:
    21 +            output = encrypt_txt(output)
    22 +            self.log.debug("EmailDistributor successfully encrypted msg.")
    23 +
    24          alternate_style = formatter.alternative_style_for(
    25              transport,
    26              event.realm,
    27 }}}
     12Signing emails will help recipients to be sure, that they got unaltered content and got it from you. Encrypting emails will help to distribute announcements to external recipients as freely as to recipients inside without concerns on discretion.
    2813
    29 === What ===
    30 Here is a description of what shall be done. Experts and GPG/PGP users may wish to skip that section and go to the [#AnnouncerEmailEncryption proposal for trac-specific use] right away.
     14== What ==
     15Here is a description of what shall be done. Experts and GPG/PGP users may wish to skip that section and go to the [#AnnouncerEmailEncryption proposal for Trac-specific use] right away.
    3116
    32 ==== OpenPGP principles ====
     17=== OpenPGP principles ===
    3318FIXME: I'll write here and cite sources for more detailed explanation of OpenPGP standard and cryptography in communication in general.
    3419
    35 ==== !AnnouncerEmailEncryption ====
    36 Now let's look at how OpenPGP could help with Trac and especially with announcements.
     20== How (proposed implementation) ==
     21It might help a lot to have a closer look at [t:wiki:TracDev/Announcer internal structure and event flow] of AnnouncerPlugin. Where to kick in is of course an essential decision. Since it's not only about mangle the message body (one of the earlier assumptions that all turned out to reach not far enough), there is not a single point but a bunch of changes needed to get cryptographic operations working.
     22 
     23There are modifications required and code to be added to `announcer/distributors/mail.py`. A new file `announcer/util/mail_crypto` introduces a new class `CryptoTxt`, that provides the cryptographic operations. The approach is generic enough to be extended beyond OpenPGP, currently the only supported crypto standard and handled by GnuPG, to i.e. use X.509 certificates as well.
    3724
    38 Imagine, you're using trac in a corporate environment, typically allowing external access to trac, repositories etc. only after authorization or not at all.
    39 However you may wish to keep business partners, support customers or even co-workers outside the tightly controlled corporate network informed about certain or all developments. Encrypting mail for external recipients will help to deal with issues as discrete as you like while still using announcements for changes in trac.
    40  
    41 To get !AnnouncerEmailEncryption up and working you'll have to do take the following steps:
     25=== !AnnouncerEmailEncryption ===
     26To get !AnnouncerEmailEncryption up and working you'll have to take the following steps:
    4227 1. install GnuPG on the same host along with Trac[[BR]]
    43   For Debian GNU/Linux a simple {{{apt-get install gnupg}}} will do.
    44  2. install python-gnupg
    45   Currently there is no Debian package available. Install from source of [http://code.google.com/p/python-gnupg/downloads/list project site] is preferred.
    46  3. configure !AnnouncerEmailEncryption in the ![announcer] section of trac.ini for the given trac environment
    47   * gnupg_dir = <path_to_dir>, will default to something like {{{<tracenv>/gnupg}}}[[BR]]
    48    If not existent, this directory would be created and populated with necessary (initially empty) files on next trac start with !AnnouncerEmailEncryption enabled.
    49   * msg_encryption = False|True, default to "False"
    50    Pretty self-explaining. External would opt for splitting recipients according to following rules into group require_encryption_group and allow_verbatim_msg_group and sending an local, verbatim as well as an encrypted version in parallel, if both recipient lists are not empty.
    51   * external recipient_rule = (list of e-mail addresses or regex describing range of e-mail addresses)
    52  4. import pubkeys and associate with users
     28  For Debian GNU/Linux a simple
     29{{{
     30    apt-get install gnupg
     31}}}
     32 will do.
     33 2. install `python-gnupg`
     34  Currently there is no Debian package available. Install from source of [http://code.google.com/p/python-gnupg/downloads/list project site] is preferred. So we get a mature and actively maintained Python interface to GnuPG.
     35 3. configure !AnnouncerEmailEncryption in the !`[announcer]` section of `trac.ini` for the given Trac environment
     36
     37  ||'''available option''' ||'''value''' ||'''default''' ||'''note''' ||
     38  ||rcpt_allow_regexp ||string ||`''` ||not strikly related to cryptography but a way to generally limit the possible scope for any announcement email ^1^||
     39  ||rcpt_local_regexp ||string ||`''` ||whitelist email addresses excluded from crypto operations ^2^||
     40  ||email_crypto ||'sign'|'encrypt'|'sign,encrypt' ||`''` ||crypto action selector ||
     41  ||gpg_binary ||<gpg_binary_name> ||`'gpg'` ||name, full path prepended optionally to allow for use of custom GnuPG installations in unusual locations or switch between several GnuPG versions installed ||
     42  ||gpg_home ||<full_path_to_keyring_file_store> ||`''` ||something like `<full_path_to_tracenv>/gnupg` ^3^||
     43  ||gpg_signing_key ||OpenPGP key by ID or fingerprint ||`None` ||key selector to set one of several available keys for signing operation ||
     44  ^1^ e-mail address(es) or valid Python regex describing range of e-mail addresses[[BR]]
     45  ^2^ string or regex like before, to sort matching recipients, sending verbatim announcement in parallel with an encrypted version, if both resulting recipient lists are not empty[[BR]]
     46  ^3^ if not existent, directory will be created and populated with necessary (initially empty) files on next announcement with !AnnouncerEmailEncryption enabled
     47
     48 4. import public keys and optionally associate with users
    5349  * admin:
    5450   * (mass-)upload from local pubkey(ring) files
     
    6258    For simplicity I'd make existence of a pubkey equivalent to an "always encrypt msg for me with this key" option.
    6359    For convenience it might still be possible to temporarily disable a key and re-enable it later without deletion and re-import as this is directly supported by GnuPG.
    64  5. add an automatic signing key for the given trac environment (optional)
    65   * upload from local pubkey(ring) file or create it on-demand
    66   * use secret key, if only one is available, else provide a drop-down or similar for selection
     60 5. add an automatic signing key for the given Trac environment (optional)
     61  * upload from local secret key(ring) file or create it on-demand
     62  * just use (only/last added) secret key, provide a drop-down or similar for selection, if more than one is available
    6763   For convenience it should be possible to temporarily disable a key and re-enable it later without deleting it.
    6864 
    69 Beware, that this is by now no code but pure concept and subject to change a lot, before public release of the code. As with current code for AnnouncerPlugin there'll be DEBUG logging embedded into all operations mentioned above.
     65Beware, that 4 and 5 is not fully covered by current development code. So this is subject to change a lot, before a public release. As with current code for AnnouncerPlugin there'll be DEBUG logging embedded into all operations mentioned above.
    7066
    71 ==== Behind current development scope ====
     67=== Q&A ===
     68 ?: ''Is it true that different users will have different keys?  If so, we can add configuration to the user's preference page.  We could have a big textbox for GPG key and if they have one entered, then use encryption.
     69  A: Yes, different users will (typically) have different keys. It might be desirable even to support multiple keys per user. Only in rare cases one key would be associated with different users/e-mail addresses, even if this might be technically perfectly valid and useful. But it indicates violation of the de-facto standard one-owner-per-key that abhor kind of group keys.
     70
     71== Beyond current development scope ==
    7272There is a [t:wiki:TracDev/Proposals/Announcer proposal] to replace current Trac Notification system with AnnouncerPlugin. This will make the effort for a really clean solution even more urgent.
    7373
    7474Consider [https://subtrac.sara.nl/oss/email2trac/ticket/186 cryptography related features] for EmailtoTracScript ([https://subtrac.sara.nl/oss/email2trac current home outside of Tack-Hacks]). It could be interesting i.e. to allow only e-mails with valid signature from known senders to pass, fighting spam at another level.
    7575
    76 ==== Q&A ====
    77  ?: ''Is it true that different users will have different keys?  If so, we can add configuration to the user's preference page.  We could have a big textbox for GPG key and if they have one entered, then use encryption.
    78   A: Yes, different users will (typically) have different keys. It might be desirable even to support multiple keys per user. Only in rare cases one key would be associated with different users/e-mail addresses, even if this might be technically perfectly valid and useful. But it indicates violation of the de-facto standard one-owner-per-key that abhor kind of group keys.
     76== Development notes ==
     77=== Discussion ===
     78Hints, recommendations? Known-good code references or popular applications? Put your comments in here, please.
    7979
    80 === How ===
    81 ==== Available interfaces with GnuPG ====
     80=== Available interfaces with GnuPG ===
    8281To make it more difficult for me to start I've found not one but several candidates for interacting with GnuPG from Python (http://wiki.python.org/moin/GnuPrivacyGuard has a listing with some more comments):
    8382 * [http://code.google.com/p/python-gnupg/ python-gnupg]
     
    112111 * [http://www.freenet.org.nz/ezPyCrypto/detail/index.html ezPyCrypto], a simpler API on top of !PyCrypto
    113112
    114 ==== The choice: python-gnupg ====
     113=== The choice: python-gnupg ===
    115114'''python-gnupg''' was tested, !PyMe a little too. It became clear, that python-gnupg just worked without much hassle. Anything else had more dependencies and was more complicated i.e. by introducing GPGME. This applies to !PyMe as well as PyGPGME. GnuPGInterface, OpenPGP, cryptlib where skipped right after the initial interface research.
    116115
    117 ==== The code ====
     116=== Q&A ===
     117[FIXME: add more Q+A here to help with code design evaluation and code review] 
     118 ?: Why not implement encryption as another IAnnouncementEmailDecorator
     119  A: Decorators are called without guaranteed order. Encryption needs control, that it'll be the last message body mangling action.
     120  ''We can change this pretty easily'' - '''doki_pen'''
     121 ?: Why not implement encryption as another IAnnouncementFormatter
     122  A: Encryption is not about encoding etc.
     123  ''Formatter is more about turning an event into a message, it shouldn't be done here.'' - '''doki_pen'''
     124 ? Doesn't smtplib or any other stock python library handle encryption?
     125  A: No. Pythons smtplib is dedicated to e-mail construction including MIME, but no PGP/MIME etc. (see http://docs.python.org/library/smtplib.html). Pythons nativ [http://docs.python.org/library/crypto.html crypto utils] currently consist of secure hash and checksum generators (md5, sha).
     126 ?: What are the explicitly handled exceptions?
     127  A: For readability let's try to put this into a table.
     128
     129  ||exception ||cause ||action/behavior ||
     130  ||missing pubkey ||fingerprint in user settings but no corresponding key in pubkeyring file ||delete recipient from recipient list of event in delivery, create new event with info "specified pubkey not in Tracs keyring" to be sent to this user and project admin ||
     131
     132 ?: Does python-gnupg support GnuPG v2?
     133  A: AFAIK yes, both versions support same CLI syntax. I'll continue to test with both versions in the future to maintain compatibility. There might be even a bonus from using GnuPG v2, since it is announced to be PGP/MIME aware. However I'll still have to look into this in detail.
     134
     135=== Development traces (history) ===
     136This is kept for reference and personal attitude to preserve historical notes.
    118137 1. step: add some code to make encryption just work '''done'''[[BR]]
    119138  * expect encryption/signing key ID hard-coded, some fixed values for variables I'd like to see as options in [annoucer] section of trac.ini and other ugliness
     
    139158 3. step: extend web_ui of AnnouncerPlugin to remote-control new options from user and/or administration settings
    140159
    141 hints, recommendations? known-good code references or popular applications?
    142 
    143 ==== Q&A ====
    144 [FIXME: add more Q+A here to help with code design evaluation and code review] 
    145  ?: Why not implement encryption as another IAnnouncementEmailDecorator
    146   A: Decorators are called without guaranteed order. Encryption needs control, that it'll be the last message body mangling action.
    147   ''We can change this pretty easily'' - '''doki_pen'''
    148  ?: Why not implement encryption as another IAnnouncementFormatter
    149   A: Encryption is not about encoding etc.
    150   ''Formatter is more about turning an event into a message, it shouldn't be done here.'' - '''doki_pen'''
    151  ? Doesn't smtplib or any other stock python library handle encryption?
    152   A: No. Pythons smtplib is dedicated to e-mail construction including MIME, but no PGP/MIME etc. (see http://docs.python.org/library/smtplib.html). Pythons nativ [http://docs.python.org/library/crypto.html crypto utils] currently consist of secure hash and checksum generators (md5, sha).
    153  ?: What are the explicitly handled exceptions?
    154   A: For readability let's try to put this into a table.
    155 
    156   ||exception ||cause ||action/behavior ||
    157   ||missing pubkey ||fingerprint in user settings but no corresponding key in pubkeyring file ||delete recipient from recipient list of event in delivery, create new event with info "specified pubkey not in Tracs keyring" to be sent to this user and project admin ||
    158 
    159  ?: Does python-gnupg support GnuPG v2?
    160   A: AFAIK yes, both versions support same CLI syntax. I'll continue to test with both versions in the future to maintain compatibility. There might be even a bonus from using GnuPG v2, since it is announced to be PGP/MIME aware. However I'll still have to look into this in detail.
    161 
    162 === Sources (for ideas and code) ===
     160== Resources (for ideas and code) ==
    163161 * GNU Privacy Guard Manual at http://www.gnupg.org/documentation/manuals/gnupg/
    164162 * Why sign&encrypt is not very secure by default see http://world.std.com/~dtd/sign_encrypt/sign_encrypt7.html (discussion about vulnerability againgst "surreptitious forwarding")