Metadata-Version: 1.1
Name: mikkoo
Version: 0.4.0
Summary: Mikkoo is a PgQ to RabbitMQ Relay
Home-page: https://github.com/gmr/mikkoo
Author: Gavin M. Roy
Author-email: gavinmroy@gmail.com
License: BSD
Description: Mikkoo
        ======
        A `PgQ <https://wiki.postgresql.org/wiki/SkyTools#PgQ>`_ to
        `RabbitMQ <https://www.rabbitmq.com>`_ relay.
        
        Mikkoo is named for the rabbit in the "Clever Rabbit and the Elephant" fable.
        
        |Version| |Downloads| |Status|
        
        PgQ Setup
        ---------
        
        1. Install ``pgq`` into your database and create the queue:
        
            .. code:: sql
        
                # CREATE EXTENSION pgq;
                CREATE EXTENSION
                # SELECT pgq.create_queue('test');
                create_queue
                --------------
                            1
                (1 row)
        
        2. Ensure that `pgqd <http://skytools.projects.pgfoundry.org/skytools-3.0/doc/pgqd.html>`_
           is running.
        
        PgQ Event to AMQP Mapping
        -------------------------
        When inserting events into a PgQ queue, the ``pgq.insert_event/7`` function
        should be used with the following field mappings:
        
        +---------------+-----------------------+
        | PgQ Event     | AMQP                  |
        +===============+=======================+
        | ``ev_type``   | Routing Key           |
        +---------------+-----------------------+
        | ``ev_data``   | Message body          |
        +---------------+-----------------------+
        | ``ev_extra1`` | Exchange              |
        +---------------+-----------------------+
        | ``ev_extra2`` | Content-Type Property |
        +---------------+-----------------------+
        | ``ev_extra3`` | AMQP Properties [1]_  |
        +---------------+-----------------------+
        | ``ev_extra4`` | Headers [2]_          |
        +---------------+-----------------------+
        
        .. [1] AMQP properties should be set as a JSON blob. Values set in the ``ev_extra3``
               field will overwrite the automatically created properties ``app_id``,
               ``content_type``, ``correlation_id``, ``headers``, and ``timestamp``.
        .. [2] If ``ev_extra4`` is specified and is a JSON key/value dictionary, it will
               be assigned to the ``headers`` AMQP property.
        
        There is a convenience schema in the `mikkoo.sql <mikkoo.sql>`_ file that adds
        stored procedures for creating properly formatted mikkoo events in PgQ. In
        addition, there is are auditing functions that allow for the creation of an
        audit-log of events that were sent to PgQ.
        
        
        AMQP Message Properties
        ^^^^^^^^^^^^^^^^^^^^^^^
        The following table defines the available fields that can be set in a JSON blob
        in the ``ev_extra3`` field when inserting an event.
        
        +----------------------+----------------+
        | Property             | PgSQL Type     |
        +======================+================+
        | ``app_id``           | text           |
        +----------------------+----------------+
        | ``content_encoding`` | text           |
        +----------------------+----------------+
        | ``content_type``     | text           |
        +----------------------+----------------+
        | ``correlation_id``   | text           |
        +----------------------+----------------+
        | ``delivery_mode``    | int2           |
        +----------------------+----------------+
        | ``expiration``       | text           |
        +----------------------+----------------+
        | ``message_id``       | text           |
        +----------------------+----------------+
        | ``headers``          | text/json [3]_ |
        +----------------------+----------------+
        | ``timestamp``        | int4           |
        +----------------------+----------------+
        | ``type``             | text           |
        +----------------------+----------------+
        | ``priority``         | int4           |
        +----------------------+----------------+
        | ``user_id``          | text           |
        +----------------------+----------------+
        
        .. [3] ``headers`` should be sent to a key/value JSON blob if specified
        
        Values assigned in the JSON blob provided to ``ev_extra3`` take precedence over
        the automatically assigned ``app_id``, ``content_type``, ``correlation_id``,
        ``headers``, and ``timestamp`` values created by Mikkoo at processing time.
        
        Event Insertion Example
        -----------------------
        
        The following example inserts a JSON blob message body of ``{"foo": "bar"}`` that
        will be published to the ``postgres`` exchange in RabbitMQ using the ``test.routing-key``
        routing key. The content type is specified in ``ev_extra2`` and the AMQP ``type``
        message property is specified in ``ev_extra3``.
        
        .. code:: sql
        
            # SELECT pgq.insert_event('test', 'test.routing-key', '{"foo": "bar"}', 'postgres', 'application/json', '{"type": "example"}', '');
             insert_event
            --------------
                        4
            (1 row)
        
        When this message is received by RabbitMQ it will have a message body of:
        
        .. code:: json
        
            {"foo": "bar"}
        
        And it will have message properties similar to the following:
        
        +----------------------+------------------------------------------+
        | Property             | Example Value                            |
        +======================+==========================================+
        | ``app_id``           | ``mikkoo``                               |
        +----------------------+------------------------------------------+
        | ``content_type``     | ``application/json``                     |
        +----------------------+------------------------------------------+
        | ``correlation_id``   | ``0ad6b212-4c84-4eb0-8782-9a44bdfe949f`` |
        +----------------------+------------------------------------------+
        | ``timestamp``        | ``1449600290``                           |
        +----------------------+------------------------------------------+
        | ``type``             | ``example``                              |
        +----------------------+------------------------------------------+
        
        Configuration
        -------------
        The Mikkoo configuration file uses `YAML <http://yaml.org>`_ for markup and allows
        for one or more PgQ queue to be processed.
        
        If you have a Sentry or a Sentry account, the ``Application/sentry_dsn`` setting
        will turn on sentry exception logging, if the
        `raven <https://pypi.python.org/pypi/raven>`_ client library is installed.
        
        Queues are configured by name under the ``Application/workers`` stanza. The
        following example configures two workers for the processing of a queue named
        ``invoices``. Each worker process connects to a local PostgreSQL and RabbitMQ
        instance using default credentials.
        
        .. code:: yaml
        
            Application:
              workers:
                 invoices:
                   postgres_url: postgresql://localhost:5432/postgres
                   rabbitmq_url: amqp://localhost:5672/%2f
                   confirm: False
        
        Queue Configuration Options
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^
        The following table details the configuration options available per queue:
        
        +--------------------+---------------------------------------------------------------------+
        | Key                | Description                                                         |
        +====================+=====================================================================+
        | ``confirm``        | Enable/Disable RabbitMQ Publisher Confirmations. Default: ``True``  |
        +--------------------+---------------------------------------------------------------------+
        | ``consumer_name``  | Overwrite the default PgQ consumer name. Default: ``mikkoo``        |
        +--------------------+---------------------------------------------------------------------+
        | ``max_failures``   | Maximum failures before discarding an event. Default: ``10``        |
        +--------------------+---------------------------------------------------------------------+
        | ``postgresql_url`` | The url for connecting to PostgreSQL                                |
        +--------------------+---------------------------------------------------------------------+
        | ``rabbitmq_url``   | The AMQP url for connecting to RabbitMQ                             |
        +--------------------+---------------------------------------------------------------------+
        | ``retry_delay``    | How long in seconds until PgQ emits failed events. Default: ``10``  |
        +--------------------+---------------------------------------------------------------------+
        | ``unregister``     | Unregister a consumer with PgQ on shutdown. Default: ``True``       |
        +--------------------+---------------------------------------------------------------------+
        | ``wait_duration``  | How long to wait before checking the queue after the last empty     |
        |                    | result. Default: ``1``                                              |
        +--------------------+---------------------------------------------------------------------+
        
        Example Configuration
        ^^^^^^^^^^^^^^^^^^^^^
        
        The following is an example of a full configuration file:
        
        .. code:: yaml
        
            Application:
        
              poll_interval: 10
              sentry_dsn: [YOUR SENTRY DSN]
        
              statsd:
                enabled: true
                host: localhost
                port: 8125
        
              workers:
                test:
                  confirm: False
                  consumer_name: my_consumer
                  max_failures: 5
                  postgres_url: postgresql://localhost:5432/postgres
                  rabbitmq_url: amqp://localhost:5672/%2f
                  retry_delay: 5
                  unregister: False
                  wait_duration: 5
        
            Daemon:
              user: mikkoo
              pidfile: /var/run/mikkoo
        
            Logging:
              version: 1
              formatters:
                verbose:
                  format: '%(levelname) -10s %(asctime)s  %(process)-6d %(processName) -20s %(name) -18s: %(message)s'
                  datefmt: '%Y-%m-%d %H:%M:%S'
              handlers:
                console:
                  class: logging.StreamHandler
                  formatter: verbose
                  debug_only: True
              loggers:
                helper:
                  handlers: [console]
                  level: INFO
                  propagate: true
                mikkoo:
                  handlers: [console]
                  level: INFO
                  propagate: true
                pika:
                  handlers: [console]
                  level: ERROR
                  propagate: true
                queries:
                  handlers: [console]
                  level: ERROR
                  propagate: true
                tornado:
                  handlers: [console]
                  level: ERROR
                  propagate: true
              root:
                handlers: [console]
                level: CRITICAL
                propagate: true
              disable_existing_loggers: true
              incremental: false
        
        
        .. |Version| image:: https://img.shields.io/pypi/v/mikkoo.svg?
           :target: http://badge.fury.io/py/mikkoo
        
        .. |Status| image:: https://img.shields.io/travis/gmr/mikkoo.svg?
           :target: https://travis-ci.org/gmr/mikkoo
        
        .. |Downloads| image:: https://img.shields.io/pypi/dm/mikkoo.svg?
           :target: https://pypi.python.org/pypi/mikkoo
        
Keywords: amqp rabbitmq postgresql
Platform: UNKNOWN
Classifier: Development Status :: 3 - Alpha
Classifier: Programming Language :: Python :: 2
Classifier: Programming Language :: Python :: 2.6
Classifier: Programming Language :: Python :: 2.7
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.4
Classifier: Programming Language :: Python :: 3.5
Classifier: Programming Language :: Python :: Implementation :: CPython
Classifier: License :: OSI Approved :: BSD License
