Metadata-Version: 2.1
Name: pgsync
Version: 1.1.24
Summary: Postgres to elasticsearch sync
Home-page: https://github.com/toluaina/pgsync
Author: Tolu Aina
Author-email: toluaina@hotmail.com
Maintainer: Tolu Aina
Maintainer-email: toluaina@hotmail.com
License: LGPLv3
Project-URL: Bug Reports, https://github.com/toluaina/pgsync/issues
Project-URL: Funding, https://patreon.com/toluaina
Project-URL: Source, https://github.com/toluaina/pgsync
Project-URL: Web, https://pgsync.com
Description: # PostgreSQL to Elasticsearch sync
        
        
        `PGSync <https://pgsync.com>`_ is a middleware for syncing data from `Postgres <https://www.postgresql.org>`_ to `Elasticsearch <https://www.elastic.co/products/elastic-stack>`_.  
        It allows you to keep `Postgres <https://www.postgresql.org>`_ as your source of truth data source and
        expose structured denormalized documents in `Elasticsearch <https://www.elastic.co/products/elastic-stack>`_.
        
        
        ### Requirements
        
        - `Python <https://www.python.org>`_ 3.6+
        - `Postgres <https://www.postgresql.org>`_ 9.4+
        - `Redis <https://redis.io>`_
        - `Elasticsearch <https://www.elastic.co/products/elastic-stack>`_ 6.3.1+
        
        ### Postgres setup
          
          Enable `logical decoding <https://www.postgresql.org/docs/current/logicaldecoding.html>`_ in your 
          Postgres setting.
        
          - You would also need to set up two parameters in your Postgres config postgresql.conf
        
            ```wal_level = logical```
        
            ```max_replication_slots = 1```
        
        ### Installation
        
        You can install PGSync from `PyPI <https://pypi.org>`_:
        
            $ pip install pgsync
        
        ### Config
        
        Create a schema for the application named e.g **schema.json**
        
        `Example schema <https://github.com/toluaina/pgsync/blob/master/examples/airbnb/schema.json>`_
        
        Example spec
        
        .. code-block::
        
            [
                {
                    "database": "[database name]",
                    "index": "[elasticsearch index]",
                    "nodes": [
                        {
                            "table": "[table A]",
                            "schema": "[table A schema]",
                            "columns": [
                                "column 1 from table A",
                                "column 2 from table A",
                                ... additional columns
                            ],
                            "children": [
                                {
                                    "table": "[table B with relationship to table A]",
                                    "schema": "[table B schema]",
                                    "columns": [
                                      "column 1 from table B",
                                      "column 2 from table B",
                                      ... additional columns
                                    ],
                                    "relationship": {
                                        "variant": "object",
                                        "type": "one_to_many"
                                    },
                                    ...
                                },
                                {
                                    ... any other additional children
                                }
                            ]
                        }
                    ]
                }
            ]
        
        ### Environment variables 
        
        Setup required environment variables for the application
        
            SCHEMA='/path/to/schema.json'
        
            ELASTICSEARCH_HOST=localhost
            ELASTICSEARCH_PORT=9200
        
            PG_HOST=localhost
            PG_USER=i-am-root # this must be a postgres superuser
            PG_PORT=5432
            PG_PASSWORD=*****
        
            REDIS_HOST=redis
            REDIS_PORT=6379
            REDIS_DB=0
            REDIS_AUTH=*****
        
        
        ## Running
        
        Bootstrap the database (one time only)
          $ bootstrap --config schema.json
        Run pgsync as a daemon
          $ pgsync --config schema.json --daemon
        
Keywords: pgsync,elasticsearch,postgres,change data capture
Platform: UNKNOWN
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Developers
Classifier: Natural Language :: English
Classifier: Programming Language :: Python :: 3.6
Classifier: Programming Language :: Python :: 3.7
Classifier: Programming Language :: Python :: 3.8
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: Implementation :: CPython
Classifier: Programming Language :: Python :: Implementation :: PyPy
Classifier: License :: OSI Approved :: GNU Lesser General Public License v3 (LGPLv3)
Requires-Python: >=3.6.0
Description-Content-Type: text/markdown
