Metadata-Version: 2.1
Name: pytest-kafka
Version: 0.5.0
Summary: Zookeeper, Kafka server, and Kafka consumer fixtures for Pytest
Home-page: https://gitlab.com/karolinepauls/pytest-kafka
Author: Karoline Pauls
Author-email: code@karolinepauls.com
License: MIT
Project-URL: Bug Tracker, https://gitlab.com/karolinepauls/pytest-kafka/issues
Project-URL: Documentation, https://pytest-kafka.readthedocs.io/
Project-URL: Source Code, https://gitlab.com/karolinepauls/pytest-kafka
Platform: UNKNOWN
Classifier: Framework :: Pytest
Classifier: Topic :: Software Development :: Testing
Classifier: License :: OSI Approved :: BSD License
Classifier: Programming Language :: Python :: 3.6
Classifier: Programming Language :: Python :: 3.7
Classifier: Programming Language :: Python :: 3.8
Classifier: Programming Language :: Python :: 3.9
Provides-Extra: dev
Provides-Extra: doc
License-File: LICENCE

pytest-kafka |pypi| |pyversions| |ci| |docs|
============================================

.. |pypi| image:: https://img.shields.io/pypi/v/pytest-kafka.svg
    :target: https://pypi.org/project/pytest-kafka/

.. |pyversions| image:: https://img.shields.io/pypi/pyversions/pytest-kafka.svg
    :target: https://pypi.org/project/pytest-kafka/

.. |ci| image:: https://gitlab.com/karolinepauls/pytest-kafka/badges/master/pipeline.svg
    :target: https://gitlab.com/karolinepauls/pytest-kafka/pipelines

.. |docs| image:: https://readthedocs.org/projects/pytest-kafka/badge/?version=latest
    :target: https://pytest-kafka.readthedocs.io/en/latest/


Pytest fixture factories for Zookeeper, Kafka server and Kafka consumer.
`Read the API docs <https://pytest-kafka.readthedocs.io>`__.

.. code:: python

    from pathlib import Path
    from pytest_kafka import (
        make_zookeeper_process, make_kafka_server, make_kafka_consumer,
        terminate,
    )

    ROOT = Path(__file__).parent
    KAFKA_SCRIPTS = ROOT / 'kafka/bin/'
    KAFKA_BIN = str(KAFKA_SCRIPTS / 'kafka-server-start.sh')
    ZOOKEEPER_BIN = str(KAFKA_SCRIPTS / 'zookeeper-server-start.sh')

    # You can pass a custom teardown function (or parametrise ours). Just don't call it `teardown`
    # or Pytest will interpret it as a module-scoped teardown function.
    teardown_fn = partial(terminate, signal_fn=Popen.kill)
    zookeeper_proc = make_zookeeper_process(ZOOKEEPER_BIN, teardown_fn=teardown_fn)
    kafka_server = make_kafka_server(KAFKA_BIN, 'zookeeper_proc', teardown_fn=teardown_fn)
    kafka_consumer = make_kafka_consumer(
        'kafka_server', seek_to_beginning=True, kafka_topics=['topic'])


This creates 3 fixtures:

#. ``zookeeper_proc`` - Zookeeper process
#. ``kafka_server`` - Kafka process
#. ``kafka_consumer`` - usable ``kafka.KafkaConsumer`` instance


``ZOOKEEPER_BIN`` and ``KAFKA_BIN`` are paths to launch scripts in your Kafka distribution. Check
this project's setup.py to see a way of installing Kafka for development.

It is advised to pass ``seek_to_beginning=True`` because otherwise some messages may not be captured
by the consumer. This requires knowing the topics upfront because without topics there's no
partitions to seek.

Kafka server is known to take a couple of seconds to terminate gracefully. You probably don't
need that, so you can pass ``partial(terminate, signal_fn=Popen.kill)`` to make it killed with
SIGKILL and waited for afterwards.

It's possible to create multiple Kafka fixtures forming a cluster by passing the same Zookeeper
fixture to them. For an example, check the `tests
<https://gitlab.com/karolinepauls/pytest-kafka/blob/master/test_pytest_kafka.py>`__.

Session-scoped fixtures are also available. Consult the `test suite
<https://gitlab.com/karolinepauls/pytest-kafka/blob/master/test_pytest_kafka.py>`__.


System requirements
-------------------

- Python 3.6+
- a JVM that can run Kafka and Zookeeper
- Kafka - https://kafka.apache.org/downloads


Development
-----------

Execute ``./setup.py develop`` in a virtualenv. This will take care of:

- installing dependencies
- updating pip
- installing dev dependencies
- installing Kafka to the project dir (for development only)


Acknowledgements
----------------

The library has been open-sourced from a codebase belonging to
`Infectious Media <https://infectiousmedia.com>`__.


