Metadata-Version: 2.1
Name: iaparc-inference
Version: 0.0.6
Summary: Inference service package for IAPARC
Author-email: Bertrand Louargant <blouargant@chapsvision.com>
Project-URL: Homepage, https://gitlab.com/ia-parc/public/pypi/iaparc_inference
Project-URL: Issues, https://gitlab.com/ia-parc/public/pypi/iaparc_inference/issues
Classifier: Programming Language :: Python :: 3
Classifier: License :: OSI Approved :: Apache Software License
Classifier: Operating System :: OS Independent
Requires-Python: >=3.8
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: nats-py~=2.6.0
Requires-Dist: pyyaml~=6.0.1
Requires-Dist: asyncio~=3.4.3
Requires-Dist: json-tricks~=3.17.3

# iaparc_inference


[![PyPI version](https://badge.fury.io/py/iaparc-inference.svg)](https://badge.fury.io/py/iaparc-inference)
![PyPI - License](https://img.shields.io/pypi/l/iaparc-inference)


The IA Parc inference plugin allows developers to easily integrate their inference pipeline into IA Parc's production module.   


## Installation

```bash
pip install iaparc-inference
```

## Usage

* If your inference pipeline support batching:
  
    ```python
    from iaparc_inference import IAPListener

    # Define a callback to query your inference pipeline
    # To load your model only once it is recommended to use a class:
    class MyModel:
        def __init__(self, model_path: str):
            ## Load your model in pytorch, tensorflow or any other backend
        
        def batch_query(batch: list) -> list:
            ## execute your pipeline on a batch input

    if __name__ == '__main__':
        # Initiate your model class
        my_model = MyModel("path/to/my/model")

        # Initiate IAParc listener
        listener = IAPListener(my_model.batch_query)
        # Start the listener
        listener.run()

    ```
* If your inference pipeline **do not** support batching:
  
    ```python
    from iaparc_inference import IAPListener

    # Define a callback to query your inference pipeline
    # To load your model only once it is recommended to use a class:
    class MyModel:
        def __init__(self, model_path: str):
            ## Load your model in pytorch, tensorflow or any other backend
        
        def single_query(one_input):
            ## execute your pipeline on a single input

    if __name__ == '__main__':
        # Initiate your model class
        my_model = MyModel("path/to/my/model")

        # Initiate IAParc listener
        listener = IAPListener(my_model.single_query, batch=1)  # Note that batch size is forced to 1 here
        # Start the listener
        listener.run()

## Features
* Dynamic batching
* Autoscalling 
* Support both synchronous and asynchronous queries
* Data agnostic


## License
This project is licensed under the Apache License Version 2.0  - see the Apache [LICENSE](https://www.apache.org/licenses/LICENSE-2.0) file for details.
