Metadata-Version: 2.1
Name: auto-function-serving
Version: 0.1.0
Summary: A python package to offload a function call to an http server automatically using a decorator.
Author-email: Arrman Anicket Saha <arrmansa99+430@gmail.com>
Project-URL: Homepage, https://github.com/arrmansa/auto-function-serving
Project-URL: Bug Tracker, https://github.com/arrmansa/auto-function-serving/issues
Classifier: Programming Language :: Python :: 3
Classifier: License :: OSI Approved :: Apache Software License
Classifier: Operating System :: OS Independent
Requires-Python: >=3.7
Description-Content-Type: text/markdown
License-File: LICENSE

# auto-function-serving

A python package to offload a function call to an http server running on localhost automatically using a decorator. Compatible with multiprocessing, pickle, flask, fastapi, async etc..

## Why

Imagine a case of a multi threaded or multiprocessing application where 1 or few functions are heavily resource (cpu or memory) intensive, but the other functions can run in parallel.\
Example - an api call followed by tokenization and classification using a large DL model followed by further API calls.\
In such a case, it would make sense to create a server (using torchserve or tfserving) to serve requests, and replace the function call with a post request to the server.\
ServerHandler does this automatically during runtime.\
AsyncServerHandler is also available which uses aiohttp to make the requests async.
AsyncServerHandler has the same usage as ServerHandler, except calls need to have await before it.

## Reccomended Usage
```python
from somemodule import someheavyfunction
from auto_function_serving.ServerHandler import ServerHandler
someheavyfunction = ServerHandler.decorator(someheavyfunction)
```
Any calls to this new **someheavyfunction** will make requests to 1 instance of a process running a [http.server.HTTPServer](https://docs.python.org/3/library/http.server.html) which runs the function within it. Even calls made from different processes, threads, multiprocessing or servers like flask.\
It can also be used like a traditional decorator for functions with no dependencies specified outside the function.
```python
from auto_function_serving.ServerHandler import ServerHandler
@ServerHandler.decorator
def someheavyfunctionwithnodependency(args,**kwargs):
    for i in range(big_number)
        someexpensivecomputation
```

## Installation

Use the package manager pip to install [auto_function_serving](https://pypi.org/project/auto-function-serving/)
```bash
pip install auto_function_serving
```

## More Usage
In general : 
```
some independent code with a callable
```
can be replaced with 
```python
from auto_function_serving.ServerHandler import ServerHandler
callable_name = ServerHandler("""
some independent code with a callable
""", "callable_name")
```

Example :
```python
import module1
import module2
def functionname(someinput):
    a = module1.function1(someinput)
    return module2.function2(a)
```
can be replaced with
```python
from auto_function_serving.ServerHandler import ServerHandler
functionname = ServerHandler("""
import module1
import module2
def functionname(someinput):
    a = module1.function1(someinput)
    return module2.function2(a)
""", "functionname")
```
# How does this work?
Code for the server is stored in ServerHandler.base_code, inspect.cleandoc and some string formatting is used to fill in the blanks.\
A port from 50000 to 60000 is chosen by hashing the input text to make it independent of the source of a function. Collisions are possible, but unlikely. The port can be specified if needed.
```python
from somemodule import someheavyfunction
from auto_function_serving.ServerHandler import ServerHandler
someheavyfunction = ServerHandler.decorator(someheavyfunction, port = 4321)
```
ServerHandler.ip_address is set as "127.0.0.1".\
The server process is started with Popen (or multiprocessing if specified), and the first thing it does is import socket and bind the port - if it's not available the code stops after an exception. Therefore only 1 instance of the server runs at a time on a machine.\
We know the function is ready after we can recieve a valid get request from the server.\
Inputs and outputs are sent as bytes, converted to and from objects using pickle.

## Performance (On my machine)
overhead for small input and output (few bytes) - 
~2ms for requests with urllib.request
~5ms for async requests with aiohttp.ClientSession

overhead for large input and output
~10ms for 0.5 mb input and output (1mb total transfer).\
~60ms for 5 mb input and output (10 mb total transfer).
~600ms for 50 mb input and output (100 mb total transfer).

## Other things to look into
Libraries : Celery, Tfserving, Torchserve, Flask\
Sending globals and locals to exec\
ast trees

## Contributing
Pull requests are welcome.

## License
[Apache License 2.0](https://choosealicense.com/licenses/apache-2.0/)
