Metadata-Version: 2.4
Name: evalyn_sdk_harishatmecs
Version: 0.0.6
Summary: Python SDK to send evaluation data to an API with support for batch processing
Author-email: Harish Nankani <harish.nankani@atmecs.com>
License: 
        ---
        
        ##  `LICENSE` (MIT License)
        
        ```text
        MIT License
        
        Copyright (c) 2025 Harish Nankani
        
        Permission is hereby granted, free of charge, to any person obtaining a copy
        of this software and associated documentation files (the "Software"), to deal
        in the Software without restriction, including without limitation the rights
        to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
        copies of the Software, and to permit persons to whom the Software is
        furnished to do so, subject to the following conditions:
        
        The above copyright notice and this permission notice shall be included in
        all copies or substantial portions of the Software.
        
        THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
        IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
        FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
        AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES, OR OTHER
        LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
        OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
        THE SOFTWARE.
        ```
Requires-Python: >=3.7
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: requests
Requires-Dist: pandas
Requires-Dist: openpyxl
Requires-Dist: python-dotenv
Dynamic: license-file

# EvalynSDK - EvaluatorSDK

**EvaluatorSDK** is a lightweight Python SDK that helps developers collect and send evaluation data to a centralized API endpoint for analysis. It supports both individual and batch submissions from datasets (Excel or JSON).

## 🚀 Features

- Read config from environment variables
- Automatically generate and track request IDs
- Add and manage parameters per request
- Send evaluation data as POST requests to API
- Support for batch processing from `.xlsx` or `.json` datasets
- Exception handling and validation

## 📦 Installation

Make sure pip is installed on your system. Then, you can install the SDK using pip:

To install the SDK, simply run:

```bash
pip install evalyn-sdk-harishatmecs==0.0.6
```

## Environment Setup

Create a .env file in your project root and add the following:

Currently it works in ATMECS network only, please contact your architect (Harish Nankani) for access.
This IP will change when it is deployed in ATMECS servers. 

```
#EVAL_API_URL=http://172.29.208.1:8001/api/proj_eval
EVAL_API_URL=http://10.10.10.154:8002/api/proj_eval
EVAL_API_KEY=your_project_code_here

DOCS_PATH=your_docs_path_here_optional
VECTORDB_PATH=your_vectordb_path_here_optional
EVAL_DATASET_FILE=your_excel_file_path_here_optional
```

The SDK uses python-dotenv to load these values automatically.

## Usage

To use the SDK, import it in your Python script and create an instance of the `EvaluatorSDK` class. Then, you can call its methods to interact with the API.

```python
from evalyn_sdk_harishatmecs.sdk.evaluator import EvaluatorSDK
import traceback

# Initialize SDK
eval_obj = EvaluatorSDK()

# Start a new request (creates unique request_id internally)
eval_obj.start_request()

# Add all relevant parameters
eval_obj.add_project_code("demo")
eval_obj.add_prompt(request.query)
eval_obj.add_no_of_docs_in_context(no_of_docs_in_faiss)
eval_obj.add_top_k(top_k)
eval_obj.add_no_of_docs_retrieved(no_of_docs_retrieved)
eval_obj.add_retrieved_ans(retrieved_ans)
eval_obj.add_final_ans(retrieved_ans)
eval_obj.add_ground_truth(ground_truth)
eval_obj.add_retrieval_time(retrieval_time)
eval_obj.add_generation_time(generation_time)
eval_obj.add_total_query_time(total_query_time)
eval_obj.add_throughput(throughput)
eval_obj.add_retrieved_docs_list(retrieved_docs_list)

# Send the collected data
try:
    # Recommended: send in background thread
    eval_obj.send_in_background()
except Exception as ex:
    print(f"Error sending evaluation data: {ex}")
    traceback.print_exc()

```

## Coming Soon

- Retry logic on failed sends
- Improved error handling and logging
- Support for different types of datasets (CSV, JSON, etc.)
- Built-in validation and schema support
- Enhanced security features
