Metadata-Version: 2.4
Name: donut-ai-evaluate
Version: 0.1.2
Summary: Donut packages for evaluating the output of data entry and redact.
Author-email: Zain Ullah <zain.ullah@siparadigm.com>, Maaz Ullah <maaz.ullah@siparadigm.com>, Ameer Hamza <ameer.hamza@siparadigm.com>
License: MIT
Requires-Python: >=3.10
Description-Content-Type: text/markdown
Requires-Dist: annotated-types==0.7.0
Requires-Dist: google-ai-generativelanguage==0.6.18
Requires-Dist: google-cloud-vision==3.3.0
Requires-Dist: httpx==0.28.1
Requires-Dist: json_repair==0.41.0
Requires-Dist: jsonpatch==1.33
Requires-Dist: langchain==0.3.23
Requires-Dist: langchain-google-genai==2.1.2
Requires-Dist: Levenshtein==0.27.1
Requires-Dist: numpy==2.2.6
Requires-Dist: opencv-python==4.11.0.86
Requires-Dist: orjson==3.10.18
Requires-Dist: pandas==2.2.3
Requires-Dist: pdf2image==1.17.0
Requires-Dist: pillow==11.2.1
Requires-Dist: pydantic==2.8.0
Requires-Dist: python-dotenv==1.1.0
Requires-Dist: PyYAML==6.0.2
Requires-Dist: RapidFuzz==3.13.0
Requires-Dist: requests==2.32.3
Requires-Dist: scipy==1.15.3
Requires-Dist: SQLAlchemy==2.0.41
Requires-Dist: tenacity==9.1.2

# Donut AI Evaluate

Install the library using

```
pip install donut-ai-evaluate
```


Before running the evaluation, make sure that the google gemini API key is present in the environment variables.

## Data Entry Evaluation

The user can either compare a single field or multiple fields with one another. 

The following code snipped shows an example of how single field comparison works:

```
from donut_ai_evaluate.data_entry import compare_single_field

data = {
    "Name" : 
    {
        "predicted" : "John",
        "ground-truth" : "Jon"
    }
}

comparison_data = compare_single_field(data)

print(comparison_data)
```

The following code snipped shows an example of how multi field comparison works:

```
from donut_ai_evaluate.data_entry import compare_batch_fields

data = [
    {
        "Name" : 
        {
            "predicted" : "John",
            "ground-truth" : "Jon"
        }
    },
    {
        "State" : 
        {
            "predicted" : "Florida",
            "ground-truth" : "Fl"
        }
    },
    {
        "Phone Number" : 
        {
            "predicted" : "922000511",
            "ground-truth" : "92-2000-511"
        }
    }
]

comparison_data = compare_batch_fields(data)

print(comparison_data)
```

## Redact Evaluation

Field redactions can be evaluated using the following code:

```
from donut_ai_evaluate.redact import is_redacted_field_gt,is_gt_field_redacted
from donut_ai_evaluate._common.eval_logger import logger


# Example Ground Truth Bboxes as lists [x, y, w, h]
gt_boxes_raw = [
    [10, 10, 50, 20],   # GT 0
    [100, 100, 80, 30],  # GT 1
    [200, 50, 40, 40]   # GT 2
]

# Example AI Detected Bboxes as lists [x, y, w, h]
# Scenario 1: Perfect match
ai_boxes_perfect_raw = [
    [11, 11, 49, 19],   # AI 0 -> matches GT 0
    [100, 100, 80, 30], # AI 1 -> matches GT 1
    [201, 51, 39, 39],   # AI 2 -> matches GT 2
    [201, 51, 41, 123] , # AI 3 -> matches None
    [300, 41, 41, 123]  # AI 4 -> matches None
]


# Check if each GT is matched by an AI box
is_gt_field_redacted(gt_boxes_raw, ai_boxes_perfect_raw) # Should be [True, True, True]

# Check if each AI box matches a GT box
is_redacted_field_gt(ai_boxes_perfect_raw, gt_boxes_raw) # Should be [True, True, True]

```
