Metadata-Version: 2.1
Name: eer
Version: 0.0.2
Summary: Python package to compute the Equal Error Rate, wrapping lrreval
Author-email: David van Leeuwen <david.vanleeuwen@gmail.com>
License: MIT License
        
        Copyright (c) 2021--2023 David A. van Leeuwen
        
        Permission is hereby granted, free of charge, to any person obtaining a copy
        of this software and associated documentation files (the "Software"), to deal
        in the Software without restriction, including without limitation the rights
        to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
        copies of the Software, and to permit persons to whom the Software is
        furnished to do so, subject to the following conditions:
        
        The above copyright notice and this permission notice shall be included in all
        copies or substantial portions of the Software.
        
        THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
        IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
        FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
        AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
        LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
        OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
        SOFTWARE.
        
Project-URL: Homepage, https://github.com/davidavidav/eer.py
Project-URL: Bug Tracker, https://github.com/davidavdav/eer.py/issues
Classifier: Programming Language :: Python :: 3
Classifier: License :: OSI Approved :: MIT License
Classifier: Operating System :: OS Independent
Requires-Python: >=3.7
Description-Content-Type: text/markdown
License-File: LICENSE

# Equal Error Rate

The Equal Error Rate is an overall performance metric for a binary classifier.  It is insensitive to

 - evaluation priors, so the metric doesn't change if the relative amounts of data points in the two classes varies
 - calibration, so the classifier only needs to produce consistent scores, and not set a threshold

This package is a thin wrapper around [llreval](https://github.com/davidavdav/llreval), which computes the equal error rate
in the ROC convex hull interpretation, which is consistent and meaningful, see [Niko Brümmer's PhD thesis](http://hdl.handle.net/10019.1/5139).

## Installation

```sh
pip install wer
```

## Usage

Collect your classifier's (floating point) scores in a `numpy` array.  Prepare a parallel array with values `0` for the class related to lower scores, and `1` for the class related to higher scores.  Then call `eer.eer(scores, labels)`.

Example:

Simulate [calibrated scores](https://www.isca-speech.org/archive/interspeech_2013/leeuwen13_interspeech.html) for a binary classifier, and compute the equal error rate.
```python
import numpy as np
from eer import eer, eer_tnt

ntar, nnon = 100_000, 1_000_000
targets = 2 + 2 * np.random.randn(ntar)
non_targets = -2 + 2 * np.random.randn(nnon)
print("EER using target and non-target scores:", eer_tnt(targets, non_targets))

scores = np.concatenate([targets, non_targets])
labels = np.concatenate([np.ones(targets.shape[0]), np.zeros(non_targets.shape[0])])
print("EER using scores and labels:", eer(scores, labels))

```
The values printed should not be too far from [0.5 + 0.5erf(-1/√2) ≈ 0.1586552](https://github.com/davidavdav/ROCAnalysis.jl).

