Metadata-Version: 2.1
Name: esnpy
Version: 0.3.0
Summary: Out-of-the-box framework for Echo State Networks
Author-email: Théo BL <biasutto.t@gmail.com>
License: The MIT License (MIT)
        
        Copyright (c) 2015 
        
        Permission is hereby granted, free of charge, to any person obtaining a copy
        of this software and associated documentation files (the "Software"), to deal
        in the Software without restriction, including without limitation the rights
        to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
        copies of the Software, and to permit persons to whom the Software is
        furnished to do so, subject to the following conditions:
        
        The above copyright notice and this permission notice shall be included in all
        copies or substantial portions of the Software.
        
        THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
        IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
        FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
        AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
        LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
        OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
        SOFTWARE.
        
        
Project-URL: GitHub, https://github.com/NiziL/esnpy
Keywords: machine learning,neural networks,echo state network
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Science/Research
Classifier: License :: OSI Approved :: MIT License
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Requires-Python: >=3.9
Description-Content-Type: text/markdown
Provides-Extra: lint
Provides-Extra: doc
License-File: LICENSE

# esnpy

`esnpy` is an out-of-the-box framework to experiment around ESN and DeepESN.  
Models have been implemented in pure NumPy/SciPy, so there is no need for a powerful GPU, or any esoteric requirements. 

Right now, the focus is on batch training, and feedback loops have not been taken into account.  
But feel free to open a ticket a discuss about anything you need, features you want, or even help !

Note from the author: *`esnpy` is a small projet I initiated during my master intership, and have recently cleaned up. I might keep working on it for fun, but If you want/need a more robust framework, [ReservoirPy](https://github.com/reservoirpy/reservoirpy) might be the one you're searching for ;)*

## Getting Started

### Installation

**From PyPI**
```bash
pip install esnpy
```

**From source**
```bash
pip install git+https://github.com/NiziL/esnpy#egg=esnpy
```
Use `github.com/NiziL/esnpy@<tag or branch>#egg=esnpy` to install from a specific branch or tag instead of main.

### Quickstart

```python
import esnpy

reservoir_builder = createBuilder()
trainer = createTrainer()
warmup, data, target = loadData()

# create the echo state network
esn = esnpy.ESN(reservoir_builder.build(), trainer)
# train it
esn.fit(warmup, data, target)
# test it
predictions = esnpy.transform(data)
print(f"error: {compute_err(target, predictions)}")
```

#### `ESN` and `DeepESN`

You can create your ESN with `esnpy.ESN`. 
The constructor needs a `esnpy.Reservoir` and an implementation of `esnpy.train.Trainer`. 

`esnpy.DeepESN` doesn't differ a lot, it just expect a list of `Reservoir` and have an optional parameters `mask` to specify from which reservoirs the `Trainer` should learn. The size of `mask` and `reservoirs` must be the same. 

Then, simply call `fit` function by passing some warm up and training data with the related targets.  
Once trained, run predictions using `transform`.

#### `Reservoir` and `ReservoirBuilder`

A `Reservoir` can easily be initialized using the `ReservoirBuilder` dataclass.  
For convenience, the configuration class is also a builder, exposing a `build()` method.
This method has an optional `seed` parameter used to make deterministic initialization, and so to ease the comparaison of two identical reservoirs.

| Parameters    | Type                     | Description                                  | Default   |
|---------------|--------------------------|----------------------------------------------|-----------|
| input_size    | `int`                    | Size of input vectors                        |           |
| size          | `int`                    | Number of units in the reservoir             |           |
| leaky         | `float`                  | Leaky parameter of the reservoir             |           |
| fn            | `Callable`               | Activation function of the reservoir         | `np.tanh` |
| input_bias    | `bool`                   | Enable the usage of a bias in the input      | `True`    |
| input_init    | `esnpy.init.Initializer` | Define how to initialize the input weights   |           |
| input_tuners  | `list[esnpy.tune.Tuner]` | Define how to tune the input weights         | `[]`      |
| intern_init   | `esnpy.init.Initializer` | Define how to intialize the internal weights |           |
| intern_tuners | `list[esnpy.init.Tuner]` | Define how to tune the internal weights      | `[]`      |

#### `Initializer` and `Tuner` 

`esnpy.init.Initializer` and `esnpy.tune.Tuner` are the abstract base classes used to setup the input and internal weights of a reservoir.

`Initializer` is defined by a `init() -> Matrix` function. 
`esnpy` provides implementations of initializer for both uniform and gaussian distribution of weights, and for both dense and sparse matrix.

`Tuner` is defined by a `init(matrix : Matrix) -> Matrix` function, which can be used to modify the weights after initialization.
For example, `esnpy` provides a `SpectralRadiusTuner` to change the spectral radius of a weights matrix.

#### `Trainer`

`esnpy.train.Trainer` is responsible to create the output weights matrix from the training data and targets.  
It is defined by a `train(inputs: Matrix, data: Matrix, target: Matrix) -> Matrix` function.

`esnpy` provides a `RidgeTrainer` to compute the output weights using a ridge regression. 
This trainer has three parameters : one float, the regularization parameter's weight `alpha`, and two optionals boolean (default to true) `use_bias` and `use_input` to control if we should use a bias and the input to compute the readout weights.

## Code Examples 

Want to see some code in action ? Take a look at the `examples/` directory:
- `MackeyGlass/` demonstrates how to learn to predict a time series,
- `TrajectoryClassification/` demonstrates how to learn to classify 2D trajectories.

## Bibliography

Based on:
- *The "echo state" approach to analysing and training recurrent neural networks* by Herbert Jaeger ([pdf](https://www.ai.rug.nl/minds/uploads/EchoStatesTechRep.pdf)),
- *A pratical guide to applying Echo State Networks* by Mantas Lukoševičius ([pdf](https://www.ai.rug.nl/minds/uploads/PracticalESN.pdf)),
- *Design of deep echo state networks* by Claudio Gallicchio and al ([link](https://www.sciencedirect.com/science/article/pii/S0893608018302223)),
- *Deep echo state network (DeepESN): A brief survey* by Claudio Gallicchio and Alessio Micheli ([pdf](https://arxiv.org/pdf/1712.04323.pdf)).

Special thanks to Mantas Lukoševičius for his [minimal ESN example](https://mantas.info/wp/wp-content/uploads/simple_esn/minimalESN.py), which greatly helped me to get started with reservoir computing.
