Metadata-Version: 2.1
Name: linfa_vi
Version: 1.3.9
Summary: A Python library for inference with normalizing flow and annealing
Author-email: resDesLab  <daniele.schiavazzi@gmail.com>
License: Copyright © 2023 <copyright holders>
        
        Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the “Software”), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:
        
        The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.
        
        THE SOFTWARE IS PROVIDED “AS IS”, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
Project-URL: Homepage, https://github.com/desResLab/LINFA
Keywords: variational inference,normalizig flow,adaptive annealing,physics-based modeling
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python
Classifier: Programming Language :: Python :: 3
Requires-Python: >=3.7
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: torch==1.13.1
Requires-Dist: numpy
Requires-Dist: matplotlib
Requires-Dist: tomli; python_version < "3.11"
Provides-Extra: dev
Requires-Dist: black; extra == "dev"
Requires-Dist: bumpver; extra == "dev"
Requires-Dist: isort; extra == "dev"
Requires-Dist: pip-tools; extra == "dev"
Requires-Dist: pytest; extra == "dev"

[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT) 
 ![example workflow](https://github.com/desResLab/LINFA/actions/workflows/test_publish_pypi.yml/badge.svg) 
[![Documentation Status](https://readthedocs.org/projects/linfa-vi/badge/?version=latest)](https://linfa-vi.readthedocs.io/en/latest/?badge=latest)

## LINFA

LINFA is a library for variational inference with normalizing flow and adaptive annealing. It is designed to accommodate computationally expensive models and difficult-to-sample posterior distributions with dependent parameters.

The code for the masked autoencoders for density estimation (MADE), masked autoregressive flow (MAF) and real non volume-preserving transformation (RealNVP) is based on the implementation provided by [Kamen Bliznashki](https://github.com/kamenbliznashki/normalizing_flows). 

### Installation

To install LINFA type

```
pip install linfa-vi
```

### Documentation 

The documentation can be found on [readthedocs](https://linfa-vi.readthedocs.io/en/latest/)

### References

Background theory and examples for LINFA are discussed in the two papers: 

- Y. Wang, F. Liu and D.E. Schiavazzi, *[Variational Inference with NoFAS: Normalizing Flow with Adaptive Surrogate for Computationally Expensive Models](https://www.sciencedirect.com/science/article/abs/pii/S0021999122005162)*
- E.R. Cobian, J.D. Hauenstein, F. Liu and D.E. Schiavazzi, *[AdaAnn: Adaptive Annealing Scheduler for Probability Density Approximation](https://www.dl.begellhouse.com/journals/52034eb04b657aea,796f39cb1acf1296,6f85fe1149ff41d9.html?sgstd=1)*

### Requirements

* PyTorch 1.13.1
* Numpy 1.22
* Matplotlib 3.6 (only plot functionalities `linfa.plot_res`)

### Numerical Benchmarks

LINFA includes five numerical benchmarks:

* Trivial example.
* High dimensional example (Sobol' function).
* Two-element Windkessel model (a.k.a. RC model).
* Three-element Windkessel model (a.k.a. RCR model).
* Friedman 1 dataset example.

The implementation of the lumped parameter network models (RC and RCR models) follows closely from the code developed by [the Schiavazzi Lab at the University of Notre Dame](https://github.com/desResLab/supplMatHarrod20).

To run the tests type
```sh
python -m unittest linfa.linfa_test_suite.NAME_example
```
To run a limited number of iterations (say 100), you can instead type
```sh
it=100 python3 -m unittest linfa.linfa_test_suite.NAME_example
```
where `NAME` need to be replaced by
* `trivial` for the trivial example (Ex 1).
* `highdim` for the high-dimensional example (Ex 2).
* `rc` for the RC model (Ex 3).
* `rcr` for the RCR model (Ex 4).
* `adaann` for the Friedman model example (Ex 5).
* `rcr_nofas_adaann` for the RCR model, combining NoFAS with adaptive annealing (AdaAnn)

If used with adaptive annealing (AdaAnn) the `it=100` option runs 100 iterations only at `T=1` (i.e., to approximate the untempered posterior distribution). Therefore the total number of iterations is generally higher than specified through the `it` option.

At regular intervals set by the parameter `experiment.save_interval` LINFA writes a few results files. The sub-string `NAME` refers to the experiment name specified in the `experiment.name` variable, and `IT` indicates the iteration at which the file is written. The results files are

* `log.txt` contains the log profile information, i.e.
  * Iteration number.
  * Annealing temperature at each iteration.
  * Loss function at each iteration.
* `NAME_grid_IT` contains the inputs where the true model was evaluated. 
* `NAME_params_IT` contains the batch of input parameters $\boldsymbol{z}_{K}$ in the physical space generated at iteration `IT`. 
* `NAME_samples_IT` contains the batch of normalized parameters (parameter values before the coordinate transformation) generated at iteration `IT`.
* `NAME_logdensity_IT` contains the value of the log posterior density corresponding to each parameter realization. 
* `NAME_outputs_IT` contains the true model (or surrogate model) outputs for each batch sample at iteration `IT`.
* `NAME_IT.nf` contains a backup of the normalizing flow parameters at iteration `IT`.

A post processing script is also available to plot all results. To run it type

```sh
python -m linfa.plot_res -n NAME -i IT -f FOLDER
```
where `NAME` and `IT` are again the experiment name and iteration number corresponding to the result file of interest, while `FOLDER` is the name of the folder with the results of the inference task are kept. Also the file format can be specified throught the `-p` option (options: `pdf`, `png`, `jpg`) and images with dark background can be generated using the `-d` flag. 

### Usage

To use LINFA with your model you need to specify the following components:

* A computational model.
* A surrogate model.
* A log-likelihood model.
* An optional transformation. 

In addition you need to specify a list of options as discussed in the [documentation](https://linfa-vi.readthedocs.io/en/latest/content/linfa_options.html).

### Tutorial

Two step-by-step tutorials ([tutorial 1](tutorial/tutorial_linfa_2d.ipynb) and [tutorial 2](tutorial/tutorial_linfa_3d.ipynb)) are also available which will guide you through the an inference problem for a ballistic simulation.

### Citation

Did you use LINFA? Please cite our paper using:
```
@article{linfa-vi-paper,
  title={LINFA: a Python library for variational inference with normalizing flow and annealing},
  author={Wang, Yu and Cobian, Emma R and Lee, Jubilee and Liu, Fang and Hauenstein, Jonathan D and Schiavazzi, Daniele E},
  journal={arXiv preprint arXiv:2307.04675},
  year={2023}
}
```
