Metadata-Version: 2.1
Name: nowcastlib
Version: 3.0.12
Summary: 🧙🔧 Utils that can be reused and shared across and beyond the ESO Nowcast project
Home-page: https://giuliostarace.com/nowcastlib/
License: GPL-3.0-only
Keywords: nowcast,turbulence,forecasting,data-science,time series
Author: Giulio Starace
Author-email: giulio.starace@gmail.com
Requires-Python: >=3.7.1,<4.0
Classifier: License :: OSI Approved :: GNU General Public License v3 (GPLv3)
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.8
Classifier: Programming Language :: Python :: 3.9
Requires-Dist: ConfigArgParse (>=1.4.1,<2.0.0)
Requires-Dist: attrs (>=21.2.0,<22.0.0)
Requires-Dist: cattrs (>=1.7.1,<2.0.0)
Requires-Dist: importlib-metadata (>=4.6.3,<5.0.0)
Requires-Dist: matplotlib (>=3.4.2,<4.0.0)
Requires-Dist: numpy (>=1.20.3,<2.0.0)
Requires-Dist: pandas (>=1.2.4,<2.0.0)
Requires-Dist: scikit-learn (>=0.24.2,<0.25.0)
Requires-Dist: skyfield (>=1.39,<2.0)
Project-URL: Repository, https://github.com/thesofakillers/nowcastlib
Description-Content-Type: text/markdown

# Nowcast Library

🧙‍♂️🔧 Utils that can be reused and shared across and beyond the ESO Nowcast
project

This is a public repository hosted on GitHub via a push mirror setup in the
[internal ESO GitLab repository](https://gitlab.eso.org/gstarace/nowcastlib/)

## Installation

Simply run

```console
pip install nowcastlib
```

## Usage and Documentation

Nowcast Library (nowcastlib) consists in a collection of functions organized in
submodules (API) and a tool accessible via the command line (CLI). The latter is
primarily intended for accessing the Nowcast Library Pipeline, an opinionated
yet configurable set of processing steps for wrangling data and evaluating
models in a consistent and rigorous way. More information can be found on the
nowcastlib pipeline index page
([link to markdown](nowcastlib/pipeline/README.md) and
[link to hosted docs](https://giuliostarace.com/nowcastlib/pipeline))

Please refer to the
[examples folder](https://github.com/thesofakillers/nowcastlib/tree/master/examples)
on GitHub for usage examples.

### API

Here is a quick example of how one may import nowcastlib and access to one of
the functions:

```python
"""Example showing how to access compute_trig_fields function"""
import nowcastlib as ncl
import pandas as pd
import numpy as np

data_df = pd.DataFrame(
    np.array([[0, 3, 4, np.NaN], [32, 4, np.NaN, 4], [56, 8, 0, np.NaN]]).T,
    columns=["A", "B", "C"],
    index=pd.date_range(start="1/1/2018", periods=4, freq="2min"),
)

result = ncl.datasets.compute_trig_fields(data_df, ["A", "C"])
```

More in-depth API documentation can be found
[here](https://giuliostarace.com/nowcastlib/).

### CLI

Some of the library's functionality is bundled in configurable subcommands
accessible via the terminal with the command `nowcastlib`:

```console
usage: nowcastlib [-h] [-v]
                  {triangulate,preprocess,sync,postprocess,datapipe} ...

positional arguments:
  {triangulate,preprocess,sync,postprocess,datapipe}
                        available commands
    triangulate         Run `nowcastlib triangulate -h` for further help
    preprocess          Run `nowcastlib preprocess -h` for further help
    sync                Run `nowcastlib sync -h` for further help
    postprocess         Run `nowcastlib postprocess -h` for further help
    datapipe            Run `nowcastlib datapipe -h` for further help

optional arguments:
  -h, --help            show this help message and exit
  -v, --verbose         increase verbosity level from INFO to DEBUG
```

### Repository Structure

The following output is generated with `tree . -I 'dist|docs|*.pyc|__pycache__'`

```bash
.
├── LICENSE
├── Makefile # currently used to build docs
├── README.md
├── de421.bsp # not committed
├── docs/ # html files for the documentation static website
├── examples
│   ├── README.md
│   ├── cli_triangulate_config.yaml
│   ├── data/  # not committed
│   ├── datasync.ipynb
│   ├── output/ # not committed
│   ├── pipeline_datapipe.json
│   ├── pipeline_preprocess.json
│   ├── pipeline_sync.json
│   ├── signals.ipynb
│   └── triangulation.ipynb
├── images
│   └── pipeline_flow.png
├── nowcastlib # the actual source code for the library
│   ├── __init__.py
│   ├── cli
│   │   ├── __init__.py
│   │   └── triangulate.py
│   ├── datasets.py
│   ├── dynlag.py
│   ├── gis.py
│   ├── pipeline
│   │   ├── README.md
│   │   ├── __init__.py
│   │   ├── cli.py
│   │   ├── process
│   │   │   ├── __init__.py
│   │   │   ├── postprocess
│   │   │   │   ├── __init__.py
│   │   │   │   ├── cli.py
│   │   │   │   └── generate.py
│   │   │   ├── preprocess
│   │   │   │   ├── __init__.py
│   │   │   │   └── cli.py
│   │   │   └── utils.py
│   │   ├── split
│   │   │   └── __init__.py
│   │   ├── structs.py
│   │   ├── sync
│   │   │   ├── __init__.py
│   │   │   └── cli.py
│   │   └── utils.py
│   ├── signals.py
│   └── utils.py
├── poetry.lock # lock file generated by python poetry for dependency mgmt
└── pyproject.toml # general information file, handled by python poetry
```

#### Directories and Files not Committed

There are a number of files and folders that are not committed due to their
large and static nature that renders them inappropriate for git version control.
The following files and folder warrant a brief explanation.

-   Certain functions (time since sunset, sun elevation) of the Nowcast Library
    rely on the use of a .bsp file, containing information on the locations
    through time of various celestial bodies in the sky. This file will be
    automatically downloaded upon using one of these functions for the first
    time.
-   The examples scripts make use of a `data/` directory containing a series of
    csv files. Most of the data used in the examples can be downloaded from the
    [ESO Ambient Condition Database](http://archive.eso.org/cms/eso-data/ambient-conditions.html).
    Users can then change the paths set in the examples to fit their needs. For
    users interested in replicating the exact structure and contents of the data
    directory, a compressed copy of it (1.08 GB) is available to ESO members
    through
    [this Microsoft Sharepoint link](https://europeansouthernobservatory.sharepoint.com/:u:/t/OpticalTurbulenceandWeatherNowcast/EeH844GlBgdBjc63uaPiO4ABrh7ylH54zH3dJV9WSIhakA?e=gPBWJ7).
-   At times the examples show the serialization functionality of the nowcastlib
    pipeline or need to output some data. In these situations the `output/`
    directory in the examples folder is used.

## Development Setup

This repository relies on [Poetry](https://python-poetry.org/) for tracking
dependencies, building and publishing. It is therefore recommended that
developers [install poetry](https://python-poetry.org/docs/#installation) and
make use of it throughout their development of the project.

### Dependencies

Make sure you are in the right Python environment and run

```console
poetry install
```

This reads [pyproject.toml](./pyproject.toml), resolves the dependencies, and
installs them.

### Deployment

The repository is published to [PyPi](https://pypi.org/), so to make it
accessible via a `pip install` command as mentioned [earlier](#install).

To publish changes follow these steps. Ideally this process is automated via a
CI tool triggered by a push/merge to the master branch:

1. Optionally run
   [`poetry version`](https://python-poetry.org/docs/cli/#version) with the
   appropriate argument based on [semver guidelines](https://semver.org/).

2. Update the documentation by running

    ```console
    make document
    ```

3. Prepare the package by running

    ```console
    poetry build
    ```

4. Ensure you have [TestPyPi](https://test.pypi.org/) and PyPi configured as
   your poetry repositories:

    ```console
    poetry config repositories.testpypi https://test.pypi.org/legacy/
    poetry config repositories.pypi https://pypi.org/
    ```

5. Publish the repository to TestPyPi, to see that everything works as expected:

    ```console
    poetry publish -r testpypi
    ```

6. Stage, commit and push your changes (to master) with git.
7. Publish the repository to PyPi:

    ```console
    poetry publish -r pypi
    ```

Upon successful deployment, the library should be available for install via
`pip`

