Metadata-Version: 2.1
Name: datalake-catalog
Version: 1.0.4
Summary: Datalake Catalog
Home-page: https://github.com/equancy/datalake-catalog
License: MIT
Author: Didier SCHMITT
Author-email: dschmitt@equancy.com
Requires-Python: >=3.8,<4.0
Classifier: Development Status :: 4 - Beta
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.8
Classifier: Programming Language :: Python :: 3.9
Classifier: Topic :: Internet :: WWW/HTTP :: WSGI :: Application
Provides-Extra: all
Provides-Extra: mysql
Provides-Extra: pgsql
Requires-Dist: Flask (>=2.1.2,<3.0.0)
Requires-Dist: Flask-JWT-Extended (>=4.4.0,<5.0.0)
Requires-Dist: PyMySQL (>=1.0.2,<2.0.0); extra == "all" or extra == "mysql"
Requires-Dist: click (>=8.1.3,<9.0.0)
Requires-Dist: gunicorn (>=20.1.0,<21.0.0)
Requires-Dist: jsonschema (>=4.5.1,<5.0.0)
Requires-Dist: pony (==0.7.16)
Requires-Dist: psycopg2-binary (>=2.9.3,<3.0.0); extra == "all" or extra == "pgsql"
Project-URL: Repository, https://github.com/equancy/datalake-catalog
Description-Content-Type: text/markdown

## Setup and start the API 

Configure the parameters with a [python file](https://flask.palletsprojects.com/en/2.0.x/config/#configuring-from-python-files) 

For example, `catalog.conf.py`

```python
SECRET_KEY = b"changemenow"
DB_STRING = "sqlite://localhost/catalog.sqlite"
```

Start the catalog 

```shell
docker run -d \
    -p '8080:8080' \
    -v 'catalog.conf.py:/etc/datacatalog/catalog.conf.py' \
    -e 'CATALOG_SETTINGS=/etc/datacatalog/catalog.conf.py' \
    public.ecr.aws/equancy-tech/datalake-catalog
```

## Generate an API token

The `catalog create-api-key` generated tokens that can be used with restricted endpoint.

- **admin** role can access all restricted endpoints
- **author** role can only access the restricted endpoints in `/catalog`

```shell
Usage: catalog create-api-key [OPTIONS] NAME

Options:
  -e, --expires INTEGER      number of days before token expires
  -r, --role [author|admin]  the role associated with the name
  --help                     Show this message and exit.
```

## Update the storages configuration

Storage configures aliases to help resolve actual buckets (S3, Azure, local fs, etc.).
It consists of a bucket name and an optional prefix.

For example:

```json
{
    "landing": {
        "bucket": "my-raw-bucket",
        "prefix": "landing"
    },
    "archive": {
        "bucket": "my-raw-bucket",
        "prefix": "archives"
    },
    "bronze": {
        "bucket": "my-bucket-bronze"
    },
    "silver": {
        "bucket": "my-bucket-silver"
    },
    "gold": {
        "bucket": "my-bucket-gold",
    }
}
```

```shell
curl -XPUT http://localhost:8080/storage \
    -H "Content-Type: application/json" \
    -H "Authorization: Bearer ${CATALOG_ADMIN_TOKEN}" \
    --data "@${STORAGE_CONFIG_FILE}" 
```

