Metadata-Version: 2.1
Name: ftp-loader
Version: 0.2.2
Summary: A package to download project data files from FTP
Home-page: https://github.com/rorni/ftp-loader
Author: Roman Rodionov
License: UNKNOWN
Platform: UNKNOWN
Classifier: Programming Language :: Python :: 3
Classifier: License :: OSI Approved :: MIT License
Requires-Python: >=3.5
Description-Content-Type: text/markdown
License-File: LICENSE.txt

# FTP Data Loader

ftp-loader is a tool to download from and upload data files to FTP server.
It is intended to use in projects with large data files. Data files are bad 
candidates for version control managment systems. They are better to be placed
at FTP server and be downloaded on demand. 

The suggested workflow is the following. Data files are placed in folders added
to gitignore. ftp-config.toml file is used to maintain list of data files and 
their locations both at project folder and FTP. This is default name, but other
\*.toml files can be used to split file index into separate parts. Files can
be compressed.

## Installation

`pip install ftp-loader`

## Usage

`ftp-loader -h`

   Shows help.

`ftp-loader [--overwrite] [ftp-config.toml]`

   Downloads and extracts data from FTP server. Index file name is optional.
   Default index file - ftp-config.toml.

`ftp-loader --upload [--overwrite] [ftp-config.toml]`

   Compresses and uploads data to FTP server. Index file name is optional. 
   Default index file - ftp-config.toml.

`--overwrite` Option instructs to overwrite existing files.

`--check FTP_URL` Checks user's initial path at ftp server.

`--base-path BASE_PATH` Specifies initial path at ftp. 


## Configuration

When user logs in its path can be different from the path to FTP root. In this case
path to FTP's root must be specified. This can be done in two ways.

1. By specifying --base-path parameter in command line.

2. By setting base path in the configuration file. The configuration file must have
   name `.ftp-loader-config.json` located at the user's home directory. 
   It must contain "hosts" key with a dictionary of hosts and corresponding paths.
   For example:
   {
       "hosts": {
           "server.ftp.ru": "/home/path1"
       }
   }

## Index file format

Index file must contain the following parameters:


1. FTP server URL.

    `url = "server.ftp.ru"`

2. Path to project's folder at FTP. For now only Unix-style is supported.

    `path = "/projects/test-data"`

3. List of files to be transferred. It is a list of file groups. Each group
   contain 3 or 4 parameters:
   ```
   [[files]]
   dst = "work"    # Destination folder name.
   src = "storage" # Source folder name relative to 'path'.
   arch = "bz2"    # Optional. Archive type. Supported archive formats:
                   # gz, bz2
   names = [       # list of file names.
       file1.txt,
       file2.csv
   ]
   ```

   Every group of files starts with `[[files]]` header. The number of groups 
   is arbitrary.

Example of index file can be found in tests folder - ftp-config.toml.



