Metadata-Version: 2.1
Name: sagecreator
Version: 0.1.1.4
Summary: Package to orchestrate architecture in AWS
Home-page: https://github.com/romanjoffee/sagecreator
Author: romanjoffee
Author-email: roman.jugai@gmail.com
License: UNKNOWN
Description: |build-status| |coverage-status| |pypi-reference| |pypi-downloads|
        
        SageCreator
        ===========
        
        | SageCreator is a package meant to simplify cluster setup for Machine Learning in AWS.
        | It does all the heavy lifting to get cluster up and running in a matter of minutes on any of the AWS instance type(s).
        | It uses **spot instances** by default which can significantly reduce total cost of running the cluster.
        | If spot instances are not available or the specified spot instance price is too low, it falls back to **on-demand** instances.
        |
        | You can access **Jupyter notebook** that can run your code against provisioned server(s). See `Jupyter access`_ for more info.
        
        Installation
        ------------
        
        Install and update using `pip`_:
        
        .. code-block:: text
        
            $ pip install sagecreator
        
        Python 3 is required and it is highly recommended to install and run the package in `virtualenv`_.
        Supported in Python >= 3.5.0.
        
        .. _pip: https://pip.pypa.io/en/stable/quickstart/
        
        .. _virtualenv: https://virtualenv.pypa.io/en/stable/
        
        
        Prerequisites
        -------------
        
        **AWS Account**
        
        To provision the cluster you need an `AWS Account`_ and an IAM user with:
        
        - Access Key ID
        - Secret Access Key
        
        User should either be in **Administrators** group as described in `IAM user`_ tutorial, or create a `custom IAM policy`_
        
        Execution
        ---------
        
        After the installation, configure the tool by specifying configuration parameters:
        
        .. code-block:: text
        
            $ sage configure
            Access key id: <AWS Access Key ID>
            Secret access key: <AWS Secret Access Key>
            Company: <Name of your organization>
            Owner: <Name of your team>
            Key pair name: <Name of the key pair> (Optional - if NOT provided it will be created with a new private key)
            Private key file: <Absolute path to private key file> (required only if Key pair name was provided)
        
        | **Company**, **Owner**, **Service** are required - those are used as tags for each instance in the cluster.
        | **Key pair name**, **Private key file** are optional - if provided, given 'key pair name' / 'private key file' will be used to provision the cluster.
        
        ---------
        
        | Provision the cluster.
        | Provision step can take up to 20 minutes depending on the network connection, cluster size, and instance type.
        
        .. code-block:: text
        
            $ sage provision
            Service: <Name of your service>
            Instance type [t3.small]: <Instance type> (Optional, defaults to t3.small)
            Spot instance price [0.1]: <Spot instance price> (Optional, defaults to $0.1 per instance)
            Cluster size [1]: <Cluster size> (Optional, defaults to 1 node)
        
        .. image:: https://s3.amazonaws.com/romanjoffee/github/sagecreator/provision1080.gif
        
        | **Important**:
        | The tool provides NO guarantee that the instance(s) will be provisioned at specified **Spot instance price**.
        | If specified price is lower than the current AWS spot instance price then **On-demand** instance(s) will be provisioned instead.
        | Thus, it is up to the user to ensure that specified price is high enough for the request to be fulfilled.
        
        ---------
        
        | Display path of the cluster configuration file.
        | Though not necessary it is possible to manually edit that file with customizations prior to running **provision** step.
        
        .. code-block:: text
        
            $ sage pwd
        
        ---------
        
        | Terminate cluster. This operation terminates all cluster nodes matching tags tuple of **Company**, **Owner**, **Service**.
        
        .. code-block:: text
        
            $ sage terminate
            Service: <Name of your service to terminate>
        
        .. _Jupyter access:
        
        Jupyter access
        --------------
        
        | Once provisioning step is done and the cluster is up you can access jupyter notebook in your browser at http://localhost:9000.
        | We have provided a sample notebook to execute. It trains the model on Fashion MNIST dataset using CNN in Keras.
        
        Under the hood
        --------------
        
        | The logic that orchestrates the cluster is written in `Ansible`_
        
        .. _custom IAM policy:
        
        Custom IAM policy
        -----------------
        
        Alternatively, instead of assigning user to **Administrators** group which has access to all AWS services (as described in `IAM user`_), you can create separate Group named **Provisioners** with more restrictive policy:
        
        .. code-block:: text
        
            {
              "Version": "2012-10-17",
              "Statement": [
                {
                  "Action": "ec2:*",
                  "Effect": "Allow",
                  "Resource": "*"
                },
                {
                  "Action": "rds:*",
                  "Effect": "Allow",
                  "Resource": "*"
                },
                {
                  "Action": "route53:*",
                  "Effect": "Allow",
                  "Resource": "*"
                }
              ]
            }
        
        Then assign the user to the **Provisioners** group which has access to a subset of AWS services that are sufficient to orchestrate the cluster.
        
        
        SSH access
        ----------
        
        | If **Key pair name** / **Private key file** were NOT provided when configuring the cluster then default key pair is created and a new private key is stored locally.
        | In order to ``ssh`` into the servers point ``ssh`` to the correct (private key) file:
        
        .. code-block:: text
        
            $ ssh -i <path to private key file> ubuntu@<host>
        
        where *path to private key file* is ``../venv/lib/python3.X/site-packages/sagebase/.ssh/pkey.pem``
        
        
        .. |build-status| image:: https://travis-ci.com/romanjoffee/sagecreator.svg?branch=master
            :target: https://travis-ci.com/romanjoffee/sagecreator
        
        .. |coverage-status| image:: https://codecov.io/gh/romanjoffee/sagecreator/branch/master/graphs/badge.svg
           :target: https://codecov.io/gh/romanjoffee/sagecreator/branch/master
        
        .. |pypi-reference| image:: https://badge.fury.io/py/sagecreator.svg
           :target: https://badge.fury.io/py/sagecreator
        
        .. |pypi-downloads| image:: https://pepy.tech/badge/sagecreator
           :target: https://pepy.tech/project/sagecreator
        
        .. _`AWS Account`: https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/get-set-up-for-amazon-ec2.html#sign-up-for-aws
        
        .. _`IAM User`: https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/get-set-up-for-amazon-ec2.html#create-an-iam-user
        
        .. _`Ansible`: https://www.ansible.com
        
Platform: UNKNOWN
Classifier: Programming Language :: Python :: 3
Classifier: License :: OSI Approved :: Apache Software License
Classifier: Operating System :: OS Independent
Description-Content-Type: text/x-rst; charset=UTF-8
