Metadata-Version: 2.1
Name: spork-cli
Version: 1.0.11
Summary: CLI For Launching Experiments Using Singularity On Slurm
Home-page: https://github.com/brandontrabucco/spork
Author: Brandon Trabucco
Author-email: brandon@btrabucco.com
License: MIT
Download-URL: https://github.com/brandontrabucco/spork/archive/v1_0_11.tar.gz
Description: # Spork: Helping You Run Slurm Jobs
        
        Spork is a quality of life package that enables rapid deployment of code on a slurm cluster with singularity installed and password enabled login. Running experiments on your cluster is as simple as a single command in the terminal using spork. See below for an example and install instructions.
        
        ## Installation
        
        Spork can be installed using the pip package.
        
        ```bash
        pip install spork-cli
        ```
        
        ## Usage
        
        You may configure spork to remember the ssh credentials to your cluster using the following example.
        
        ```bash
        spork set --ssh-username username --ssh-password password --ssh-host compute.example.com
        ```
        
        Running your first command on your cluster is then as simple as one line in the terminal.
        
        ```bash
        spork remote echo "my first command"
        ```
        
        Certain workloads require uploading certain data files into the singularity image on the remote machine before running experiments. This can be done with the following command.
        
        ```bash
        spork upload --recursive --exclude "*.pkl" ./local_dir remote_dir/in/image
        ```
        
        Additionally, you can download files from the remote machine with a single command. The following command will download the results folder inside the remote singularity image to a location in the current local working directory. The remote path is always taken with respect to the singularity image path.
        
        ```bash
        spork download --recursive --exclude "*.pkl" results ./
        ```
        
        ## Experimentation
        
        A typical experiment creation pipeline involved working on code locally, testing the code locally, then running in on a server. This can be done easily using spork.
        
        ```bash
        python do_my_experiment.py
        ```
        
        Once your code is ready for deployment, tell spork how to install your code from github.
        
        ```bash
        spork set --post-env-commands "git clone https://github.com/username/repo /code/repo" \
                  --post-env-commands "pip install -e /code/repo"
        ```
        
        Then, point spork to where your code working directory is stored locally.
        
        ```bash
        spork set --sync-with ./ --sync-target /code/repo --exclude-from-sync "*.sif"
        ```
        
        Then test your code in a local copy of the singularity to make sure it works as expected.
        
        ```bash
        spork local python /code/repo/do_my_experiment.py
        ```
        
        Then run it on the cluster.
        
        ```bash
        spork remote python /code/repo/do_my_experiment.py
        ```
        
        In this example, spork will automatically copy code from your repository on the local disk to code folder in your remote singularity image. This is especially helpful when changes aren't committed.
        
        
Keywords: Deep Learning,Research,Management
Platform: UNKNOWN
Classifier: Intended Audience :: Developers
Classifier: Intended Audience :: Science/Research
Classifier: Topic :: Scientific/Engineering
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Classifier: Topic :: Scientific/Engineering :: Mathematics
Classifier: Topic :: Software Development
Classifier: Topic :: Software Development :: Libraries
Classifier: Topic :: Software Development :: Libraries :: Python Modules
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.7
Classifier: Programming Language :: Python :: 3.8
Description-Content-Type: text/markdown
