
.. Welcome!
.. As you can see, these are comments: they start with two dots and a space
.. Sphinx is very sensitive to spaces, empty lines, etc. so it can sometimes be frustrating
.. Two dots and a space are also used for special tagging, inclusion, etc.  Like here, where we are creating an internal link:

.. _about:

.. So, lets start writing the documentation
.. Title fonts are written like this:

Tutorial - start here
=====================

Well, this module does actually nothing.

It's a cute_mongo_forms module that servers as a template for python(3) projects:

.. Let's add an image.  Important note here: always when adding a tag with ".." remember to leave a blank line before it!

.. image:: images/skull.png
   :width: 10 %
   
   
Some features:

  * Autogenerated documentation
  * Python packaging
  * Development and production downloads from git
  
.. _impatient:
  
For the impatient
-----------------

To start developing your own project ASAP, do the following:

Let's assume your development python packages are in **~/python3_packages**.  Your pythonpath is **$PYTHONPATH**.  Your name is *Janne Jantunen* and you have decided to create a new package named *cute_mongo_forms*.  Proceed like this:

::

  cd ~/python3_packages
  git clone https://github.com/elsampsa/cute_mongo_forms
  mv cute_mongo_forms cute_mongo_forms
  cd cute_mongo_forms
  ./reinit.bash
  ./setauthor.bash "Janne Jantunen"
  ./setver.bash "0.1"
  ln -s $PWD/cute_mongo_forms $PYTHONPATH/

  
Mod the documentation by editing .rst files in the "docs/" directory.  The idea is that you hide/delete this page ("tutorial.rst") from your own module.

After this you can re-compile and check the documentation with

::

  cd docs
  ./compile.bash
  firefox index.html

  
If you want to push your newly minted python project into a git repo, do:

::
  
  git init
  git remote add origin https://[your-personal-git-repository]/cute_mongo_forms.git
  git add *
  git commit -m "initial commit"
  git push -u origin master
  
Things to keep in mind:

  * The name of your package (that is also the name of your python module) should not contain "weird characters", i.e. ".,-" etc.
  * Remember to add git tags with (here we create tag "v1.2")
  
  ::
  
    git tag v1.2 -m "cool version v1.2"
    git push origin --tags

  * ..now production users can install with pip(3) certain revisions of the software (see :ref:`here <production>`)
    
  
  
Dependencies
------------

Keep up with your dependencies!

Edit the setup.py and there, the "install_requires" field.  This way, when installing your package with pip3, depencencies are automagically resolved.  Keep them up to date also in the file *docs/snippets/requirements.txt*.


Files
-----

This page has been produced with the file "docs/tutorial.rst".  Go ahead and open it in your editor.  Open also "index.rst".  Edit them accordingly.  Said that, let's take a closer look at the directory structure:

* cute_mongo_forms/ : This is a "scaffold" directory, a necessary evil for packaging

  ================   ===============================================================================================================================
  README.md          Readme file in online markdown format (nice for github). Play around with online markdown `here <https://stackedit.io/editor>`_
  LICENSE.*          License text (say, LICENSE.LGPL3, LICENSE.MIT, etc.)
  CHANGELOG          Recent changes in the code
  setup.py           Python setup script for creating packages
  MANIFEST.in        used by setup.py
  changestr.bash     Helper script to change strings in your package
  setver.bash        Helper script to change the version of your package
  setauthor.bash     Helper script to change the author of the package
  reinit.bash        Reinitializes the package name
  cleanpy.bash       Removes __pycache__ directories and .pyc files; Do maybe before "git add ."
  ================   ===============================================================================================================================
|
* cute_mongo_forms/docs/  : Documentation and autogenerated documentation lives here
  
  ================   ===============================================================================================================================
  index.html         Redirects to the autogenerated html [don't touch]
  index.rst          The main index start from here
  tutorial.rst       The file you are staring at the moment (or html version of it) 
  intro.rst          Introduction of the module
  requirements.rst   What the user needs in order to use this module 
  examples.rst       Copy-pastable examples for the api user 
  submodules.rst     Documentation generated automatically from source code.  Just add a new entry here for each submodule, don't touch otherwise 
  license.rst        Copyright and License 
  authors.rst        Who is the maintainer & author
  compile.bash       Run this always after modifying your documents / source code [don't edit]
  clean.bash         Clean autogenerated documentation [don't edit]
  conf.py            Sphinx configuration file [maybe don't edit]
  .nojekyll          Dummy file.  This is needed for the on-line documentation to work with GitHub [don't touch]
  snippets/          Example snippet source files and scripts for generating pages from them.  Edit also the file requirements.txt here.
  generated/         Auto-generated documents [don't touch]
  ================   ===============================================================================================================================
|
* cute_mongo_forms/cute_mongo_forms : This is where the actual python module lives
  
  ================   ===============================================================================================================================
  __init__.py        Python module initialization
  greeters/          Submodule
  data/              Static data the module needs and that is included in the python package
  ================   ===============================================================================================================================
  

The scaffolding "double-directory" cute_mongo_forms/cute_mongo_forms structure might seem inconvenient, but it is necessary for packaging.  Here is what we just did in the :ref:`For the impatient <impatient>` section:

* Keep "scaffolded" python modules in, say "~/python3_packages"
* Link the python module directories to your $PYTHONPATH

It's time to start documenting!  Edit the files "docs/*.rst".  Here are `some nice tips <http://www.sphinx-doc.org/en/stable/rest.html>`_ for using Sphinx and here `are some more <http://www.sphinx-doc.org/en/stable/markup/inline.html>`_

To create and to recreate the docs (after changing the code, etc.), do (in the "docs/" directory):
  
::

  ./compile.bash
  
To see your documentation, launch

:: 

  firefox index.html
  

Online autodocumentation
------------------------
  
**Github users**
  
After creating the git repo, create also a site for your project like this:  Settings => GitHub pages => Source : master branch / docs folder => press Save.  Now your documentation is online in github! (don't forget to include the "docs/" folder into git).
    
**Gitlab users**
  
Using a generic service that uses a gitlab server?  No problem.  There is a simple hack to put the autodocs online.

  * In gitlab, create a wiki page for your project.  Write there the words "Hello world".  Next, clone the wiki repository (not *the* repository, just the *wiki repository*) with that button on the right that says "clone repository".
  * Edit your projects "docs/compile.bash".  Uncomment the options for Gitlab and set the directory of your wiki repository correctly.  Run "./compile.bash".  Now the whole documentation tree has been copied to your wiki repo directory.
  * In the wiki repo directory, open "home.md" in an editor.  Modify it to look like this:
  
  ::
  
    For documentation
    [click here](_build/html/index.html)
    
  * You still need to use git to add and push all the files to the wiki repository online (there are instructions in "compile.bash")

    
Organizing large python projects
--------------------------------

Let's say we have a python project that consists of "entities" and those consist of submodules.  Imagine a web-service of some sort with frontend, backend and some machine_learning behind them:

.. Some more markup:
.. the line "|" forces the lines to be breaken explicitly
.. two stars "**" make boldface, single star italics

**macro_project**
  *frontend module*
    | submodule1
    | submodule2
  *backend module*
    | submodule1
    | submodule2
  *machine_learning module*
    | submodule1
    | submodule2
    

.. A numbered list
    
Here are some possibilities:

1. Clusterfuck
  * Keep everything in the same repository.  Avoid defining API interfaces.  Everybody can access all code.
  * We have all done this and it's not a good idea..
  
.. Nested lists must be separated with a blank line
  
2. Each module in a separate repository
  * Each module should have a maintainer and a well-defined API interface
  * Development 
  
    * Keep the cloned local git repositories at ~/python3_packages (as explained above)
    
  * Production
  
    * You can use pip(3) to install and update the packages directly from the git repository - convenient if the development/production cycle is fast
    * When installed with pip(3) there is no fiddling with paths and symlinks, it just works (if you're interested in details, pip(3) installs it under "~/.local/lib")

  * See how this is managed in the :ref:`Getting started <started>` section


Code Organization
-----------------

So, let's get back to this tutorial.  This is a module that has a single submodule called "greeters" that is organized as follows:

.. remember: "|" forces line-breaks

greeters
  | __init__.py
  | base.py
  | fancy.py
  
The idea is that "base.py" has some base class definitions that are used bu "fancy.py" to create derived classes.  __init__.py has been tweaked so that user can import with

::

  from cute_mongo_forms.greeters import UberFancyHelloWorld


instead of the cumbersome

::

  from cute_mongo_forms.fancy.greeters import UberFancyHelloWorld
  

(also, the api user should not be bothered with the internal inheritance, etc.)

So, I suggest you open in your editor all files of the "greeters" submodule and study them a bit.


Quick testing
-------------

Each module file under "greeters" works also as a stand-alone test.  Go to the "greeters" directory and try (either ipython or ipython3):

::
  
    ipython3
    %run fancy.py
    
Or just (either python or python3):

::

    python3 fancy.py


This is not proper testing, though.  It's quick'n'dirty testing during development.  Separate testing programs for systematic testing are a good idea to include in your module package.


Packaging
---------
  
If you have done everything as instructed here, creating a distributable python package can be done as follows:
  

::

  cd ~/python3_packages/cute_mongo_forms
  python3 setup.py sdist

  
Your distributable python package is now in directory "dist/".  You can install it with:

:: 

  pip3 install --upgrade dist/cute_mongo_forms-version.tar.gz
    
The setup.py script automatically finds and includes python packages to the distribution package.  In "MANIFEST.in" we also tell it to include the complete "docs/" directory and an auxiliary file from the "greeters" submodule.  See "setup.py" for more instructions.

Polish your setup script with the following cycle:

::

  rm dist/*
  python3 setup.py sdist
  pip3 install --upgrade --verbose dist/cute_mongo_forms-version.tar.gz

The verbose option is nice to see any problems, with say, your post-installing script defined in setup.py.  Once you have got rid of all the errors, you can be sure that it works also when people install your python package directly from git using pip(3). 

When testing the pip(3) installation directly from git, use virtualenv to see that you got the dependencies right.  First, create a virtualenv:

::

  virtualenv --no-site-packages -p python3 test

Then let's use that virtualenv (we clean up PYTHONPATH so that this is truly an isolated installation) to test your production system:
  
::

  cd test
  source bin/activate
  export PYTHONPATH=
  pip3 install --upgrade git+git://[your-personal-git-repository]/cute_mongo_forms

See :ref:`here <started>` how the end-user would be using your python module directly from git.
  
To exit from virtualenv, use:

::
  
  deactivate


Examples
--------

If you are creating big-time examples of how to use your module, the best idea is to create a separate repository for them, in this case it would be called "cute_mongo_forms_examples".

For a bit smaller example project you might opt for an examples submodule, i.e. for "cute_mongo_forms.examples"

Here we are considering example *snippets*, i.e. small and compact example programs to get the api user exited.  Here is an example:

The following python code can be downloaded from :download:`[here]<snippets/example1.py>`

.. include:: snippets/example1.py_

Check out directory "docs/snippets/".  It serves as a collection for small example programs.  Open and edit "form_snippets.bash" there.  You can then run it.  Remember to recompile the documentation once you've run that script.


  
Miscellaneous
-------------

* Check out in the source code, how I init and check a large number of parameters in the constructor
* It is also a good idea to add "exceptions.py" under "greeters".  This way you can quickly see what custom exceptions must be handled.


Internal links
--------------

Remember that reference we did up there?  Here is a :ref:`link <intro>` to it.


     
