Pyrouette
=========

Submodules

----------

CMACs in Python

Author: Jeremy Stober
Contact: stober@gmail.com
Version: 1.0

This is a standalone Python module that implements CMACS. The
implementation relies on Pythons own internal hashing methods to
implement the algorithm in a simple "Pythonic" way. Below the CMAC
class an example that learns the 3d structure of a wave surface is
included and should serve as an out of the box test that you have the
necessary prerequisites (numpy and matplotlib) to run this code.

A setup.py script is included that allows for easy installation.

CMACs are useful function approximation methods but suffer from
problems if the dimensionality of the input data is large.

Requires: numpy, pylab

---------

utils

Author: Jeremy Stober
Contact: stober@gmail.com
Version: 0.1

Python utilities.


----------

Gridworlds and Markov Decision Processes in Python

Author: Jeremy Stober
Contact: stober@gmail.com
License: BSD (see LICENSE)

This package contains implementations of discrete mdps known as
gridworlds that are convenient for testing reinforcement learning
algorithms. Some visualization is provided, along with a few other
mdps drawn from the literature on reinforcement learning.

Note: This package now depends on https://github.com/stober/utils for sparse feature generation.

-----------

Cartpole Simulation in Python

Author: Jeremy Stober
Contact: stober@gmail.com
Version: 0.01

This is a Python implementation that is numerically as close as
possible to the publicly available cart pole simulation found here:
http://webdocs.cs.ualberta.ca/~sutton/book/code/pole.c.

This is a useful simple continuous control problem for testing RL
algorithms.

------------

Temporal Difference Learning in Python

Author: Jeremy Stober
Contact: stober@gmail.com
Version: 0.01

This is a Python implementation of some common temporal difference
learning algorithms. The implementations use discrete, linear, or CMAC
value function representations and include eligibility traces
(ie. TD(\lambda)).

Requires:
    numpy,
    pybrain (for nfq comparison)


-------------

Least Squares Policy Iteration in Python

Author: Jeremy Stober
Contact: stober@gmail.com
Version: 0.1

This is a Python implementation of LSPI from Lagoudakis and Parr in
JMLR (2003). The code depends on having an environment that provides a
function phi for generating features from state-action pairs, and a
function linear_policy for evaluating the policy. The gridworld
package (https://github.com/stober/gridworld) provides example
environments. Both lspi.py and lstdq.py contain example code using a
simple chainworld environment from the original paper (included in the
gridworld package).

--------------

Principle Component Analysis in Python

Author: Jeremy Stober
Contact: stober@gmail.com
Version: 0.1

This is PCA for cases where sample size is much smaller than the
dimensionality of the thing being sampled (e.g. Eigenfaces). There are
two versions of the main compute_pca function. One is pure Python and
not necessarily memory efficient (due in part to the strange memory
inefficiency of large np.dot operations and the fact that no
operations are done inplace. There is a faster, more efficient in
place version (which replaces the input samples with PCs) that is
coded using Cython.

---------------

Multidimensional Scaling in Python

Author: Jeremy Stober
Contact: stober@gmail.com
Version: 0.1

Includes the classic version of multidimensional scaling.

----------------

Isomap

Author: Jeremy Stober
Contact: stober@gmail.com
Version: 0.1

This is a Python implementation of Isomap built on top of my mds library (https://github.com/stober/mds/). This supports both k and epsilon nearest neighbor graph computations prior to determining isometric distances between data points. You can find more info about isomap, as well as a Matlab implementation here: http://isomap.stanford.edu/.

---------------

Dynamic Time Warping in Python

Author: Jeremy Stober
Contact: stober@gmail.com
Version: 0.1

This is a collection of dynamic time warping algorithms (including
related algorithms like edit-distance).

Requires: numpy