.. |binder| image:: https://img.shields.io/badge/launch-jscatter-F5A252.svg?logo=data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAFkAAABZCAMAAABi1XidAAAB8lBMVEX///9XmsrmZYH1olJXmsr1olJXmsrmZYH1olJXmsr1olJXmsrmZYH1olL1olJXmsr1olJXmsrmZYH1olL1olJXmsrmZYH1olJXmsr1olL1olJXmsrmZYH1olL1olJXmsrmZYH1olL1olL0nFf1olJXmsrmZYH1olJXmsq8dZb1olJXmsrmZYH1olJXmspXmspXmsr1olL1olJXmsrmZYH1olJXmsr1olL1olJXmsrmZYH1olL1olLeaIVXmsrmZYH1olL1olL1olJXmsrmZYH1olLna31Xmsr1olJXmsr1olJXmsrmZYH1olLqoVr1olJXmsr1olJXmsrmZYH1olL1olKkfaPobXvviGabgadXmsqThKuofKHmZ4Dobnr1olJXmsr1olJXmspXmsr1olJXmsrfZ4TuhWn1olL1olJXmsqBi7X1olJXmspZmslbmMhbmsdemsVfl8ZgmsNim8Jpk8F0m7R4m7F5nLB6jbh7jbiDirOEibOGnKaMhq+PnaCVg6qWg6qegKaff6WhnpKofKGtnomxeZy3noG6dZi+n3vCcpPDcpPGn3bLb4/Mb47UbIrVa4rYoGjdaIbeaIXhoWHmZYHobXvpcHjqdHXreHLroVrsfG/uhGnuh2bwj2Hxk17yl1vzmljzm1j0nlX1olL3AJXWAAAAbXRSTlMAEBAQHx8gICAuLjAwMDw9PUBAQEpQUFBXV1hgYGBkcHBwcXl8gICAgoiIkJCQlJicnJ2goKCmqK+wsLC4usDAwMjP0NDQ1NbW3Nzg4ODi5+3v8PDw8/T09PX29vb39/f5+fr7+/z8/Pz9/v7+zczCxgAABC5JREFUeAHN1ul3k0UUBvCb1CTVpmpaitAGSLSpSuKCLWpbTKNJFGlcSMAFF63iUmRccNG6gLbuxkXU66JAUef/9LSpmXnyLr3T5AO/rzl5zj137p136BISy44fKJXuGN/d19PUfYeO67Znqtf2KH33Id1psXoFdW30sPZ1sMvs2D060AHqws4FHeJojLZqnw53cmfvg+XR8mC0OEjuxrXEkX5ydeVJLVIlV0e10PXk5k7dYeHu7Cj1j+49uKg7uLU61tGLw1lq27ugQYlclHC4bgv7VQ+TAyj5Zc/UjsPvs1sd5cWryWObtvWT2EPa4rtnWW3JkpjggEpbOsPr7F7EyNewtpBIslA7p43HCsnwooXTEc3UmPmCNn5lrqTJxy6nRmcavGZVt/3Da2pD5NHvsOHJCrdc1G2r3DITpU7yic7w/7Rxnjc0kt5GC4djiv2Sz3Fb2iEZg41/ddsFDoyuYrIkmFehz0HR2thPgQqMyQYb2OtB0WxsZ3BeG3+wpRb1vzl2UYBog8FfGhttFKjtAclnZYrRo9ryG9uG/FZQU4AEg8ZE9LjGMzTmqKXPLnlWVnIlQQTvxJf8ip7VgjZjyVPrjw1te5otM7RmP7xm+sK2Gv9I8Gi++BRbEkR9EBw8zRUcKxwp73xkaLiqQb+kGduJTNHG72zcW9LoJgqQxpP3/Tj//c3yB0tqzaml05/+orHLksVO+95kX7/7qgJvnjlrfr2Ggsyx0eoy9uPzN5SPd86aXggOsEKW2Prz7du3VID3/tzs/sSRs2w7ovVHKtjrX2pd7ZMlTxAYfBAL9jiDwfLkq55Tm7ifhMlTGPyCAs7RFRhn47JnlcB9RM5T97ASuZXIcVNuUDIndpDbdsfrqsOppeXl5Y+XVKdjFCTh+zGaVuj0d9zy05PPK3QzBamxdwtTCrzyg/2Rvf2EstUjordGwa/kx9mSJLr8mLLtCW8HHGJc2R5hS219IiF6PnTusOqcMl57gm0Z8kanKMAQg0qSyuZfn7zItsbGyO9QlnxY0eCuD1XL2ys/MsrQhltE7Ug0uFOzufJFE2PxBo/YAx8XPPdDwWN0MrDRYIZF0mSMKCNHgaIVFoBbNoLJ7tEQDKxGF0kcLQimojCZopv0OkNOyWCCg9XMVAi7ARJzQdM2QUh0gmBozjc3Skg6dSBRqDGYSUOu66Zg+I2fNZs/M3/f/Grl/XnyF1Gw3VKCez0PN5IUfFLqvgUN4C0qNqYs5YhPL+aVZYDE4IpUk57oSFnJm4FyCqqOE0jhY2SMyLFoo56zyo6becOS5UVDdj7Vih0zp+tcMhwRpBeLyqtIjlJKAIZSbI8SGSF3k0pA3mR5tHuwPFoa7N7reoq2bqCsAk1HqCu5uvI1n6JuRXI+S1Mco54YmYTwcn6Aeic+kssXi8XpXC4V3t7/ADuTNKaQJdScAAAAAElFTkSuQmCC
            :alt: link to mybinder
            :target: https://mybinder.org/v2/gl/biehl%2Fjscatter/master?filepath=src%2Fjscatter%2Fexamples%2Fnotebooks


Installation
============

Dependencies
------------
-  numpy, scipy  ->  Mandatory, automatically installed by pip
-  cubature      ->  Mandatory for multidimensional quadrature
-  Pillow        ->  Mandatory, for reading of SAXS images, automatic install by pip
-  matplotlib    ->  Mandatory, for 3D plots and on Windows
-  Ipython       ->  Optional, for convenience as a powerful python shell
-  gfortran      ->  Optional, without some functions dont work or use a slower python version
-  xmgrace       ->  Optional, preferred plotting on Unix like (use matplotlib on Windows)

Installation of gfortran/xmgrace may need root privileges. Use "sudo" on Linux and MacOS if needed.

Pip installation/upgrade
------------------------
(use pip3 if NOT your default Python, latest python2.7 compatible version was 1.1.0) ::

  sudo pip install jscatter

As user in home directory (pip default installation directory is in ~/.local/).
No sudo needed, only user privileges, you don't need the admin to install/update::

 pip install jscatter --user

from a local repository (development versions)::

  pip install jscatter --user --upgrade --pre --find-links /where/the/file/is/saved

  options
  --user       : Install in user directory (folder defined by PYTHONUSERBASE or the default ~/.local)
  --find-links : look in the given path for package links e.g development releases
  --upgrade    : to install upgrades also for dependencies
  --pre        : to install also development versions

Installation directly from  from git repository branch. For different branch use branch name
e.g. as development version use dev.
(You may need a user account) ::

 pip install git+https://gitlab.com/biehl/jscatter.git@master

Linux
-----
* Ubuntu, all Debian related  ::

   sudo apt-get install build-essential  # install gcc and more
   sudo apt-get install gfortran grace python-matplotlib  # or python3-matplotlib e.g. Debian,Fedora, RedHat
   sudo pip install cython
   sudo pip install ipython
   sudo pip install jscatter

* CentOs, Suse, Fedora ... do same as above but with yum/zypper...

* Manjaro Linux (yaourt asks for permission as root or prepend sudo as above)  ::

   # install gfortran
   yaourt gcc-fortran
   # install xmgrace (only found in AUR), fonts are needed for the interface, fonts are loaded after restart
   yaourt xorg-fonts-75 xorg-fonts-100 grace-openmotif python-matplotlib
   # tk might be missing for tkinter as matplotlib backend
   sudo pacman -S tk
   pip install ipython cython
   pip install jscatter

* CONTIN in DLS module (Only if needed).

  See DLS module documentation for details how to get and compile the original fortran code.

MacOs
-----
**Homebrew** is a MacOs package manager.

Install Homebrew first as given on their web page (see `Homebrew <https://brew.sh/>`_).
Homebrew installs things to /usr/local/ so you don’t need sudo permissions
but you need to set the path to prevent usage of the MacOS system Python.

Check that PATH points to the Homebrew location (not to the system Python).

Append the following line to your .profile (or .bash_profile) file and restart the terminal: ::

 export PATH=/usr/local/bin:/usr/local/share/python:$PATH

Install the needed packages ::

   # install XQuartz from homebrew or from the AppStore
   brew cask install xquartz
   # install xmgrace and gfortran (included in gcc)
   brew install grace gcc
   # install python (python3 is now the default in homebrew)
   brew install python
   # then use pip (this should call python3 and use pip3)
   python3 -m pip install ipython cython
   # dependencies are installed automatically
   python3 -m pip install jscatter
   # check if correct python is used ( /usr/local/bin/python )
   which python3

Brew installs python3 by default. On older MacOS python2.7 might be the default, so start by `ipython3`.

If you have trouble like `missing xcrun ..` you might have to re-register it or update it to the latest version.
Try `xcode-select --install`.

**Using the system Python** for experts

You may just use the MacOs system Python installing xmgrace from source and using pip for python.
In newer MacOs (e.g. Catalina) you may get an error "stdio.h not found"
which is related to the missing /usr/local/include folder as decided by Apple.

Add to .zshrc or corresponding to include this path in CPATH::

 export CPATH=/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/


Windows
-------

Windows Subsystem for Linux (WSL) with Ubuntu way
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
A new way is based on  Windows Subsystem for Linux (WSL), which
"lets developers run GNU/Linux environment
-- including most command-line tools, utilities, and applications -- directly on Windows,
unmodified, without the overhead of a virtual machine."
(See `WSL <https://docs.microsoft.com/de-de/windows/wsl/about>`_)

This allows to use any Linux application including XmGrace or Linux editors.
You work on the same filesystem as your Windows user account without the need of
syncing folders or using a shared folder from a virtual machine.
Also gfortran compiled code is working.

The basic procedure is to activate WSL, install Linux and Xserver, add your needed Linux software in the usual way.

- Install **WSL**

  See `<https://docs.microsoft.com/de-de/windows/wsl/install-win10>`_

  or in Powershell as administrator ::

    Enable-WindowsOptionalFeature -Online -FeatureName Microsoft-Windows-Subsystem-Linux

- Then **install a linux distribution** of choice from Windows store (tested with Ubuntu 18.04).
  (The install is per user, so do it under your user account, not as administrator)

  Open a terminal, start bash and set user and password for Linux.
  (See https://docs.microsoft.com/en-us/windows/wsl/initialize-distro )

- For graphical applications: **Install Xserver** VcXsrv (or an other Xserver).

  See e.g. https://seanthegeek.net/234/graphical-linux-applications-bash-ubuntu-windows/ .
  Download from https://sourceforge.net/projects/vcxsrv/ .
  Install (start always manually or add to Windows autostart).

- **Configure bash** to use the local X server

  Add this to .bashrc ::

   export DISPLAY=localhost:0.0

  or run this inside bash ::

   echo "export DISPLAY=localhost:0.0" >> ~/.bashrc

  Restart bash or load again .bashrc ``source ~/.bashrc``

- **Install** xmgrace, python3 and **Jscatter** in Linux subsystem

  open terminal, start bash ::

   sudo apt-get update && upgrade                                   # upgrade linux
   sudo apt-get install python3 python3-pip ipython3 gfortran gcc   # python, ipython and pip3
   sudo apt-get install numpy python3-matplotlib xmgrace python-tk  # mainly for graphical output
   pip3 install cython
   pip3 install jscatter                                            # jscatter, scipy, Pillow...

 Run test and example.


Anaconda way
^^^^^^^^^^^^
`Anaconda <https://www.anaconda.com/distribution>`_ is a python distribution as alternative
with numpy, scipy, matplotlib, Ipython preinstalled.
Need of sudo depends on how Anaconda was installed (root or user).
Maybe the matplotlib backend needs to be configured on Windows to work properly.

And there was more to adjust, when i stopped waisting my time.
In my testcase it was a pain, but test and examples work (no Xmgrace, no fortran).

I strongly advise to use the  **WSL way** from above, as it is easier to install, update and use.
And even gfortran is working in WSL!!
::

   # install jscatter on working anaconda environment
   pip install jscatter

Jupyter Notebook
----------------
Jscatter works on Jupyter Notebooks. You may try Jscatter live at |binder|.

The example notebooks are included in the example subfolder of your Jscatter installation
or can be downloaded from
`Gitlab/Jscatter <https://gitlab.com/biehl/jscatter/tree/master/src/jscatter/examples/notebooks>`_ .

To install jscatter on a server Jupyter installation (Jscatter not preinstalled)
prepend this on your script
::

 import sys
 # install jscatter as user in the current Jupyter kernel
 !{sys.executable} -m pip install jscatter

There is some trouble with inline plots (which are not interactive and cannot be updated)
dependent on the installed backend. This is related to the used matplotlib backend.

Use this (before importing anything else) to get interactive plots inline. ::

 %matplotlib notebook

Then import Jscatter. ::

 import jscatter as js

 js.usempl(True)  # use matplotlib
 js.usempl(False) # default, use grace on your computer with xmgrace in external window.

If you work over http on a server (usual no display of Xwindows applications) use the same to
switch to matplotlib.

Using a cluster/headless mode
-----------------------------
For long running scripts or fitting one might want to use a
`cluster <https://en.wikipedia.org/wiki/Computer_cluster>`_.

Cluster allow often no graphical output (as they are used e.g. trough ssh running a script.).
To run your script as usual (after testing on your PC) Jscatter implements a *headless* mode
which can be activated using::

 import jscatter as js
 js.headless(True)

In *headless* mode all errPlots are saved automatically to *lastErrPlot.agr* and open matplotlib plots
are saved as *lastopenedplots_i.png* if js.mpl.show() is used.
Both allow to retrieve an updated graphical output as stored image instead of updated windows.

Please save important plots in the usual way to store them with a useful name.
Additional any data output should be saved in the usual way.
Xmgrace is advantageous as .agr format allows later change of the plots including rescaling.

Examples are also adapted and saved to jscatterExamples/example_name/
under the current working directory.

For experts:
 Xmgrace uses *gracebat* and matplotlib uses the non-interactive backend "Agg".

 For matplotlib create figures using *fig=js.mpl.mplot* or *fig=matplotlib.figure.Figure*
 and later save by *fig.savefig*. Dont use pyplot as it generates errors in headless mode without interactive loop.

To switch back it is best to restart ipython (at least for matplotlib).

Testing
-------
You can test basic functionality of jscatter after installation::

   import jscatter as js
   js.test.doTest()
   #basic graphics and fitting
   js.examples.runExample('example_SinusoidalFitting.py')

The Example shows :
 - 3 sine fit plots with one sine
 - a fit plot with 5 sine curves fitted simultaneous
 - a simple plot with 5 points ( phase against Amplitude of the 5 sines)


During development::

 python setup.py test


Troubleshooting and tips
------------------------

If xmgrace is not found by jscatter the path to the executable may be not on your PATH variable.
Check this by calling xmgrace in a shell.
Change your PATH in your .bashrc by adding::

   export PATH=/path/to/xmgrace:$(PATH) )

To open .agr files by klicking add a new file association e.g. to KDE.
 In SystemSettings/FileAssociations add a new type xmgrace.
 Inside this add FilenamePatterns '*.agr' and similar.

   In 'ApplicationPreferenceOrder' add xmgrace and edit this new application :
    In General :  edit name to "xmgrace" (keep it).
    In Application : edit name to "xmgracefree" and command to "xmgrace -free".
    This will open files in free floating size format.
   In 'ApplicationPreferenceOrder' add again application xmgrace (no changes)
   The second opens files in fixed size format (no '-free').





