Metadata-Version: 1.1
Name: ndexutil
Version: 0.11.0a2
Summary: Unsupported NDEx utilities
Home-page: https://github.com/ndexbio/ndexutils
Author: The NDEx Project
Author-email: contact@ndexbio.org
License: BSD
Description: ndexutil
        ==========
        
        .. _NDEx: https://ndexbio.org
        .. _NDEx CX: https://www.home.ndexbio.org/data-model/
        
        **Warning: This repository is for development and features may change.
        Please use this at your own risk.**
        
        .. image:: https://img.shields.io/pypi/v/ndexutil.svg
                :target: https://pypi.python.org/pypi/ndexutil
        
        .. image:: https://img.shields.io/travis/ndexbio/ndexutil.svg
                :target: https://travis-ci.org/ndexbio/ndexutil
        
        .. image:: https://coveralls.io/repos/github/ndexbio/ndexutils/badge.svg?branch=master
                :target: https://coveralls.io/github/ndexbio/ndexutils?branch=master
        
        .. image:: https://readthedocs.org/projects/ndexutils/badge/?version=latest
                :target: https://ndexutil.readthedocs.io/en/latest/?badge=latest
                :alt: Documentation Status
        
        Dependencies
        ------------
        
        * `ndex2 <https://pypi.org/project/ndex2>`_
        * `networkx <https://pypi.org/project/networkx>`_
        * `biothings_client <https://pypi.org/project/biothings-client>`_
        * `requests <https://pypi.org/project/requests>`_
        * `requests-toolbelt <https://pypi.org/project/requests_toolbelt>`_
        * `pandas <https://pypi.org/project/pandas>`_
        * `mygene <https://pypi.org/project/mygene>`_
        * `enum34 <https://pypi.org/project/enum34>`_
        * `jsonschema <https://pypi.org/project/jsonschema>`_
        * `urllib3 <https://pypi.org/project/urllib3>`_
        
        Optional Dependencies
        ---------------------
        
        Installed if `pip install ndexutil[cytoscape]` is run
        
        * `py4cytoscape <https://pypi.org/project/py4cytoscape>`_
        
        Compatibility
        -------------
        
        * Python 3.4+
        
        Installation
        ------------
        
        .. code-block::
        
           git clone https://github.com/ndexbio/ndexutils
           cd ndexutils
           make dist
           pip install dist/ndexutil*whl
        
        OR via `PyPI <https://pypi.org/ndexutils>`_
        
        .. code-block::
        
           pip install ndexutil
        
        ndexmisctools.py
        -----------------
        
        **WARNING:** Please consider this tool alpha and probably contains errors/bugs and could cause data corruption or loss. You have been warned
        
        **ndexmisctools.py** lets caller perform some operations on `NDEx`_ via the
        command line.
        This tool follows format of ``ndexmisctools.py <COMMAND> <args>``
        
        For more information run ``ndexmisctools.py --help`` and ``ndexmisctools.py <COMMAND> --help``
        
        **COMMANDS**:
        
        * **addnodeattrib** - adds an attribute to all nodes in a network
        
          Credentials to `NDEx`_ server must be stored in the configuration (default ``~/.ndexutils.conf``)
          and be formatted as follows:
        
          .. code-block::
        
            [addprofile]
            user=bob
            password=6ea8f0ab0b2e
            server=public.ndexbio.org
        
          The following command adds **foo** node attribute with value
          ``my new value`` to all nodes in the network ``9025480b-6fbc-4efe-9cd8-b575ce49dfda``
        
          .. code-block::
        
            ndexmisctools.py --profile addprofile addnodeattrib --uuid 9025480b-6fbc-4efe-9cd8-b575ce49dfda --name foo --value 'my new value'
        
          .. warning::
        
             This command changes the network in place, so it is a destructive call with NO undo
        
        * **copynetwork** - copies `NDEx`_ network between accounts and even servers
        
          For copying the source and destination credentials must be stored in the configuration (default ``~/.ndexutils.conf``)
          and be formatted as follows:
        
          .. code-block::
        
           [mycopyprofile]
           source_user=bob
           source_password=6ea8f0ab0b2e
           source_server=public.ndexbio.org
           dest_user=smith
           dest_password=4efe9cd8
           dest_server=public.ndexbio.org
        
          The following command copies the network ``9025480b-6fbc-4efe-9cd8-b575ce49dfda`` from source credentials defined in configuration to dest
        
          .. code-block::
        
            ndexmisctools.py --profile mycopyprofile copynetwork --uuid 9025480b-6fbc-4efe-9cd8-b575ce49dfda
        
        * **cytoscapelayout** - updates layout for network in `NDEx`_ using Cytoscape
        
          **NOTE:** This tool requires installation of `py4cytoscape <https://pypi.org/project/py4cytoscape>`_
                    and Cytoscape to be activately running (tested with Cytsocape 3.8+)
        
          This command requires four positional parameters. The first is the layout algorithm
          to use followed by (``username``, ``password``, and ``server``) which are credentials for
          `NDEx`_ server to update the layout on the network.
        
          Any of these credential fields set to **'-'** will
          force this tool to obtain the information from (default ``~/.ndexutils.conf``) configuration file
          under the profile specified by the ``--profile`` field in this format:
        
          .. code-block::
        
              [<value of --profile>]
              user = <NDEx username>
              password = <NDEx password>
              server = <NDEx server ie public.ndexbio.org>
        
          **NOTE:** For version 0.11.0a1 this command will fully replace the network on NDEx.
        
          Using credentials from `myattrib` profile, the following command adds grid
          layout to the network ``9025480b-6fbc-4efe-9cd8-b575ce49dfda``
        
          .. code-block::
        
            ndexmisctools.py --profile myattrib networkxlayout grid - - - --uuid 9025480b-6fbc-4efe-9cd8-b575ce49dfda
        
          To get a list of available layouts printed to standard out, replace the layout with ``listlayout``. Just put
          fake values in for credentials and any text for the --uuid flag:
        
          .. code-block::
        
            ndexmisctools.py cytoscapelayout listlayout x y z --uuid ignored
        
        
        * **deletenetwork** - deletes a network or all networks under a given networkset from `NDEx`_
        
          Credentials to `NDEx`_ server must be stored in the configuration (default ``~/.ndexutils.conf``)
          and be formatted as follows:
        
          .. code-block::
        
            [deleteprofile]
            user=bob
            password=6ea8f0ab0b2e
            server=public.ndexbio.org
        
          The following command deletes the network with UUID ``9025480b-6fbc-4efe-9cd8-b575ce49dfda`` from `NDEx`_
        
          .. code-block::
        
            ndexmisctools.py --profile deleteprofile deletenetwork --uuid 9025480b-6fbc-4efe-9cd8-b575ce49dfda
        
        * **featurednetworkreport** - generates report about featurend networks on `NDEx`_
        
          This command requires one positional parameter ``server``.
          For production use ``public.ndexbio.org``
        
        
          If the positional parameter is set to **'-'**, it will
          force this tool to obtain the information from (default ``~/.ndexutils.conf``) configuration file
          under the profile specified by the ``--profile`` field in this format:
        
          .. code-block::
        
              [<value of --profile>]
              server = <NDEx server ie public.ndexbio.org>
        
        
          The command below queries the production server for the featured networks json file
          and then queries `NDEx`_ for all networks that are part of the featured networks.
        
          .. code-block::
        
            ndexmisctools.py featurednetworkreport public.ndexbio.org --output foo.csv
        
          **NOTE:** If a networkset contains more then 500 networks, only the first 500 are examined
        
        
        * **networkattribupdate** - updates network attributes on network in `NDEx`_
        
          **WARNING:** Currently **name, version, and description** CANNOT be updated with this command.
        
          Credentials must be stored in the configuration (default ``~/.ndexutils.conf``)
          and be formatted as follows:
        
          .. code-block::
        
            [myattrib]
            user=bob
            password=6ea8f0ab0b2e
            server=public.ndexbio.org
        
          The following command updates **foo** network attribute on the network ``9025480b-6fbc-4efe-9cd8-b575ce49dfda``
        
          .. code-block::
        
            ndexmisctools.py --profile myattrib networkattribupdate --uuid 9025480b-6fbc-4efe-9cd8-b575ce49dfda --name foo --type string --value 'my new value'
        
        * **networkxlayout** - updates layout for network in `NDEx`_ using networkx library
        
        
          This command requires four positional parameters. The first is the layout algorithm
          to use followed by (``username``, ``password``, and ``server``) which are credentials for
          `NDEx`_ server to update the layout on the network.
        
          Any of these credential fields set to **'-'** will
          force this tool to obtain the information from (default ``~/.ndexutils.conf``) configuration file
          under the profile specified by the ``--profile`` field in this format:
        
          .. code-block::
        
              [<value of --profile>]
              user = <NDEx username>
              password = <NDEx password>
              server = <NDEx server ie public.ndexbio.org>
        
          By default this tool only updates the ``cartesianLayout`` aspect. To update the full network
          use add the ``--updatefullnetwork`` flag.
        
          Using credentials from `myattrib` profile, the following command adds spring
          layout to the network ``9025480b-6fbc-4efe-9cd8-b575ce49dfda``
        
          .. code-block::
        
            ndexmisctools.py --profile myattrib networkxlayout spring - - - --uuid 9025480b-6fbc-4efe-9cd8-b575ce49dfda
        
        
        * **removenodeattrib** - removes an attribute from all nodes in a network
        
          Credentials to `NDEx`_ server must be stored in the configuration (default ``~/.ndexutils.conf``)
          and be formatted as follows:
        
          .. code-block::
        
            [removeprofile]
            user=bob
            password=6ea8f0ab0b2e
            server=public.ndexbio.org
        
          The following command removes **foo** node attribute from all nodes
          in the network ``9025480b-6fbc-4efe-9cd8-b575ce49dfda``
        
          .. code-block::
        
            ndexmisctools.py --profile removeprofile removenodeattrib --uuid 9025480b-6fbc-4efe-9cd8-b575ce49dfda --name foo
        
          .. warning::
        
             This command changes the network in place, so it is a destructive call with NO undo
        
        * **systemproperty** - updates showcase, visibility, and indexing for single network or all networks in networkset in `NDEx`_
        
          **NOTE:** ``--showcase`` has no effect if network visibility is ``private``
        
          Credentials must be stored in the configuration (default ``~/.ndexutils.conf``)
          and be formatted as follows:
        
          .. code-block::
        
            [myattrib]
            user=bob
            password=6ea8f0ab0b2e
            server=public.ndexbio.org
        
          The following command enables showcase and sets indexing to `meta` for network with id ``9025480b-6fbc-4efe-9cd8-b575ce49dfda``
        
          .. code-block::
        
            ndexmisctools.py --profile myattrib systemproperty --uuid 9025480b-6fbc-4efe-9cd8-b575ce49dfda --showcase --indexlevel meta
        
          The following command sets visibility to `public` for all networks in networkset with id ``e9580d43-ec14-4be8-9977-9de88e1d410a``
        
          .. code-block::
        
            ndexmisctools.py --profile myattrib systemproperty --networksetid e9580d43-ec14-4be8-9977-9de88e1d410a --visibility public
        
        * **styleupdate** - update style of network in `NDEx`_
        
        
          Credentials must be stored in the configuration (default ``~/.ndexutils.conf``)
          and be formatted as follows:
        
          .. code-block::
        
            [styleprofile]
            user=bob
            password=6ea8f0ab0b2e
            server=public.ndexbio.org
        
          The following command updates style of network with UUID ``9025480b-6fbc-4efe-9cd8-b575ce49dfda`` with style from network specified by this UUID ``e9580d43-ec14-4be8-9977-9de88e1d410a``
        
          .. code-block::
        
            ndexmisctools.py --profile styleprofile styleupdate --uuid 9025480b-6fbc-4efe-9cd8-b575ce49dfda --styleuuid e9580d43-ec14-4be8-9977-9de88e1d410a
        
        
        * **tsvloader** - Loads TSV files as networks into `NDEx`_
        
          The **tsvloader** command loads an edge list file in tab separated format (hence TSV) and using a load plan, loads that data as a network into `NDEx <https://ndexbio.org>`_.
          This tool attempts to mimic behavior of the older ``tsv_uploader.py`` script located here: https://github.com/ndexbio/load-content
          This new version uses the more memory efficient StreamTSVLoader.
        
          This command requires five positional parameters.
          The first three (``username`, ``password``, and ``server``) are credentials for
          `NDEx`_ server to upload the network.
        
          Any of these first three credential fields set to **'-'** will
          force this tool to obtain the information from (default ``~/.ndexutils.conf``) configuration file
          under the profile specified by the ``--profile`` field in this format:
        
          .. code-block::
        
              [<value of --profile>]
              user = <NDEx username>
              password = <NDEx password>
              server = <NDEx server ie public.ndexbio.org>
        
          The forth positional parameter ``tsv_file`` (see _`TSV Loader` section below) should be
          set to edge list file in tab separated format and the
          fifth or last positional parameter ``load_plan`` should be
          set to the load plan. The load plan is a JSON formatted text
          file that maps the columns to nodes, edges, and attributes
          in the network.
        
          By default this tool does not generate much output to
          standard out/error. For more verbosity add three to five ``-v`` parameters
          to left of command name **tsvloader** as seen in examples below.
        
        
          In example below the ``- - -`` tells the program to read the credentials
          from the configuration file
        
          .. code-block::
        
              ndexmisctools.py -vvvv tsvloader - - - datafile.tsv load.plan
        
          Here is an example where the name and description of the network is set
          and the ``-t`` specifies a template network used to get style and in this case
          since ``--copyattribs`` is set the network attributes (minus ``@context``) are
          also copied to the new network
        
          .. code-block::
        
              ndexmisctools.py -vvv tsvloader bob xx public.ndexbio.org \
                               datafile.tsv loadplan.json --uppercaseheader  \
                               -t dafe07ca-0676-11ea-93e0-525400c25d22 \
                               --name mynetwork --description 'some text' --copyattribs
        
        
          In this example an alternate header is prepended via the ``--header`` flag.
        
          **NOTE:** The ``--header`` flag does **NOT** remove an existing header
        
          .. code-block::
        
              ndexmisctools.py -vv --profile foo tsvloader - - public.ndexbio.org \
                               datafile.tsv loadplan.json \
                               --header 'col1	col2	col3' \
                               -t some_cx_file.cx \
                               -u 48a26aa0-0677-11ea-93e0-525400c25d22
        
          If successful ``0`` is returned otherwise there was an error.
        
        .. _TSV_Loader: .
        
        TSV Loader
        ----------
        
        This module contains the Tab Separated Variable Loader (TSV Loader, see `ndexutil/streamtsvloader.py` module) which generates
        an `NDEx CX`_ file from a tab separated
        text file of edge data and attributes.
        
        To load data a load plan must be created. This plan tells the loader how to map the
        columns in the file to nodes, and edges. This load plan needs to validate against
        `this load plan JSON schema <https://github.com/ndexbio/ndexutils/blob/master/ndexutil/tsv/loading_plan_schema.json>`_
        
        **Example TSV file**
        
        .. code-block::
        
            SOURCE  TARGET  WEIGHT
            ABCD    AAA1    0.555
            GGGG    BBBB    0.305
        
        **SOURCE** is the source node, **TARGET** is target node
        
        A schema that could be:
        
        .. code-block::
        
            {
            "source_plan":
                {
                    "node_name_column": "SOURCE"
                },
                "target_plan":
                {
                    "node_name_column": "TARGET"
                },
                "edge_plan":
                {
                    "default_predicate": "unknown",
                    "property_columns": [
                      {
                        "column_name": "WEIGHT",
                        "attribute_name": "weight",
                        "data_type": "double"
                      }
                    ]
                }
            }
        
        
        
        Example below assumes the following:
        
        * **./loadplan.json** is the load plan in JSON format
        * **./style.cx** is a `NDEx CX`_ with a style.
        
        .. code-block::
        
            import ndex2
            from ndexutil.tsv.streamtsvloader import StreamTSVLoader
        
            # using ndex2 client library read CX file as NiceCXNetwork object
            style_network = ndex2.create_nice_cx_from_file('./style.cx')
        
            loader = StreamTSVLoader('./loadplan.json', style_network)
            with open('./input.tsv', 'r') as tsvfile:
                with open('./output.cx', 'w') as outfile:
                    loader.write_cx_network(tsvfile, outfile)
        
        
        Credits
        -------
        
        
        
        =======
        History
        =======
        
        0.11.0 2020-10-26
        ------------------------
        
        * Added new command to **ndexmisctools.py** named *networkxlayout* that lets caller apply networkx layouts to network on NDEx
        
        * Added new command to **ndexmisctools.py** named *cytoscapelayout* that lets caller apply Cytoscape layouts to network on NDEx
        
        * Added new command to **ndexmisctools.py** named *featurednetworkreport* that generates a report about featured networks on NDEx
        
        
        0.10.0 2020-02-19
        ------------------------
        
        * Added new command to **ndexmisctools.py** named *addnodeattrib* that lets caller add an attribute to all nodes in a network
        
        * Added new command to **ndexmisctools.py** named *removenodeattrib* that lets caller remove an attribute from all nodes in a network
        
        0.9.0.post1 2019-12-11
        ------------------------
        
        * Updated this history file adding a note about addition of *styleupdate* command to **ndexmisctools.py** in 0.9.0 release
        
        0.9.0 2019-12-11
        ------------------
        
        * Added *--typeonly* flag to *networkattribupdate* command in **ndexmisctools.py** that lets caller just update the data type for a given network attribute
        
        * Added new command to **ndexmisctools.py** named *deletenetwork* that lets caller delete a network or delete all networks in a given networkset from NDEx
        
        * Added new command to **ndexmisctools.py** named *styleupdate* that lets caller update the style of a network or all networks in a given set on NDEx
        
        
        0.8.1 2019-11-15
        ------------------
        
        * Fixed bug where **ndexmisctools.py** failed to run due to argparse bug in Python versions below 3.7
        
        0.8.0 2019-11-15
        ------------------
        
        * Fixed bug in StreamTSVLoader where 2 \@context network attributes were added to network if \@context was set in load plan and \@context was passed in the network attributes parameter for `write_cx_network()`
        
        * Added new command to **ndexmisctools.py** named *tsvloader* that lets caller load TSV files as networks into NDEx
        
        0.7.0 2019-09-11
        -----------------
        
        * Added new command to **ndexmisctools.py** named *systemproperty* that lets
          caller update showcase, visibility, and indexing for a single network or
          all networks in a networkset in NDEx
        
        0.6.1 2019-07-12
        ----------------
        
        * Fixed bug where **ndexmisctools.py** *networkattributeupdate* was creating
          duplicate network attributes for name, description, and value. This is
          due to server bug. To deal with this the code removes those entries when
          doing the network attribute update and forbids caller from trying to
          update those attributes
        
        0.6.0 2019-07-10
        ----------------
        
        * Added new commandline utility *ndexmisctools.py* which lets caller
          copy a network from one NDEx account to another and lets one update
          network attributes of a network in NDEx.
          WARNING: THIS IS AN UNTESTED ALPHA RELEASE AND MAY CONTAIN ERRORS
        
        0.5.0 2019-06-06
        ----------------
        
        * Added GeneSymbolSearcher class to loaderutils.py module
        
        * Minor bug fix in NetworkIssueReport get_fullreport_as_string() where
          issue text is wrapped in str() in case its not a string 
        
        0.4.0 2019-05-23
        ----------------
        
        * Added loaderutils.py module with two new classes, NetworkIssueReport and
          an abstract class NetworkUpdator
        
        0.3.0 2019-05-14
        ----------------
        
        * tsv2nicecx2.convert_pandas_to_nice_cx_with_load_plan now loads @context
          data into a @context network attribute instead of a separate aspect
        
        0.2.0 2019-04-01
        ----------------
        
        * StreamTSVLoader class added which enables loading of TSV data into
          in streaming format to handle loading of large networks
        
        
        
        
Keywords: network analysis biology
Platform: UNKNOWN
Classifier: Development Status :: 3 - Alpha
Classifier: Natural Language :: English
Classifier: Intended Audience :: Science/Research
Classifier: Topic :: Scientific/Engineering :: Information Analysis
Classifier: Topic :: Scientific/Engineering :: Bio-Informatics
Classifier: Topic :: Scientific/Engineering :: Medical Science Apps.
Classifier: License :: OSI Approved :: BSD License
Classifier: Programming Language :: Python :: 3.4
Classifier: Programming Language :: Python :: 3.5
Classifier: Programming Language :: Python :: 3.6
Classifier: Programming Language :: Python :: 3.7
