Metadata-Version: 2.1
Name: assistant_improve_toolkit
Version: 1.2.2
Summary: Assistant Improve Toolkit
Home-page: https://github.com/watson-developer-cloud/assistant-improve-recommendations-notebook
Author: IBM Watson
Author-email: watdevex@us.ibm.com
Maintainer: Zhe Zhang
Maintainer-email: zhangzhe@us.ibm.com
License: Apache 2.0
Platform: any
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: Apache Software License
Classifier: Programming Language :: Python
Classifier: Programming Language :: Python :: 3.6
Classifier: Programming Language :: Python :: 3.7
Classifier: Programming Language :: Python :: 3.8
Classifier: Operating System :: OS Independent
Classifier: Topic :: Software Development :: Libraries :: Python Modules
License-File: LICENSE

Watson Assistant Improve Notebooks
==================================

|Build Status| |Slack| |Latest Stable Version| |CLA assistant|

This repository houses Watson Assistant Improve notebooks and the
underlying assistant improve toolkit library.

Introduction
------------

To help improving your Watson Assistant after you have deployed it to
production, we prepared the following Jupyter notebooks. These notebooks
include practical steps for measuring, analyzing, and actively improving
your assistant in a continuous manner. Check out `IBM Watson Assistant
Continuous Improvement Best
Practices <https://github.com/watson-developer-cloud/assistant-improve-recommendations-notebook/raw/master/notebook/IBM%20Watson%20Assistant%20Continuous%20Improvement%20Best%20Practices.pdf>`__
for more details.

-  **Measure notebook** contains a set of automated metrics that help
   you monitor and understand the behavior of your system. The goal is
   to understand where your assistant is doing well vs where it isn’t,
   and to focus your improvement effort to one of the problem areas
   identified. This notebook generates an assessment spreadsheet for you
   to use to label problematic conversations, and then feed to the
   Effectiveness notebook.

-  **Effectiveness notebook** helps you understand the relative
   performance of each intent and entity as well as the confusion
   between your intents. This information helps you prioritize your
   improvement effort. The input to this notebook is an assessment
   spreadsheet generated from the Measure notebook. Update the marked
   columns in the spreadsheet with your labels and load it into the
   Effectiveness notebook for analysis.

-  **Customer Effort notebook** helps you measure and analyze the
   performance improvement after enabling the
   `Disambiguation <https://cloud.ibm.com/docs/assistant?topic=assistant-dialog-runtime#dialog-runtime-disambiguation>`__
   and
   `Autolearning <https://cloud.ibm.com/docs/assistant?topic=assistant-autolearn>`__
   features

-  **Logs notebook** helps you fetch logs using Watson Assistant API.
   You can fetch logs with various filters, and save them as a JSON
   file, or export the utterances in the logs into a CSV file. The JSON
   file can be loaded into the Measure notebook. The CSV file can be
   used for `intent recommendation
   service <https://cloud.ibm.com/docs/assistant?topic=assistant-intent-recommendations#intent-recommendations-get-intent-recommendations-task>`__.
   Alternatively, you can run python scripts
   ```fetch_logs`` <https://github.com/watson-developer-cloud/assistant-improve-recommendations-notebook/blob/master/src/main/python/fetch_logs.py>`__
   and
   ```export_csv_for_intent_recommendation`` <https://github.com/watson-developer-cloud/assistant-improve-recommendations-notebook/blob/master/src/main/python/export_csv_for_intent_recommendation.py>`__
   to fetch logs and export them to `intent recommendation
   CSV <https://cloud.ibm.com/docs/assistant?topic=assistant-intent-recommendations#intent-recommendations-data-resources>`__,
   respectively. Run ``python get_logs -h`` and
   ``python export_csv_for_intent_recommendation.py -h`` for usage.

-  **Dialog Flow Analysis notebook** help you assess and analyze user
   journeys and issues related to the dialog flow of ineffective (low
   quality) conversations based on production logs. Check out `Dialog
   Flow
   Analysis <https://github.com/watson-developer-cloud/assistant-dialog-flow-analysis>`__
   for more details.

-  **Dialog Skill Analysis notebook** help you analyze characteristics
   of your data such as the number of training examples for each intent
   or the terms which seem to be characteristic of a specific intent.
   Check out `Dialog Skill
   Analysis <https://github.com/watson-developer-cloud/assistant-dialog-skill-analysis>`__
   for more details.

Getting Started
---------------

You can either run the notebooks locally or in `IBM Watson
Studio <https://dataplatform.cloud.ibm.com/docs/content/wsj/analyze-data/notebooks-parent.html>`__.

-  **Run locally**

   1. Install Jupyter Notebook, see `Jupyter/IPython Notebook Quick
      Start
      Guide <https://jupyter-notebook-beginner-guide.readthedocs.io/en/latest/install.html>`__
      for more details.
   2. Download the Jupyter notebooks available in this repository's
      `notebook <https://github.com/watson-developer-cloud/assistant-improve-recommendations-notebook/tree/master/notebook>`__
      directory. **Note: These notebook files are not designed for
      Watson Studio environment**
   3. Start jupyter server ``jupyter notebook``
   4. Follow the instructions in each of the notebooks. Be sure to add
      your Watson Assistant credentials if necessary.

-  **Run in Watson Studio**

   1. | Create a Watson Studio account.
      | Sign up in `Watson
        Studio <https://www.ibm.com/cloud/watson-studio>`__, or use an
        existing account. Lite plan is free to use.

   2. | Create a new project and add a Cloud Object Storage (COS)
        account.
      | For more information regarding COS plans, see
        `Pricing <https://www.ibm.com/cloud-computing/bluemix/pricing-object-storage>`__.

   3. Copy
      `Measure <https://dataplatform.cloud.ibm.com/exchange/public/entry/view/133dfc4cd1480bbe4eaa78d3f635e568>`__
      or
      `Effectiveness <https://dataplatform.cloud.ibm.com/exchange/public/entry/view/133dfc4cd1480bbe4eaa78d3f636921c>`__
      notebook from Watson Studio community into your project.

   4. Follow the instructions in each notebook to add project tokens and
      Watson Assistant credentials if necessary.

Guides
------

-  Learn more about our measure and effectiveness notebook on Medium:
   `Continuously Improve Your Watson Assistant with Jupyter
   Notebooks <https://medium.com/ibm-watson/continuously-improve-your-watson-assistant-with-jupiter-notebooks-60231df4f01f>`__

Contributing
------------

See `CONTRIBUTING.md <CONTRIBUTING.md>`__ for more details on how to
contribute

License
-------

This library is licensed under the `Apache 2.0
license <http://www.apache.org/licenses/LICENSE-2.0>`__.

.. |Build Status| image:: https://travis-ci.org/watson-developer-cloud/assistant-improve-recommendations-notebook.svg?branch=master
   :target: https://travis-ci.org/github/watson-developer-cloud/assistant-improve-recommendations-notebook
.. |Slack| image:: https://wdc-slack-inviter.mybluemix.net/badge.svg
   :target: https://wdc-slack-inviter.mybluemix.net
.. |Latest Stable Version| image:: https://img.shields.io/pypi/v/assistant-improve-toolkit
   :target: https://pypi.org/project/assistant-improve-toolkit/
.. |CLA assistant| image:: https://cla-assistant.io/readme/badge/watson-developer-cloud/assistant-improve-recommendations-notebook
   :target: https://cla-assistant.io/watson-developer-cloud/assistant-improve-recommendations-notebook


