Metadata-Version: 2.1
Name: azdsdr
Version: 1.220929.10
Summary: UNKNOWN
Home-page: https://github.com/xhinker/azdsdr
Author: Andrew Zhu
Author-email: xhinker@hotmail.com
License: Apache License
Keywords: DS Data Reader
Platform: UNKNOWN
Description-Content-Type: text/markdown

# AZDSDR - Data Scientist's Data Reader

This package includes data reader for DS to access data in a easy way. 

Covered data platforms
    * Dremio
    * Kusto

## Installation

Use pip to install the package and all of the dependences

```
pip install -U azdsdr
```

The `-U` will help update your old version to the newest

Or, you can clone the repository and copy over the `readers.py` file to your project folder.  

## Use Dremimo Reader

### Step 1. Install Dremio Connector

You will need to install the Dremio ODBC driver first to use `DremioReader` from this package. 

**For Windows user**

Please download the [dremio-connector](https://github.com/xhinker/azdsdr/tree/main/drivers) file from the drivers folder. 


### Step 2. Generate a [Personal Access Token(PAT)](https://docs.dremio.com/cloud/security/authentication/personal-access-token/#creating-a-token)

- Recommend storing this personal access token in a safe location, such as a user environment variable on your local machine.  
- Start Menu -> “Edit Environment variables For Your Account”.  
- Click “New” under environment variables.  
- Enter a new variable with name “DREMIO_TOKEN” and set the value to the PAT you generated earlier.  


### Step 3. Configure driver
- Go to Start Menu -> “ODBC Data Sources (64-bit)”.
- Under User DSN, click “Add”.
- Add Dremio Connector
- Configure as follows, with your own <alias>@host.com as the username.

![](README/2022-09-29-13-26-26.png)

- Click Ok/Save

**For Linux and Mac User**

You can download the driver from [Dremio's ODBC Driver](https://www.dremio.com/drivers/odbc/) page. 

### Sample Query

```python
from azdsdr.readers import DremioReader
username    = "name@host.com"
token       = "token string"
dr          = DremioReader(username=username,token=token)

sql = '''
select 
    * 
from 
    sample_table
limit 10
'''
r = dr.run_sql(sql)
```


## Use Kusto Reader

Sign in with Azure CLI so that you can run Kusto script without login everytime. 

```
az login
```

### Sample Query

```python 
from readers import KustoReader

cluster = "https://help.kusto.windows.net"
db      = "Samples"
kr = KustoReader(cluster=cluster,db=db)

kql = "StormEvents | take 10"
r = kr.run_kql(kql)
```

The Kusto Reader is test in Windows 10, in theroy should also work in Linux and Mac. 


