Metadata-Version: 2.1
Name: lovely-tensors
Version: 0.1.0
Summary: Lovely Tensors
Home-page: https://github.com/xl0/lovely-tensors
Author: Alexey Zaytsev
Author-email: alexey.zaytsev@gmail.com
License: Apache Software License 2.0
Keywords: jupyter pytorch tensor visualisation
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Developers
Classifier: Natural Language :: English
Classifier: Programming Language :: Python :: 3.7
Classifier: Programming Language :: Python :: 3.8
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: License :: OSI Approved :: Apache Software License
Requires-Python: >=3.7
Description-Content-Type: text/markdown
Provides-Extra: dev
License-File: LICENSE

Lovely Tensors
================

<!-- WARNING: THIS FILE WAS AUTOGENERATED! DO NOT EDIT! -->

## Read full docs [here](https://xl0.github.io/lovely-tensors/)

## Install

``` sh
pip install lovely-tensors
```

## How to use

How often do you find yourself debugging PyTorch code? You dump a tensor
to the cell output, and see this:

``` python
numbers
```

    tensor([[[-0.3541, -0.3369, -0.4054,  ..., -0.5596, -0.4739,  2.2489],
             [-0.4054, -0.4226, -0.4911,  ..., -0.9192, -0.8507,  2.1633],
             [-0.4739, -0.4739, -0.5424,  ..., -1.0390, -1.0390,  2.1975],
             ...,
             [-0.9020, -0.8335, -0.9363,  ..., -1.4672, -1.2959,  2.2318],
             [-0.8507, -0.7822, -0.9363,  ..., -1.6042, -1.5014,  2.1804],
             [-0.8335, -0.8164, -0.9705,  ..., -1.6555, -1.5528,  2.1119]],

            [[-0.1975, -0.1975, -0.3025,  ..., -0.4776, -0.3725,  2.4111],
             [-0.2500, -0.2325, -0.3375,  ..., -0.7052, -0.6702,  2.3585],
             [-0.3025, -0.2850, -0.3901,  ..., -0.7402, -0.8102,  2.3761],
             ...,
             [-0.4251, -0.2325, -0.3725,  ..., -1.0903, -1.0203,  2.4286],
             [-0.3901, -0.2325, -0.4251,  ..., -1.2304, -1.2304,  2.4111],
             [-0.4076, -0.2850, -0.4776,  ..., -1.2829, -1.2829,  2.3410]],

            [[-0.6715, -0.9853, -0.8807,  ..., -0.9678, -0.6890,  2.3960],
             [-0.7238, -1.0724, -0.9678,  ..., -1.2467, -1.0201,  2.3263],
             [-0.8284, -1.1247, -1.0201,  ..., -1.2641, -1.1596,  2.3786],
             ...,
             [-1.2293, -1.4733, -1.3861,  ..., -1.5081, -1.2641,  2.5180],
             [-1.1944, -1.4559, -1.4210,  ..., -1.6476, -1.4733,  2.4308],
             [-1.2293, -1.5256, -1.5081,  ..., -1.6824, -1.5256,  2.3611]]])

Was it really useful for me, as a human, to see all these numbers?

What is the shape?  
What are the statistics?  
Are any of the values `nan` or `inf`?  
Is it an image of a man holding a tench?

``` python
import lovely_tensors as lt
```

``` python
lt.monkey_patch()
```

## `__repr__`

``` python
numbers
```

    tensor[3, 196, 196] n=115248 x∈[-2.118, 2.640] μ=-0.388 σ=1.073

Better, huh?

``` python
numbers[1,:6,1] # Still shows values if there are not too many.
```

    tensor[6] x∈[-0.443, -0.197] μ=-0.311 σ=0.091 [-0.197, -0.232, -0.285, -0.373, -0.443, -0.338]

``` python
spicy = numbers.flatten()[:12].clone()

spicy[0] *= 10000
spicy[1] /= 10000
spicy[2] = float('inf')
spicy[3] = float('-inf')
spicy[4] = float('nan')

spicy = spicy.reshape((2,6))
spicy # Spicy stuff
```

    tensor[2, 6] n=12 x∈[-3.541e+03, -3.369e-05] μ=-393.776 σ=1.180e+03 +inf! -inf! nan!

``` python
torch.zeros(10, 10) # A zero tensor - make it obvious
```

    tensor[10, 10] n=100 all_zeros

``` python
spicy.verbose
```

    tensor[2, 6] n=12 x∈[-3.541e+03, -3.369e-05] μ=-393.776 σ=1.180e+03 +inf! -inf! nan!
    [[-3.5405e+03, -3.3693e-05,         inf,        -inf,         nan, -4.0543e-01],
     [-4.2255e-01, -4.9105e-01, -5.0818e-01, -5.5955e-01, -5.4243e-01, -5.0818e-01]]

``` python
spicy.plain # The old way
```

    [[-3.5405e+03, -3.3693e-05,         inf,        -inf,         nan, -4.0543e-01],
     [-4.2255e-01, -4.9105e-01, -5.0818e-01, -5.5955e-01, -5.4243e-01, -5.0818e-01]]

## Going `.deeper`

``` python
numbers.deeper
```

    tensor[3, 196, 196] n=115248 x∈[-2.118, 2.640] μ=-0.388 σ=1.073
      tensor[196, 196] n=38416 x∈[-2.118, 2.249] μ=-0.324 σ=1.036
      tensor[196, 196] n=38416 x∈[-1.966, 2.429] μ=-0.274 σ=0.973
      tensor[196, 196] n=38416 x∈[-1.804, 2.640] μ=-0.567 σ=1.178

``` python
# You can go deeper if you need to
dt = numbers[0,:45,0].view(3,3,5)
dt.deeper(2)
```

    tensor[3, 3, 5] n=45 x∈[-0.988, -0.337] μ=-0.538 σ=0.152
      tensor[3, 5] n=15 x∈[-0.611, -0.354] μ=-0.526 σ=0.086
        tensor[5] x∈[-0.474, -0.354] μ=-0.419 σ=0.044 [-0.354, -0.405, -0.474, -0.440, -0.423]
        tensor[5] x∈[-0.611, -0.542] μ=-0.590 σ=0.028 [-0.611, -0.611, -0.594, -0.594, -0.542]
        tensor[5] x∈[-0.611, -0.542] μ=-0.570 σ=0.038 [-0.542, -0.542, -0.542, -0.611, -0.611]
      tensor[3, 5] n=15 x∈[-0.988, -0.337] μ=-0.597 σ=0.233
        tensor[5] x∈[-0.988, -0.560] μ=-0.748 σ=0.199 [-0.560, -0.560, -0.714, -0.919, -0.988]
        tensor[5] x∈[-0.971, -0.405] μ=-0.662 σ=0.235 [-0.971, -0.816, -0.645, -0.474, -0.405]
        tensor[5] x∈[-0.474, -0.337] μ=-0.381 σ=0.056 [-0.388, -0.371, -0.337, -0.337, -0.474]
      tensor[3, 5] n=15 x∈[-0.679, -0.423] μ=-0.490 σ=0.069
        tensor[5] x∈[-0.491, -0.423] μ=-0.457 σ=0.027 [-0.457, -0.423, -0.440, -0.474, -0.491]
        tensor[5] x∈[-0.491, -0.440] μ=-0.464 σ=0.020 [-0.491, -0.457, -0.440, -0.457, -0.474]
        tensor[5] x∈[-0.679, -0.457] μ=-0.549 σ=0.094 [-0.474, -0.457, -0.525, -0.611, -0.679]

## Now in `.rgb` color

The important queston - is it our man?

``` python
numbers.rgb
```

![](index_files/figure-gfm/cell-13-output-1.png)

*Maaaaybe?* Looks like someone normalized him.

``` python
in_stats = ( (0.485, 0.456, 0.406),     # mean 
             (0.229, 0.224, 0.225) )    # std
numbers.rgb(in_stats)
```

![](index_files/figure-gfm/cell-14-output-1.png)

It’s indeed our hero, the Tenchman!

## `.plt` the statistics

``` python
(numbers+3).plt
```

![](index_files/figure-gfm/cell-15-output-1.png)

``` python
(numbers+3).plt(center="mean", max_s=1000)
```

![](index_files/figure-gfm/cell-16-output-1.png)

``` python
(numbers+3).plt(center="range")
```

![](index_files/figure-gfm/cell-17-output-1.png)

## See the `.chans`

``` python
# .chans will map values betwen [0,1] to colors.
# Make our values fit into that range to avoid clipping.
mean = torch.tensor(in_stats[0])[:,None,None]
std = torch.tensor(in_stats[1])[:,None,None]
numbers_01 = (numbers*std + mean)
numbers_01
```

    tensor[3, 196, 196] n=115248 x∈[0., 1.000] μ=0.361 σ=0.248

``` python
numbers_01.chans
```

![](index_files/figure-gfm/cell-19-output-1.png)

Let’s try with a Convolutional Neural Network

``` python
from torchvision.models import vgg11, VGG11_Weights
```

``` python
features = vgg11().features

# Note: I only saved the first 5 layers in "features.pt"
_ = features.load_state_dict(torch.load("../features.pt"), strict=False)
```

``` python
# Activatons of the second max pool layer of VGG11
print(features[5])

acts = (features[:6](numbers[None])[0]/2) # /2 to reduce clipping
acts
```

    MaxPool2d(kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)

    tensor[128, 49, 49] n=307328 x∈[0., 12.508] μ=0.367 σ=0.634 grad DivBackward0

``` python
acts.chans
```

![](index_files/figure-gfm/cell-23-output-1.png)

## Without `.monkey_patch`

``` python
lt.lovely(spicy)
```

    tensor[2, 6] n=12 x∈[-3.541e+03, -3.369e-05] μ=-393.776 σ=1.180e+03 +inf! -inf! nan!

``` python
lt.lovely(spicy, verbose=True)
```

    tensor[2, 6] n=12 x∈[-3.541e+03, -3.369e-05] μ=-393.776 σ=1.180e+03 +inf! -inf! nan!
    [[-3.5405e+03, -3.3693e-05,         inf,        -inf,         nan, -4.0543e-01],
     [-4.2255e-01, -4.9105e-01, -5.0818e-01, -5.5955e-01, -5.4243e-01, -5.0818e-01]]

``` python
lt.lovely(numbers, depth=1)
```

    tensor[3, 196, 196] n=115248 x∈[-2.118, 2.640] μ=-0.388 σ=1.073
      tensor[196, 196] n=38416 x∈[-2.118, 2.249] μ=-0.324 σ=1.036
      tensor[196, 196] n=38416 x∈[-1.966, 2.429] μ=-0.274 σ=0.973
      tensor[196, 196] n=38416 x∈[-1.804, 2.640] μ=-0.567 σ=1.178

``` python
lt.rgb(numbers, in_stats)
```

![](index_files/figure-gfm/cell-27-output-1.png)

``` python
lt.plot(numbers, center="mean")
```

![](index_files/figure-gfm/cell-28-output-1.png)

``` python
lt.chans(numbers_01)
```

![](index_files/figure-gfm/cell-29-output-1.png)
