Metadata-Version: 2.1
Name: sdenet
Version: 1.0
Summary: UNKNOWN
Home-page: UNKNOWN
Author: Philippe Remy
License: MIT
Description: ## SDE Net (Keras)
        This repo contains the code for the paper:
        
        Lingkai Kong, Jimeng Sun and Chao Zhang, SDE-Net: Equipping Deep Neural Network with Uncertainty Estimates, ICML2020.
        
        [[paper](https://arxiv.org/abs/2008.10546)] [[video](https://www.youtube.com/watch?v=RylZA4Ioc3M)]
        
        ![SDE-Net](figure/illustration.png)
        
        ### Package installation
        
        ```bash
        virtualenv -p python3 venv && source venv/bin/activate # optional but recommended.
        pip install -r requirements.txt && pip install -e . # install the package.
        ```
        
        ### Training & Evaluation
        
        Supported datasets are: MNIST, SVHN, CIFAR10, CIFAR100. Supported models are RESNET and SDENET.
        
        Look at the bash run scripts at the root of the repository to get started for training and evaluation.
        
        
        ### Comparison between official Pytorch implementation and Keras
        
        This comparison is just the result of one run. No runs were handpicked. Overall it's very similar.
        
        Except probably SDENET on SVHN (95% vs 94%).
        
        #### Pytorch
        
        ```
        MNIST RESNET
        _________________________________
        
        Final Accuracy: 9945/10000 (99.45%)
        
        generate log  from out-of-distribution data
        calculate metrics for OOD
        OOD  Performance of Baseline detector
        TNR at TPR 95%:            88.783%
        AUROC:                     95.939%
        Detection acc:             92.169%
        AUPR In:                   86.441%
        AUPR Out:                  98.434%
        
        calculate metrics for mis
        mis  Performance of Baseline detector
        TNR at TPR 95%:            89.791%
        AUROC:                     97.510%
        Detection acc:             93.041%
        AUPR In:                   99.985%
        AUPR Out:                  34.000%
        
        
        MNIST SDENET
        _________________________________
        
        Final Accuracy: 9927/10000 (99.27%)
        
        generate log  from out-of-distribution data
        calculate metrics for OOD
        OOD  Performance of Baseline detector
        TNR at TPR 95%:            99.372%
        AUROC:                     99.804%
        Detection acc:             98.692%
        AUPR In:                   99.483%
        AUPR Out:                  99.887%
        calculate metrics for mis
        mis  Performance of Baseline detector
        TNR at TPR 95%:            92.544%
        AUROC:                     97.525%
        Detection acc:             94.485%
        AUPR In:                   99.979%
        AUPR Out:                  41.739%
        
        
        SVHN RESNET
        _________________________________
        
        Final Accuracy: 24609/25856 (95.18%)
        
        generate log  from out-of-distribution data
        calculate metrics for OOD
        OOD  Performance of Baseline detector
        TNR at TPR 95%:            66.552%
        AUROC:                     94.421%
        Detection acc:             90.136%
        AUPR In:                   97.639%
        AUPR Out:                  84.998%
        calculate metrics for mis
        mis  Performance of Baseline detector
        TNR at TPR 95%:            64.376%
        AUROC:                     90.458%
        Detection acc:             85.371%
        AUPR In:                   99.301%
        AUPR Out:                  44.899%
        
        
        SVHN SDENET
        _________________________________
        
        Final Accuracy: 24588/25856 (95.10%)
        
        generate log  from out-of-distribution data
        calculate metrics for OOD
        OOD  Performance of Baseline detector
        TNR at TPR 95%:            65.215%
        AUROC:                     94.308%
        Detection acc:             89.746%
        AUPR In:                   97.694%
        AUPR Out:                  84.017%
        calculate metrics for mis
        mis  Performance of Baseline detector
        TNR at TPR 95%:            67.831%
        AUROC:                     91.267%
        Detection acc:             86.501%
        AUPR In:                   99.270%
        AUPR Out:                  48.871%
        
        ```
        
        #### Keras
        ```
        MNIST RESNET
        _________________________________
        
         Final Accuracy: 9944/10000 (99.44%)
        
        generate log  from out-of-distribution data
        calculate metrics for OOD
        OOD  Performance of Baseline detector
        TNR at TPR 95%:            93.162%
        AUROC:                     97.946%
        Detection acc:             94.250%
        AUPR In:                   94.842%
        AUPR Out:                  99.215%
        calculate metrics for mis
        mis  Performance of Baseline detector
        TNR at TPR 95%:            96.997%
        AUROC:                     98.863%
        Detection acc:             96.697%
        AUPR In:                   99.994%
        AUPR Out:                  26.744%
        
        MNIST SDENET
        _________________________________
        
        Final Accuracy: 9934/10000 (99.34%)
        
        generate log  from out-of-distribution data
        calculate metrics for OOD
        OOD  Performance of Baseline detector
        TNR at TPR 95%:            98.425%
        AUROC:                     99.567%
        Detection acc:             97.804%
        AUPR In:                   98.613%
        AUPR Out:                  99.872%
        calculate metrics for mis
        mis  Performance of Baseline detector
        TNR at TPR 95%:            95.515%
        AUROC:                     98.763%
        Detection acc:             95.825%
        AUPR In:                   99.992%
        AUPR Out:                  32.524%
        
        SVHN RESNET
        _________________________________
        
         Final Accuracy: 24487/25856 (94.71%)
        
        generate log  from out-of-distribution data
        calculate metrics for OOD
        OOD  Performance of Baseline detector
        TNR at TPR 95%:            56.648%
        AUROC:                     93.602%
        Detection acc:             87.504%
        AUPR In:                   97.627%
        AUPR Out:                  81.664%
        calculate metrics for mis
        mis  Performance of Baseline detector
        TNR at TPR 95%:            63.765%
        AUROC:                     91.843%
        Detection acc:             85.721%
        AUPR In:                   99.386%
        AUPR Out:                  46.231%
        
        SVHN SDENET
        _________________________________
        
        Final Accuracy: 24261/25856 (93.83%)
        
        generate log  from out-of-distribution data
        calculate metrics for OOD
        OOD  Performance of Baseline detector
        TNR at TPR 95%:            55.568%
        AUROC:                     93.406%
        Detection acc:             87.942%
        AUPR In:                   97.538%
        AUPR Out:                  82.881%
        calculate metrics for mis
        mis  Performance of Baseline detector
        TNR at TPR 95%:            61.798%
        AUROC:                     89.845%
        Detection acc:             85.974%
        AUPR In:                   99.183%
        AUPR Out:                  47.664%
        ```
        
Platform: UNKNOWN
Requires-Python: >=3
Description-Content-Type: text/markdown
