Metadata-Version: 2.1
Name: transformer-implementations
Version: 0.0.4
Summary: A bunch of transformer implementations
Home-page: https://github.com/UdbhavPrasad072300/Transformer-Implementations
Author: Udbhav Prasad
Author-email: udbhavprasad072300@gmail.com
License: MIT
Description: # Transformer Implementations 
        
        Transformer Implementations and some examples with them
        
        Implemented:
        <ul>
          <li>Vanilla Transformer</li>
          <li>ViT - Vision Transformers</li>
          <li>DeiT - Data efficient image Transformers</li>
        </ul>
        
        ## Installation
        
        <a href="https://pypi.org/project/transformer-implementations/">PyPi Installation</a>
        
        ```bash
        $ pip install transformer-implementations
        ```
        
        ## Language Translation
        
        from "Attention is All You Need": https://arxiv.org/pdf/1706.03762.pdf
        
        Models trained with Implementation:
        <ul>
          <li><a href="https://github.com/UdbhavPrasad072300/Transformer-Implementations/blob/main/notebooks/Multi30k%20-%20Language%20Translation.ipynb">Multi30k - German to English</a></li>
        </ul>
        
        ## Multi-class Image Classification with Vision Transformers (ViT)
        
        from "An Image is Worth 16x16 words: Transformers for image recognition at scale": https://arxiv.org/pdf/2010.11929v1.pdf
        
        Models trained with Implementation:
        <ul>
         <li><a href="https://github.com/UdbhavPrasad072300/Transformer-Implementations/blob/main/notebooks/MNIST%20Classification%20-%20ViT.ipynb">MNIST - Grayscale Images</a></li>
          <li><a href="https://github.com/UdbhavPrasad072300/Transformer-Implementations/blob/main/notebooks/CIFAR10%20Classification%20-%20ViT.ipynb">CIFAR10 - MultiChannel Images</a></li>
        </ul>
        
        ## Multi-class Image Classification with Data-efficient image Transformers (DeiT)
        
        from "Training data-efficient image transformers & distillation through attention": https://arxiv.org/pdf/2012.12877v1.pdf
        
        Models trained with Implementation:
        <ul>
         <li><a href="https://github.com/UdbhavPrasad072300/Transformer-Implementations/blob/main/pre-train/VGG16_CIFAR10.ipynb">Pretraining Teacher model for Distillation</a></li>
         <li><a href="https://github.com/UdbhavPrasad072300/Transformer-Implementations/blob/main/notebooks/CIFAR10%20Classification%20-%20DeiT.ipynb">CIFAR10 - Low Res Images</a></li>
        </ul>
        
Platform: UNKNOWN
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Science/Research
Classifier: Operating System :: OS Independent
Classifier: License :: OSI Approved :: MIT License
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Classifier: Programming Language :: Python :: 3 :: Only
Description-Content-Type: text/markdown
