Metadata-Version: 2.1
Name: transformers-stream-generator
Version: 0.0.4
Summary: This is a text generation method which returns a generator, streaming out each token in real-time during inference, based on Huggingface/Transformers.
Home-page: https://github.com/LowinLi/transformers-stream-generator
Author: LowinLi
Author-email: lowinli@outlook.com
License: MIT License
Project-URL: Repo, https://github.com/LowinLi/transformers-stream-generator
Project-URL: Bug Tracker, https://github.com/LowinLi/transformers-stream-generator/issues
Description: # transformers-stream-generator
        
        [![PyPI - Python Version](https://img.shields.io/pypi/pyversions/transformers-stream-generator.svg)](https://pypi.org/project/transformers-stream-generator/)
        [![PyPI](https://img.shields.io/pypi/v/transformers-stream-generator.svg)](https://pypi.org/project/transformers-stream-generator/)
        [![GitHub license badge](https://img.shields.io/github/license/LowinLi/transformers-stream-generator)](https://github.com/LowinLi/transformers-stream-generator/blob/main/LICENSE)
        [![Blog](https://img.shields.io/badge/blog-LowinLi-important)](https://lowin.li)
        [![Downloads](https://pepy.tech/badge/transformers-stream-generator)](https://pepy.tech/project/transformers-stream-generator)
        
        ### Description
        This is a text generation method which returns a generator, streaming out each token in real-time during inference, based on Huggingface/Transformers. 
        
        ### Web Demo
        + original
        ![](./pic/original.gif)
        + stream
        ![](./pic/stream.gif)
        
        ### Installation
        ```bash
        pip install transformers-stream-generator
        ```
        
        ### Usage
        1. just add two lines of code before your original code
        ```python
        from transformers_stream_generator import init_stream_support
        init_stream_support()
        ```
        
        2. add `do_stream=True` in `model.generate` function and keep `do_sample=True`, then you can get a generator
        ```python
        generator = model.generate(input_ids, do_stream=True, do_sample=True)
        for token in generator:
            word = tokenizer.decode(token)
            print(word)
        ```
        
        ### Example
        + run python script [example](./example/run.py) by gpt2
        + run web [example](./example/run_web.py)  by gpt2 and test in client [example](./example/test_client.py)
Keywords: GPT,stream,transformers,NLP,model hub,transformer,text generation,summarization,translation,q&a,qg,machine learning,CausalLM
Platform: UNKNOWN
Classifier: Intended Audience :: Developers
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Classifier: License :: OSI Approved :: Apache Software License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.5
Classifier: Programming Language :: Python :: 3.6
Classifier: Programming Language :: Python :: 3.7
Classifier: Programming Language :: Python :: 3.8
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Requires-Python: >=3.5
Description-Content-Type: text/markdown
