Metadata-Version: 1.1
Name: rotating-free-proxies
Version: 0.1.1
Summary: Rotating proxies for Scrapy
Home-page: https://github.com/nabinkhadka/rotating-free-proxies
Author: Nabin Khadka
Author-email: nbnkhadka14@gmail.com
License: MIT license
Description: rotating-free-proxies
        ============================
        Avoid being banned by webpages when you crawl them. This is an extension to the amazing scrapy-rotating-proxies_ library. The main target of this library is to get proxies dynamically when the spider is running. This library automatically fetches freely available lists of proxies from free-proxy-list.net_.
        
        
        .. _scrapy-rotating-proxies: https://pypi.python.org/pypi/rotating-free-proxies
        .. _free-proxy-list.net: https://free-proxy-list.net/
        
        
        Installation
        ------------
        ::
        
            pip install rotating-free-proxies
        
        Usage
        -----
        
        After installing you need to do just add following two variables in settings.py of your Scrapy project ::
        
        
            ROTATING_PROXY_LIST_PATH = '/my/path/proxies.txt' # Path that this library uses to store list of proxies
            NUMBER_OF_PROXIES_TO_FETCH = 5 # Controls how many proxies to use
        
        
            DOWNLOADER_MIDDLEWARES = {
                'rotating_free_proxies.middlewares.RotatingProxyMiddleware': 610,
                'rotating_free_proxies.middlewares.BanDetectionMiddleware': 620,
            }
        
        
        
        For further details on using this library, refer to the original readme_.
        
        .. _readme: https://github.com/TeamHG-Memex/scrapy-rotating-proxies/blob/master/README.rst
        
        
        Thank you!
        
        
        CHANGES
        =======
        
        0.1.1 (2020-05-28)
        ------------------
        
        Fixes bug when user does not create file to store proxies.
        Adds settings to control number of proxies to use
        
        
        0.1.0 (2020-05-01)
        ------------------
        
        Initial release
        
        
Platform: UNKNOWN
Classifier: Development Status :: 3 - Alpha
Classifier: License :: OSI Approved :: MIT License
Classifier: Intended Audience :: Developers
Classifier: Operating System :: OS Independent
Classifier: Framework :: Scrapy
Classifier: Programming Language :: Python
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.7
Classifier: Programming Language :: Python :: 3.8
