Metadata-Version: 2.1
Name: django-easy-scraper
Version: 1.0.1
Summary: Dango app to scrape web page
Home-page: https://github.com/dearopen/django-easy-scraper
Author: dearopen
Author-email: 
License: UNKNOWN
Description: # Django Easy Scraper
        
        An standalone django app that can be used/intigrated with both django and no-django application easily. the scraping mechanism is fully based on `Regular Expression` but you can extends based on what you are familiar with very easily.
        
        It requires to install python `requests` modules
        
        
        ## Install
        
        `pip install django-easy-scraper`
        
        ## Basic Uses
        
        ```
        from django_easy_scraper import scraper
        
        class ScrapeExampleDotCom(scraper.Scraper):
        
            regex_fields = {
                'price': "Write Your Regex pattern for price here",
                'title': "Write your regex pattern for title here",
                # Like above way you can add as much fields/keys as you want
            }
        
        ```
        
        ### Scrape now
        ```
        url = 'www.example.com/bla-bla-details-page/
        data = ScrapeExampleDotCom.regex_url_scraper(url)
        
        print(data)
        ```
        and the response should look like this if your regex pattern are correct:
        ```
        {
            'price': 4,
            'title': 'an scraped title',
        }
        ```
        
        `regex_url_scraper` method always gives you json response,
        
        So if you add many regex pattern in `regex_fields`, it will give you response that number of dictionary key with result that you added in that dictionary.
        
        
        # Multiple Sites Scraping together
        
        You don't need to call different method for different site all the time !! Just call once and scrape all, something fun, right?
        
        ### Like you are gonna scrape three sites:
        > www.example.com
        
        > www.exampletwo.com
        
        > www.examplethree.com
        
        
        
        But how will those site product scrape automatically, it scares you ? 
        
        - Wirte Regex pattern for all above site with the fields that you want to scrape:
        
        
        ```
        from django_easy_scraper import scraper
        
        class ScrapeExampleDotCom(scraper.Scraper):
            regex_fields = {
                'price': "Write Your Regex pattern for price here",
                'title': "Write your regex pattern for title here",
                # Like above way you can add as much fields/keys as you want
            }
        
        class ScrapeExampleTwo(scraper.Scraper):
            regex_fields = {
                'price': "Write Your Regex pattern for price here",
                'title': "Write your regex pattern for title here",
                # Like above way you can add as much fields/keys as you want
            }
        
        class ScrapeExampleThree(scraper.Scraper):
            regex_fields = {
                'price': "Write Your Regex pattern for price here",
                'title': "Write your regex pattern for title here",
                # Like above way you can add as much fields/keys as you want
            }
        
        
        ```
        
        
        You have written regular expression for you all the site you are gonna scrape,
        
        Now it's time use our `Switch` class that will route your script/class based on the site you are gonna scrape? Cool, right ? !!
        
        It's where the magic really begins:
        
        Just place all your class in the dictionary `switcher`.
        
        
        > Important Note:
        
        `key` name should be domain name, pure domain name, no www or http or slash, dont add anything prefix/suffix
        `value` should be the class of that domain you have written for and place it's method `regex_url_scraper
        
        ```
        from django_easy_scraper import switch
        
        class Switch(switch.BaseSwitch):
            switcher = {
                'example.com': ScrapeExampleDotCom.regex_url_scraper,
                'exampletwo.com': ScrapeExampleTwo.regex_url_scraper,
                'examplethree.com': ScrapeExampleThree.regex_url_scraper,
            }
        
        ```
        
        So you have done routing your script/class based on the url it gets.
        
        ### Get response data as python dictionary like above site:
        
        ```
        url = 'Any of site you have written class for the site and added in switch class'
        
        response = Switch.get_data(url=url, raise_exception=False)
        
        print(response) # Will give you an object of data that you trying to scrape
        
        ```
        
        Switch class is giving you facilities to route your scraping class automacally based whatever site link pass to it's `get_data` method.
        
        `get_data` method's `raise_exception` is it handle if you want to raise excepiton when your expected fields not found
        
        
        ### Got an issue ?
        Please open an issue on our github repo: https://github.com/dearopen/django-easy-scraper
        
        Don't forget to star to this project if you like this.
        
        Happy Scraping !!
Platform: UNKNOWN
Classifier: Programming Language :: Python :: 3
Classifier: License :: OSI Approved :: MIT License
Classifier: Operating System :: OS Independent
Requires-Python: >=3.6
Description-Content-Type: text/markdown
