Metadata-Version: 2.1
Name: deepcrawl-robots
Version: 0.0.4
Summary: A package to bulk match urls to robots.txt files
Home-page: UNKNOWN
Author: Andrei Mutu & Richard Barrett
Author-email: support@deepcrawl.com
License: UNKNOWN
Description: UNKNOWN
Platform: UNKNOWN
Classifier: Programming Language :: Python :: 3
Classifier: License :: OSI Approved :: MIT License
Classifier: Operating System :: OS Independent
Requires-Python: >=3.6
Description-Content-Type: text/x-rst
