Metadata-Version: 2.1
Name: proxy_master
Version: 1.0.0
Summary: My first package to scrap free proxies from open resources
Author-email: 555Russich <Kutsokondima@yandex.ru>
Project-URL: Homepage, https://github.com/555Russich/proxy_master
Classifier: Programming Language :: Python :: 3
Classifier: License :: OSI Approved :: MIT License
Classifier: Operating System :: OS Independent
Requires-Python: >=3.10
Description-Content-Type: text/markdown
License-File: LICENSE

This module provide asynchronously scraping a bunch of `http`, `https`, `socks4`, `socks5` proxies from websites such as:
1) [free-proxy-list.net](https://free-proxy-list.net)
2) [geonode.com](https://geonode.com)
3) [hidemy.name](https://hidemy.name)

Just now proxy-master scraped `23265 proxies` from websites above. Count of unique by ip:port is `18383`, 
but some servers support different protocols.

> I add more websites to scrap from time to time, **BUT** looking for help and advices 
from people with experience

## Dependencies 

`python3.10` and higher. Pattern matching using in project

<sub>Will be installed automatically with `pip`</sub>

[aiohttp](https://github.com/aio-libs/aiohttp)
[bs4](https://pypi.org/project/beautifulsoup4/)
[lxml](https://pypi.org/project/lxml/)
[pycountry](https://github.com/flyingcircusio/pycountry)
[aiohttp_socks](https://github.com/romis2012/aiohttp-socks)

## Installation
`pip install proxy-master`

## Usage

1. This will create `proxy_master.json` in you home directory and return list of proxies.
```python
import proxy_master as pm
proxies = pm.get_proxies(
    protocol='http',
    do_prints=True
)
```
Then you can test proxies:
```python
working_proxies = pm.test_proxies(
    proxies,
    proxy_protocol: 'http',
    website_protocol: 'http'
)
```

## Features
- [x] Scrap different type of proxies include `https`, `socks4`, `socks5`
- [x] <i>Recursively</i> scraping. Use already collected proxies to scrap another website
- [x] test_public_ip(...) `socks4`, `socks5` using <a href="https://github.com/Skactor/aiohttp-proxy">aiohttp-proxy</a>

## In plans
- [] https://freeproxylists.net
- [] https://spys.one/en/free-proxy-list/
- 
