This is Async Web Crawler: |____what can this project: |____1. It parse urls and get from they hrefs with absolute links. |____2. This code can render JavaScript sites (React, Vue, Angular) if text of response don't have any links. |____3. It can imitate human behavior what help with antibot in sites. |____4. all urls from crawler saved into domains.bd (sqlite) |____Dependencies: |____All dependencies in requirements.txt

How to start this Web Crawler: |____1. install all dependencies |____2. run main.py file

Features: |____1. You will can run code in cmd |____2. Add some optimization |____3. Upgrade Gui in Terminal (now gui in terminal bad)

author - ShineXmRedT14 LICENSE (MIT)