This project is a distributed web scraper designed to efficiently gather data from multiple websites. The scraper is built using a distributed systems architecture to ensure scalability, fault tolerance, and high performance.
-
Clone the repository:
git clone https://github.com/marians002/Distributed-Scraper.git cd Distributed-Scraper -
Run startup script:
./startup.sh