Skip to content

Fix scraper timeout in Docker#47

Merged
atharvashahane28 merged 15 commits intomainfrom
fix/42-scraper-timeout
Oct 9, 2025
Merged

Fix scraper timeout in Docker#47
atharvashahane28 merged 15 commits intomainfrom
fix/42-scraper-timeout

Conversation

@atharvashahane28
Copy link
Collaborator

This PR will:

  • Add a Dockerfile with instructions to build a Docker image that can run the data pipeline
  • Add necessary additional options to Selenium and Chromedriver
  • Add Shell & Python scripts to run the parts of the pipeline

Tests done:

  • Pipeline successfully runs in Windows, Linux, and MacOS
  • Pipeline successfully runs in Docker (locally)

@atharvashahane28 atharvashahane28 self-assigned this Oct 9, 2025
@atharvashahane28 atharvashahane28 linked an issue Oct 9, 2025 that may be closed by this pull request
@atharvashahane28 atharvashahane28 merged commit 05cb716 into main Oct 9, 2025
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Scraper timeout in Docker

2 participants

Comments