Skip to content

Feat/fetch image script#41

Merged
Perry2004 merged 5 commits intomainfrom
feat/fetch-image-script
Oct 12, 2025
Merged

Feat/fetch image script#41
Perry2004 merged 5 commits intomainfrom
feat/fetch-image-script

Conversation

@Perry2004
Copy link
Owner

No description provided.

Copilot AI review requested due to automatic review settings October 12, 2025 23:08
Copy link

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR introduces a Python-based script to automatically fetch image URLs from a Pexels photographer's profile page and generates the necessary configuration for the project. The implementation uses Selenium WebDriver to scrape images and automatically handles "Load More" buttons to gather comprehensive image collections.

  • Adds automated image fetching capability using Selenium and BeautifulSoup
  • Integrates image preparation into the build pipeline with shell script automation
  • Updates package.json scripts to run image preparation before dev/build commands

Reviewed Changes

Copilot reviewed 8 out of 11 changed files in this pull request and generated 3 comments.

Show a summary per file
File Description
scripts/main.py Core Python script that uses Selenium to scrape Pexels images with automated "Load More" clicking
scripts/prepare-image-links.sh Shell script that orchestrates the image fetching process and integrates with uv package manager
scripts/pyproject.toml Python project configuration defining dependencies for web scraping (Selenium, BeautifulSoup, chromedriver)
scripts/test.json Sample/test data containing example Pexels image URLs
public/data/rolling-images.json Updated image data file with new URLs fetched by the script
package.json Modified dev/build scripts to run image preparation automatically

Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.

@@ -0,0 +1,160 @@
from fileinput import filename
Copy link

Copilot AI Oct 12, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The fileinput module import is unused and should be removed. The filename identifier from this import is not used anywhere in the code.

Suggested change
from fileinput import filename

Copilot uses AI. Check for mistakes.
scripts/main.py Outdated
Comment on lines +94 to +96
# # Wait for new content to load
# print("Waiting for new content to load...")
# time.sleep(3)
Copy link

Copilot AI Oct 12, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Remove commented-out code. If this sleep functionality might be needed, either implement it or document why it's disabled.

Suggested change
# # Wait for new content to load
# print("Waiting for new content to load...")
# time.sleep(3)
# No sleep after clicking "Load More" button; page loads fast enough and additional waits are handled elsewhere.

Copilot uses AI. Check for mistakes.
# time.sleep(3)

except (StaleElementReferenceException, Exception) as e:
# print(f"Error clicking button: {e}")
Copy link

Copilot AI Oct 12, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Remove commented-out debug print statement or replace with proper logging if error information is needed.

Copilot uses AI. Check for mistakes.
Copilot AI review requested due to automatic review settings October 12, 2025 23:13
Copy link

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

Copilot reviewed 10 out of 12 changed files in this pull request and generated 2 comments.


Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.

scripts/main.py Outdated
Comment on lines +94 to +96
# # Wait for new content to load
# print("Waiting for new content to load...")
# time.sleep(3)
Copy link

Copilot AI Oct 12, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Commented-out code should be removed rather than left in the codebase. If this delay logic might be needed later, consider making it configurable or documenting why it's disabled.

Copilot uses AI. Check for mistakes.
# time.sleep(3)

except (StaleElementReferenceException, Exception) as e:
# print(f"Error clicking button: {e}")
Copy link

Copilot AI Oct 12, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Commented-out error logging should either be enabled for debugging purposes or removed entirely. Silent error handling can make troubleshooting difficult.

Suggested change
# print(f"Error clicking button: {e}")
print(f"Error clicking button: {e}")

Copilot uses AI. Check for mistakes.
@Perry2004 Perry2004 merged commit b5b8c91 into main Oct 12, 2025
3 checks passed
@Perry2004 Perry2004 deleted the feat/fetch-image-script branch October 12, 2025 23:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants