feat: crawl_and_import_data

[LuckySugar0716]

No description provided.

[PACHAKUTlQ]

Nice, but:

1. Code not formatted. And you seemingly did not configure pre-commit git hook correctly
2. Too much comments. And should not use non-ascii characters
3. Do not use that hardcoded dummy cookie, read from env or stdin
4. Use logging instead of print

[Copilot]

Pull Request Overview

This PR introduces a new interactive course data crawling and import system to replace the old ORC crawling functionality. The new system provides a unified interface for crawling course data from multiple sources (course selection system and official website) with interactive management capabilities.

• Replaces hardcoded ORC crawling with an interactive spider management system
• Implements comprehensive course data integration from multiple APIs
• Adds asynchronous crawling capabilities for improved performance

Reviewed Changes

Copilot reviewed 6 out of 8 changed files in this pull request and generated 2 comments.

Show a summary per file

File	Description
scripts/init.py	Replaces old crawling logic with new interactive spider manager
pyproject.toml	Adds aiohttp dependency for async HTTP requests
apps/spider/manager.py	New comprehensive spider management system with cache and import functionality
apps/spider/crawlers/orc.py	Complete rewrite with new CourseSelCrawler class and async capabilities
apps/spider/crawlers/README.md	Documentation for the new API endpoints and data structures
Makefile	Removes frontend formatting requirement

---

_{Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.}

[Copilot]

This replacement operation does nothing since both arguments are identical. The comment suggests converting Chinese comma (，) to English comma (,), but both parameters are the same English comma.

Suggested change

	",", ","
	"，", ","

[zzjc1234]

CourseReview/apps/spider/crawlers/medians.py

Lines 10 to 11 in 137aac7

	MEDIAN_PAGE_INDEX_URL = "http://www.dartmouth.edu/reg/transcript/medians/"
	MEDIANS_URL_FMT = "http://www.dartmouth.edu/reg/transcript/medians/{term}.html"

old url, plz change

[zzjc1234]

CourseReview/apps/spider/crawlers/orc.py

Line 106 in 137aac7

self.jsessionid = jsessionid

consider using environment variables

[zzjc1234]

CourseReview/apps/spider/crawlers/orc.py

Lines 116 to 119 in 137aac7

	if not self.jsessionid:
	print("Please enter your JSESSIONID cookie:")
	print("(Found in browser dev tools under Network or Application tabs)")
	self.jsessionid = input("JSESSIONID: ").strip()

...

[zzjc1234]

CourseReview/apps/spider/crawlers/timetable.py

Line 9 in 137aac7

TIMETABLE_URL = "http://oracle-www.dartmouth.edu/dart/groucho/timetable.display_courses"

...