-
Notifications
You must be signed in to change notification settings - Fork 0
Open
Description
Summary
Read JSON files, format, and store them in a database
Motivation
JustInsight will need to have articles formated and stored to facilitate processing
Scope
None
Acceptance Criteria
-[ ] test that loads article and verifies the schema
Additional Context
- Create /etl/normalize.py to:
- Read raw JSON files from ./data/raw/
- Extract and clean fields: title, body, timestamp, source
- Insert into MongoDB collection articles
- Add connection config via environment variables
- Write a smoke test that loads one article and verifies the schema
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
No labels