Skip to content

Refactor into a modular, configurable CLI app with logging, retries, and tests #4

@jlwaldeck

Description

@jlwaldeck

Summary
The repository currently centers around a few top-level scripts (script.py, scheduler.py) and a large tickers.csv file. To make the app easier to maintain, test, and run in different environments, refactor it into a small package with clear modules, add a CLI, centralized configuration, structured logging, and basic tests/CI.

Why

  • Improve readability and separation of concerns (data I/O, strategy, execution, scheduling).
  • Make it easier to run the app locally, on a scheduler, or in CI with consistent config.
  • Gain observability (logs) and resiliency (retries/backoff, graceful error handling).
  • Enable incremental unit testing and safer future changes.

Proposed changes

  1. Restructure code into a package
  • Create an app/ directory and split responsibilities, for example:
    • app/config.py – load configuration from env/.env and provide defaults.
    • app/data.py – load/save tickers and any market data.
    • app/strategy.py – trading logic and signals.
    • app/broker.py – order placement/wrappers around any API used.
    • app/schedule.py – scheduling helpers used by scheduler.py.
    • app/main.py – CLI entry point.
  • Keep scheduler.py but make it import functions from app/ instead of calling script-level code.
  1. Add a CLI
  • Use argparse (or Typer/Click) to expose options like:
    • --symbols
    • --mode {paper,live}
    • --log-level {INFO,DEBUG}
    • --config <path to .env or YAML>
    • --once / --loop / --schedule
  • Example: python -m app --symbols tickers.csv --mode paper --once
  1. Centralized configuration
  • Load config from environment variables and optional .env (.env.example committed):
    • API_KEY, API_SECRET, PAPER_TRADING=true, TIMEZONE=America/New_York, etc.
  • Use python-dotenv to support .env during local development.
  1. Structured logging and error handling
  • Add logging with timestamps, levels, and module names; write to console and logs/app.log (RotatingFileHandler).
  • Wrap network/API calls with retries and backoff (e.g., tenacity); handle rate limits and timeouts.
  • Ensure exceptions are caught at the top level so the process exits with a clear message and non-zero status.
  1. Tests and CI
  • Add pytest and a tests/ directory with at least:
    • Unit tests for config loading (env overrides),
    • A simple strategy function test,
    • A data loader test for tickers.csv (happy path + missing file).
  • Add a minimal GitHub Actions workflow to run tests and lint on push/PR.
  1. Dependency and repo hygiene
  • Pin versions in requirements.txt (e.g., pandas==x.y.z, requests==x.y.z, python-dotenv==x.y.z, tenacity==x.y.z, pytest==x.y.z).
  • Move tickers.csv into a data/ directory and document its format; optionally provide a small sample (tickers.sample.csv) and ignore large files via .gitignore if they are regenerated.
  • Add a README with setup, usage examples, and the new CLI help output.

Acceptance criteria

  • Running python -m app --help shows available commands and options.
  • script.py logic is split across modules under app/ with type hints on public functions.
  • Logs are written to console and logs/app.log with INFO as default level.
  • API/network calls have retry/backoff and fail gracefully with clear messages.
  • pytest passes locally and in CI (GitHub Actions workflow green).
  • README documents setup, configuration, and examples for both one-off runs and scheduled runs.

Nice to have (optional)

  • Pre-commit hooks with black, isort, flake8/ruff, mypy.
  • Dockerfile for reproducible runs and scheduling.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions