Skip to content

mablic/ML_Trade_Model

BranchesTags

Repository files navigation

QuantModel Backend

A production-ready, cloud-deployed machine learning trading system that uses LSTM neural networks to predict stock price movements and execute automated trading decisions. The system runs on Google Cloud Run with automated scheduling and integrates with Firebase for data persistence.

🚀 Overview

QuantModelBackend is a fully automated Python-based trading system that:

  • Trains LSTM models on historical stock data with hyperparameter optimization
  • Tests model stability through Monte Carlo simulations
  • Predicts trading signals every 15 minutes during market hours
  • Executes automated trading decisions based on ensemble model predictions
  • Stores all data, models, and results in Firebase (Firestore + Storage)
  • Deploys on Google Cloud Run with automated scheduling via Cloud Scheduler

📋 Table of Contents

🏗️ Architecture

QuantModelBackend/
├── models/                    # Neural network model definitions
│   └── lstm_strategy_a.py     # LSTM model architecture
├── training/                  # Model training and evaluation
│   ├── trainer.py             # LSTM_Trainer class
│   └── __init__.py
├── data/                      # Data loading and preprocessing
│   └── data_loader.py         # DataLoader for yfinance integration
├── lib/                       # Firebase and utility modules
│   ├── firebase_init.py       # Firebase Admin SDK initialization
│   ├── save_model_to_firestore.py    # Model upload/download
│   └── save_next_result_to_firebase.py  # Position and trade data storage
├── cloud_run_server.py        # Cloud Run HTTP server
├── main.py                    # Core trading logic and functions
├── Dockerfile                 # Container configuration
├── requirements.txt           # Python dependencies
├── deploy_cloud_run.sh        # Deployment script
└── setup_cloud_run_schedulers.sh  # Scheduler setup script

System Flow

┌─────────────────┐
│ Cloud Scheduler │  (Every 15 min, Mon-Fri, 8:01 AM - 4:46 PM CST)
└────────┬────────┘
         │ HTTP POST
         ▼
┌─────────────────┐
│  Cloud Run      │  (trading-predictions service)
│  HTTP Server    │
└────────┬────────┘
         │
         ▼
┌─────────────────┐
│  main.py        │  make_predictions_on_next_data()
│  - Fetch data   │
│  - Run models   │
│  - Make decision│
└────────┬────────┘
         │
         ▼
┌─────────────────┐
│   Firebase      │
│  - Firestore    │  (positions, trades, strategy data)
│  - Storage      │  (model files)
└─────────────────┘

🔧 Core Components

1. LSTM_Strategy_A_Selection()

Performs comprehensive model selection and training:

  • Hyperparameter Search: Grid search over:

    • mv_low: Moving average lookback period (5-15)
    • mv_high: Moving average window (6-20)
    • lookback_periods: LSTM sequence length (3-13)

  • Stability Testing: Runs 100 Monte Carlo simulations per configuration

  • Model Selection: Selects top 3 models based on:

    • Mean accuracy across runs
    • Standard deviation (lower = more stable)
    • Consistent performance

  • Output:

    • Saves top 3 models to Firebase Storage
    • Stores model parameters in Firestore
    • Updates strategy configuration

Usage:

from main import LSTM_Strategy_A_Selection
LSTM_Strategy_A_Selection()  # Takes ~30-60 minutes

2. make_predictions_on_next_data()

Main prediction function called by Cloud Run:

  • Data Fetching: Loads latest 15-minute market data

  • Feature Engineering: Computes:

    • MV_Change_Percent: Moving average trend
    • Volume_Change: Volume momentum
    • Direction: Price direction indicator

  • Ensemble Prediction:

    • Runs all 3 trained models
    • Aggregates predictions (majority voting)
    • Signal: BUY if ≥2 models predict UP, else HOLD/SELL

  • Firebase Storage:

    • Saves prediction results to Firestore
    • Updates position data
    • Records trade history

Output: Boolean (True = BUY, False = HOLD)

Called by: Cloud Run /weekday endpoint (automated every 15 minutes)

3. backtest()

Generates historical predictions for analysis:

  • Loads 50 days of 15-minute data
  • Generates predictions for each candle
  • Returns DataFrame with:
    • Date, Close price
    • Individual model predictions
    • Aggregated signal (Result_Sum)

4. calculate_profit(df_results)

Calculates P&L from backtest results:

  • Simulates trades: Entry on BUY signal, Exit on SELL signal
  • Calculates profit per trade
  • Returns total cumulative profit

🚢 Deployment

Prerequisites

  1. Google Cloud SDK: Install and authenticate

    gcloud auth login
gcloud config set project autotrader-5998a

  2. Firebase Admin SDK: Service account key

    • Place lib/autotrader.json in the lib/ directory
    • Or use default credentials in Cloud Run

  3. Docker: For building container images

Deploy to Cloud Run

Option 1: Using deployment script (Recommended)

cd QuantModelBackend
chmod +x deploy_cloud_run.sh
./deploy_cloud_run.sh

Option 2: Manual deployment

# Build and deploy
gcloud builds submit --tag gcr.io/autotrader-5998a/trading-predictions
gcloud run deploy trading-predictions \
  --image gcr.io/autotrader-5998a/trading-predictions \
  --platform managed \
  --region us-central1 \
  --memory 2Gi \
  --cpu 2 \
  --timeout 3600 \
  --allow-unauthenticated

Setup Cloud Scheduler

After deploying Cloud Run, set up automated scheduling:

chmod +x setup_cloud_run_schedulers.sh
./setup_cloud_run_schedulers.sh

This creates a Cloud Scheduler job that:

  • Triggers every 15 minutes during market hours
  • Schedule: 1,16,31,46 8-16 * * 1-5 (CST, Monday-Friday)
  • Calls /weekday endpoint on Cloud Run service

Verify Deployment

  1. Check Cloud Run service:

    gcloud run services describe trading-predictions --region us-central1

  2. View logs:

    gcloud run services logs read trading-predictions --region us-central1 --limit 50

  3. Test manually:

    curl -X POST https://trading-predictions-xxx.run.app/weekday

🔥 Firebase Integration

Firestore Collections

positions/{strategy_name}:

  • current_position: Current holding (shares)
  • running_pnl: Unrealized P&L
  • realized_pnl: Total realized profit/loss
  • data: 15-minute candlestick data
  • history_trades: Completed trade history
  • params: Model parameters (MA periods, etc.)

strategies/{strategy_name}:

  • Strategy metadata and configuration
  • Model parameters
  • Performance metrics

Firebase Storage

models/:

  • {model_name}_{model_number}.pth: PyTorch model weights
  • Models are uploaded after training
  • Downloaded on-demand for predictions

Firebase Initialization

The system uses lib/firebase_init.py which:

  • Automatically detects local vs. cloud environment
  • Uses service account key locally (lib/autotrader.json)
  • Uses default credentials in Cloud Run
  • Handles lazy initialization to prevent startup errors

⏰ Automated Scheduling

Current Schedule

Weekday Predictions:

  • Frequency: Every 15 minutes
  • Hours: 8:01 AM - 4:46 PM CST
  • Days: Monday - Friday
  • Cron: 1,16,31,46 8-16 * * 1-5
  • Endpoint: /weekday
  • Function: make_predictions_on_next_data()

Scheduler Configuration

Managed via setup_cloud_run_schedulers.sh:

  • Creates HTTP-triggered Cloud Scheduler job
  • Uses OIDC authentication
  • 10-minute timeout
  • 3 retry attempts

💻 Local Development

Setup

  1. Install dependencies:

    cd QuantModelBackend
python -m venv .venv
source .venv/bin/activate  # On Windows: .venv\Scripts\activate
pip install -r requirements.txt

  2. Configure Firebase:

    • Place autotrader.json in lib/ directory
    • Or set GOOGLE_APPLICATION_CREDENTIALS environment variable

  3. Run locally:

    from main import make_predictions_on_next_data
result = make_predictions_on_next_data()
print(f"Should buy: {result}")

Testing Cloud Run Locally

# Install functions-framework
pip install functions-framework

# Run locally
functions-framework --target=run_server --source=cloud_run_server.py --port=8080

# Test endpoint
curl -X POST http://localhost:8080/weekday

🌐 API Endpoints

Cloud Run Endpoints

POST /weekday

  • Runs make_predictions_on_next_data()
  • Returns JSON response with status
  • Called by Cloud Scheduler

GET /health

  • Health check endpoint
  • Returns {"status": "healthy"}

Response Format:

{
  "status": "success",
  "message": "Weekday predictions completed and saved to Firebase",
  "timestamp": "2025-12-30T14:30:00"
}

⚙️ Configuration

Environment Variables

  • GOOGLE_CLOUD_PROJECT: Project ID (default: autotrader-5998a)
  • GOOGLE_APPLICATION_CREDENTIALS: Path to service account key (local only)
  • PORT: Server port (default: 8080)

Model Configuration

Edit constants in main.py:

  • MODEL_TICKER: Stock symbol (default: "AAPL")
  • INTERVAL: Data interval (default: "15m")
  • MV_LOW_RANGE, MV_HIGH_RANGE: Moving average ranges
  • LOOKBACK_PERIODS_RANGE: LSTM sequence length range
  • SIMULATIONS: Number of stability tests (default: 100)

Cloud Run Configuration

Edit deploy_cloud_run.sh:

  • PROJECT_ID: Google Cloud project ID
  • REGION: Deployment region
  • SERVICE_NAME: Cloud Run service name
  • MEMORY: Container memory (default: 2Gi)
  • CPU: Container CPU (default: 2)
  • TIMEOUT: Request timeout (default: 3600s)

📊 Model Architecture

LSTM_Strategy_A:

  • Input: 3 features (MV_Change_Percent, Volume_Change, Direction)
  • Architecture:
    • LSTM layer: 64 hidden units
    • Dropout: 0.2
    • Fully connected output layer
  • Output: Binary classification (0 = DOWN, 1 = UP)
  • Loss: Binary Cross-Entropy
  • Optimizer: Adam

📈 Data Source

  • Provider: yfinance (Yahoo Finance API)
  • Ticker: AAPL (configurable)
  • Interval: 15-minute candles
  • Lookback:
    • Training: 50 days
    • Prediction: 1 day (latest data)

🔍 Monitoring & Logging

Cloud Run Logs

View logs in Google Cloud Console:

gcloud run services logs read trading-predictions --region us-central1

Firebase Console

  • Firestore: Monitor position updates and trade history
  • Storage: Check model uploads/downloads
  • Usage: Track API calls and data reads

🛠️ Troubleshooting

Common Issues

1. Model not found error

  • Ensure models are uploaded to Firebase Storage
  • Run LSTM_Strategy_A_Selection() to train and upload models

2. Firebase initialization error

  • Check service account key exists (lib/autotrader.json)
  • Verify Firebase Admin SDK credentials
  • Check project ID matches in firebase_init.py

3. Cloud Run deployment fails

  • Verify Dockerfile is correct
  • Check requirements.txt has all dependencies
  • Ensure Cloud Build API is enabled

4. Scheduler not triggering

  • Verify Cloud Scheduler job exists
  • Check job schedule is correct
  • Verify Cloud Run service URL is correct
  • Check OIDC service account permissions

5. Timeout errors

  • Increase Cloud Run timeout (max: 3600s)
  • Check prediction function isn't hanging
  • Review logs for slow operations

📦 Dependencies

See requirements.txt for full list:

  • torch>=2.0.0: PyTorch deep learning framework
  • yfinance>=0.2.0: Stock market data
  • pandas>=1.3.0: Data manipulation
  • numpy>=1.21.0: Numerical computing
  • firebase-admin>=6.5.0: Firebase integration
  • functions-framework>=3.0.0: Cloud Run server
  • scikit-learn>=1.0.0: Feature scaling
  • pytz>=2021.3: Timezone handling

🔐 Security

  • Service account keys stored securely (not in git)
  • Cloud Run uses OIDC authentication for scheduler
  • Firebase security rules restrict access
  • Environment variables for sensitive data

📝 License

Private project - All rights reserved

Last Updated: December 2025
Version: 1.0.0
Status: Production

About

ML_Trade_Model

Resources

Readme
Activity

Stars

0 stars

Watchers

0 watching

Forks

0 forks
Report repository

Releases

No releases published

Packages

No packages published

Languages