README-postgres.md

Spring Boot Batch Processing Application

A minimized Spring Boot application for processing course invitations using batch processing with PostgreSQL database.

Features

• Batch Processing: Process multiple email invitations efficiently using Spring Batch
• Email Service: Mock email service that logs email content (can be configured for real sending)
• Course Management: Basic CRUD operations for courses
• Invitation Management: Create, validate, and resend course invitations
• PostgreSQL Database: Persistent database with Docker support
• REST API: Simple endpoints for testing

Fixed Issues

1. Multiple Email Processing: Fixed the issue where only one email was processed by properly parsing comma-separated email parameters in batch configuration
2. Batch Configuration: Simplified and fixed the batch reader, processor, and writer components
3. Exception Handling: Improved error handling and logging throughout the application
4. Minimized Structure: Removed unnecessary components and simplified the architecture
5. Database Migration: Migrated from H2 to PostgreSQL for better persistence

Quick Start

Prerequisites

• Java 11 or higher
• Maven 3.6 or higher
• Docker and Docker Compose

1. Start PostgreSQL Database

Start the PostgreSQL container using Docker Compose:

docker-compose up -d

This will:

• Start a PostgreSQL 15 container
• Create a database named batch_processing
• Use username batch_user and password batch_password
• Expose PostgreSQL on port 5432

2. Verify Database is Running

Check if PostgreSQL is running:

docker-compose ps

You should see the batch-processing-postgres container running.

3. Running the Application

1. Navigate to the project directory:

   cd /home/stark/dev/batch-processing

2. Compile and run the application:

   mvn clean compile
   mvn spring-boot:run

3. The application will start on http://localhost:8080

Sample Data

The application automatically creates sample data:

• 2 courses (IDs: 1, 2)
• 3 users (students and a teacher)

Testing the Batch Processing

1. Send Batch Invitations

curl -X POST http://localhost:8080/api/invitations/batch \
  -H "Content-Type: application/json" \
  -d '{
    "courseId": 1,
    "emails": ["test1@example.com", "test2@example.com", "test3@example.com"]
  }'

2. Send Single Invitations (Non-batch)

curl -X POST http://localhost:8080/api/invitations/single \
  -H "Content-Type: application/json" \
  -d '{
    "courseId": 1,
    "emails": ["test4@example.com", "test5@example.com"]
  }'

3. Validate an Invitation

First, check the logs for the invitation code generated, then:

curl -X POST "http://localhost:8080/api/invitations/validate?invitationCode=ABC123&email=test1@example.com&courseId=1"

4. Create a New Course

curl -X POST http://localhost:8080/api/courses \
  -H "Content-Type: application/json" \
  -d '{
    "title": "New Course",
    "description": "A new course description",
    "isPrivate": true
  }'

5. Get Course Details

curl -X GET http://localhost:8080/api/courses/1

Database Management

Connecting to PostgreSQL

Using psql command line:

docker exec -it batch-processing-postgres psql -U batch_user -d batch_processing

Useful SQL Queries

-- List all tables
\dt

-- Check invitations
SELECT * FROM student_course_invitation;

-- Check courses
SELECT * FROM course;

-- Check users
SELECT * FROM users;

-- Check batch job executions
SELECT * FROM batch_job_execution;

Stop and Clean Database

# Stop containers
docker-compose down

# Stop and remove volumes (clears all data)
docker-compose down -v

Configuration

Database Configuration

The application is configured to connect to PostgreSQL with these settings:

• Host: localhost:5432
• Database: batch_processing
• Username: batch_user
• Password: batch_password

These settings can be modified in:

• src/main/resources/application.yml (application config)
• docker-compose.yml (database container config)

Email Configuration

By default, the application logs email content instead of sending real emails. To enable real email sending:

1. Set environment variables:

   export MAIL_USERNAME=your-email@gmail.com
   export MAIL_PASSWORD=your-app-password

2. Uncomment the mailSender.send(message) lines in EmailService.java

Monitoring and Troubleshooting

View Application Logs

# If running with mvn spring-boot:run, logs appear in console
# For background processes:
tail -f /path/to/application.log

View Database Logs

docker-compose logs postgres

Check Database Health

# Using docker health check
docker-compose ps

# Manual connection test
docker exec batch-processing-postgres pg_isready -U batch_user -d batch_processing

Project Structure

src/main/java/com/example/batch/
├── BatchProcessingApplication.java          # Main application class
├── config/
│   ├── BatchConfiguration.java              # Basic batch configuration
│   ├── DataInitializer.java                 # Sample data initialization
│   └── StudentInvitationBatchConfig.java    # Batch job configuration
├── controller/
│   ├── CourseController.java                # Course REST endpoints
│   └── InvitationController.java            # Invitation REST endpoints
├── dto/
│   └── StudentInvitationRequest.java        # Request DTO
├── model/
│   ├── Course.java                          # Course entity
│   ├── StudentCourseInvitation.java         # Invitation entity
│   └── User.java                            # User entity
├── repository/
│   ├── CourseRepository.java                # Course data access
│   ├── StudentCourseInvitationRepository.java # Invitation data access
│   └── UserRepository.java                  # User data access
├── service/
│   ├── CourseService.java                   # Course business logic
│   ├── EmailService.java                    # Email sending service
│   ├── InvitationBatchService.java          # Batch job launcher
│   └── StudentCourseInvitationService.java  # Invitation business logic
└── util/
    └── CodeGenerator.java                   # Utility for generating codes

Docker Files:
├── docker-compose.yml                       # PostgreSQL container configuration
└── init-db.sql                             # Database initialization script

Architecture Overview

• Spring Boot 2.7.18: Compatible with Java 11
• Spring Batch: For processing multiple email invitations efficiently
• Spring Data JPA: For database operations with Hibernate
• PostgreSQL: Persistent database with Docker support
• REST API: Simple endpoints for testing and integration
• Docker Compose: Easy database setup and management

Key Improvements Made

1. Fixed Multiple Email Processing: The main issue where only one email was processed is now resolved by properly parsing comma-separated email parameters in the batch configuration
2. Java 11 Compatibility: Downgraded from Spring Boot 3.x to 2.7.18 and fixed text block syntax
3. PostgreSQL Integration: Migrated from H2 to PostgreSQL for better persistence and production readiness
4. Docker Support: Added Docker Compose for easy database setup
5. Minimized Structure: Removed unnecessary components and simplified the architecture
6. Improved Error Handling: Centralized error handling with proper logging throughout the application
7. Mock Email Service: Easy to test without requiring real email configuration - logs email content instead
8. Automatic Sample Data: Ready-to-use test data (2 courses, 3 users) created on startup
9. Fixed Batch Configuration: Properly handles comma-separated emails and processes them in chunks
10. Enhanced Logging: Better visibility into batch processing steps and email sending

Testing Multiple Email Processing

The application now correctly processes multiple emails in batch operations. You can verify this by:

1. Sending a batch invitation with multiple emails
2. Checking the application logs for processing of each email
3. Querying the database to see all invitation records created
4. Verifying that each email gets a unique invitation code

Development Tips

• Use docker-compose logs -f postgres to monitor database logs
• Use mvn spring-boot:run -Dspring-boot.run.profiles=dev for development profile
• Check /batch_job_execution table for batch job status and results
• Monitor the application logs to see detailed batch processing information