Feat/on chain sync bg jobs

[caxtonacollins]

feat: On-Chain Sync & Background Jobs #65

Description

This PR implements durable background processing for blockchain events and notification deliveries to improve system scalability and reliability. It introduces a message queue-based architecture using BullMQ, an Event Bridge for Soroban RPC polling, and specialized Workers with exactly-once processing guarantees and retry mechanisms.

Key Changes

1. Queues and Workers

• Queue System: Replaced generic queue handling with specialized queues in QueueService:
  • email-queue: Handles email delivery jobs.
  • push-queue: Handles push notifications.
  • sync-queue: Handles critical payment synchronization (e.g., updating DB on PAY_DONE).
  • blockchain-tx-queue: Handles outbound transaction submission.
• Worker Implementation: Refactored src/workers/index.ts to instantiate separate workers for each queue, improving concurrency control and separation of concerns.

2. Event Bridge & Soroban Polling

• Event Bridge: Implemented EventBridgeService to poll Soroban contract events (PAY_DONE, TRANSFER_DONE) from the blockchain.
• Soroban Service: Added getEvents method in SorobanService leveraging @stellar/stellar-sdk to fetch contract events.
• Decoupling: The Event Bridge now pushes events to the sync-queue instead of directly updating the database, ensuring resilience against DB downtime.

3. Reliability & Idempotency

• Exactly-Once Processing: The Sync Worker implements an idempotency check:
  • Verifies if the Payment or Transfer is already marked as COMPLETED in the database.
  • Skips processing if already completed, preventing duplicate actions.
• Retry Mechanism: Configured default exponential backoff for all jobs (starts at 1s, max 5 attempts) in QueueService.

Integration Points

• Database: Uses prisma.payment.update to mark payments as COMPLETED.
• Dependencies: Requires a running Redis instance (configured via .env) and a valid SOROBAN_RPC_URL.

How to Test

1. Environment Setup: Ensure Redis is running and .env has correct REDIS_HOST/PORT.
2. Verify Sync Worker:
   • Use the provided verify_sync.ts script (or manually dispatch a SYNC job via Redis/BullMQ console).
   • Observe logs: Processing SYNC job -> Payment not found (if using dummy ID) or Payment marked as COMPLETED.
3. Run Dev Server: pnpm dev and check for "Event Bridge started" logs.

Checklist

• Implementation of specific queues (EMAIL, PUSH, SYNC).
• Proper worker separation and concurrency settings.
• Event Bridge polling implementation.
• Exactly-once logic in processSync.
• Retry strategy configuration.
• Verified with test scripts.

Related Issues

Closes #65

[caxtonacollins]

@anonfedora please review

[caxtonacollins]

@ONEONUORA please review

[caxtonacollins]

@ONEONUORA please review

[anonfedora]

fix merge conflicts, please

[anonfedora]

Hello, you still have conflicts

[caxtonacollins]

Hello, you still have conflicts
this is not nice at al

[anonfedora]

Can you make the CI pass?