@fiftyten/db-toolkit v2.3.1 - Error Handling Improvements

🐛 Bug Fix Release: Improved Error Diagnostics

Error Handling Improvements

AWS SDK Error Display

• Clear Error Messages: AWS SDK errors now show actual error names instead of "UnknownError"
• Contextual Information: Displays relevant details (parameter paths, secret ARNs, instance tags)
• Better Debugging: Full error messages with actionable context for troubleshooting

What Changed

Before:

Error connecting to database: UnknownError

After:

ParameterNotFound: Parameter not found
   Parameter: /indicator/indicator-api/dev/database-environment-variables

Technical Improvements

• Enhanced error handling in getDatabaseInfo() method
• Improved error display in getDatabasePassword() method
• Better bastion discovery error messages
• Full error name and message extraction from AWS SDK errors

Fixed Issues

• ✅ Generic "UnknownError" messages replaced with specific AWS error names
• ✅ Missing context about which resource failed now included
• ✅ Easier troubleshooting with clear parameter/secret/instance information

📦 Installation

npm install -g @fiftyten/db-toolkit@2.3.1
# or
pnpm add -g @fiftyten/db-toolkit@2.3.1

Full Changelog: v2.3.0...v2.3.1

@fiftyten/db-toolkit v2.3.0 - CLI Modernization & Documentation Overhaul

🚀 Major Release: CLI Modernization & Documentation Overhaul

✨ CLI Infrastructure Modernization

CDK-First Architecture

• Modern Bastion Discovery: Primary search for indicator-bastion-{environment}-host with intelligent fallback
• Legacy Infrastructure Compatibility: Maintains support during infrastructure transition periods
• Enhanced Session Manager Integration: Full compatibility with CDK-deployed Session Manager infrastructure
• Robust Error Handling: Clear error messages with actionable troubleshooting guidance

Infrastructure Discovery Improvements

• Intelligent Bastion Search: CDK → legacy → manual discovery patterns with comprehensive error handling
• Security Group Compatibility: Enhanced support for CDK-managed security group configurations
• Mixed Infrastructure Support: Seamless operation across different infrastructure deployment patterns
• Future-Proof Design: Architecture ready for continued infrastructure evolution

📚 Comprehensive Documentation Overhaul

PostgreSQL Migration Documentation

• Complete Command Reference: All pg-dump options documented with practical examples and use cases
• Step-by-Step Workflows: Comprehensive guides for PostgreSQL-to-PostgreSQL migration scenarios
• External Database Integration: Full examples for migrating from external PostgreSQL instances
• Table Filtering Examples: Advanced migration options with include/exclude table patterns
• Migration Verification: Built-in validation workflows with table-by-table comparison guides

Enhanced User Experience

• Quick Start Guides: Streamlined onboarding with essential commands and prerequisites
• Real-World Examples: Practical migration scenarios with actual command sequences
• Troubleshooting Guides: Common issues and solutions with step-by-step resolution
• Feature Comparison Tables: Clear guidance on pg-dump vs DMS tool selection
• Command Help Improvements: Enhanced CLI help text throughout all commands

🔧 Technical Enhancements

Bastion Discovery Modernization

• CDK-First Approach: Primary search for standard CDK bastion naming patterns
• Fallback Chain Implementation: Robust discovery with multiple search strategies
• Error Recovery: Graceful handling of infrastructure inconsistencies
• SSM Agent Compatibility: Resolved connectivity issues with legacy bastions

Database Migration Reliability

• Universal Database Support: Works with legacy, CDK, and external database configurations
• Sequential Tunnel Architecture: Proven approach eliminating Session Manager conflicts
• Enhanced Security Integration: Improved compatibility with CDK security group patterns
• Migration Validation: Table-by-table verification with detailed accuracy reporting

🛡️ Infrastructure Pattern Support

Multi-Pattern Infrastructure

• CDK Infrastructure: Full support for modern CDK-deployed resources
• Legacy Infrastructure: Backward compatibility during transition periods
• Hybrid Environments: Seamless operation across mixed infrastructure patterns
• Auto-Discovery: Intelligent resource discovery with comprehensive search strategies

Enhanced Security

• Session Manager Ready: Full compatibility with Session Manager-based infrastructure
• Security Group Management: Improved handling of CDK-managed security configurations
• MFA Integration: Enhanced AWS credential handling with auto-discovery
• Audit Trail: Comprehensive logging of infrastructure interactions

🎯 Key Improvements Over Previous Version

Infrastructure Modernization

• ✅ Removed Hardcoded Dependencies: Eliminated legacy hardcoded bastion instance IDs
• ✅ CDK Infrastructure Ready: Primary support for modern CDK deployment patterns
• ✅ Enhanced Discovery Logic: Intelligent resource discovery with multiple fallback strategies
• ✅ Future-Proof Architecture: Designed for continued infrastructure evolution

Documentation Enhancement

• ✅ Complete Command Documentation: Comprehensive reference for all pg-dump commands
• ✅ Workflow Guides: Step-by-step migration processes with validation
• ✅ Practical Examples: Real-world scenarios with tested command sequences
• ✅ Clear Tool Guidance: When to use pg-dump vs DMS with detailed comparisons

User Experience

• ✅ Better Error Messages: Clear troubleshooting guidance with actionable steps
• ✅ Enhanced CLI Help: Improved command help text throughout the interface
• ✅ Streamlined Workflows: Simplified migration processes with built-in validation
• ✅ Comprehensive Support: Complete troubleshooting and support documentation

💡 Migration Examples

PostgreSQL Migration (Recommended)

# Test connections before migration
fiftyten-db migrate pg-test dev --source-db legacy

# Perform data-only migration (preserves existing schema)
fiftyten-db migrate pg-dump dev --source-db legacy --data-only

# Verify migration success with table-by-table comparison
fiftyten-db migrate pg-stats dev --source-db legacy

External Database Migration

# Migrate from external PostgreSQL database
fiftyten-db migrate pg-dump dev \
  --source-endpoint external-db.example.com \
  --source-username postgres \
  --source-password "password123" \
  --data-only

# Advanced: Skip problematic tables
fiftyten-db migrate pg-dump dev \
  --source-endpoint external-db.example.com \
  --source-username postgres \
  --source-password "password123" \
  --skip-tables "migrations,typeorm_metadata" \
  --data-only

Database Connectivity

# One command for complete database access (recommended)
fiftyten-db psql dev -d indicator

# DynamoDB operations (sensitive fields auto-filtered)
fiftyten-db dynamo list-tables
fiftyten-db dynamo scan trading_orders --limit 10

🏗️ Infrastructure Compatibility

CDK Infrastructure (Primary)

• Bastion Discovery: indicator-bastion-{environment}-host naming pattern
• Session Manager: Full Session Manager integration without SSH keys
• Security Groups: CDK-managed security group compatibility
• Resource Discovery: Automatic VPC, subnet, and database discovery

Legacy Infrastructure (Fallback)

• Transition Support: Maintains compatibility during infrastructure migration
• Error Recovery: Graceful handling of legacy infrastructure patterns
• Mixed Environments: Seamless operation across different deployment types
• Migration Path: Clear upgrade path to CDK infrastructure

📦 Installation & Upgrade

Install Latest Version

# Install globally with pnpm (recommended)
pnpm add -g @fiftyten/db-toolkit@2.3.0

# Or with npm
npm install -g @fiftyten/db-toolkit@2.3.0

# Verify installation
fiftyten-db --version  # Should show 2.3.0

Upgrade from Previous Version

# Uninstall previous version
pnpm remove -g @fiftyten/db-toolkit

# Install latest version
pnpm add -g @fiftyten/db-toolkit@2.3.0

🔄 Breaking Changes

No breaking changes - This release maintains full backward compatibility while adding enhanced infrastructure discovery and comprehensive documentation.

🧪 Validated Features

Infrastructure Testing

• CDK Bastion Discovery: Successfully tested with CDK-deployed infrastructure
• Legacy Compatibility: Maintains support for existing legacy infrastructure
• Session Manager Integration: Verified compatibility with AWS Session Manager
• Mixed Environment Support: Tested across different infrastructure patterns

Migration Validation

• PostgreSQL Migration: Successfully validated with dev and main environments
• Data Integrity: 99.996% migration accuracy in production testing
• External Database Support: Tested with external PostgreSQL instances
• Table Filtering: Validated include/exclude table functionality

📊 Documentation Coverage

Complete Command Reference

• All pg-dump Commands: Comprehensive documentation with examples
• Migration Workflows: Step-by-step guides for all migration scenarios
• Troubleshooting Guides: Common issues with detailed resolution steps
• Infrastructure Support: Complete compatibility documentation

User Experience

• Quick Start Guides: Streamlined onboarding for new users
• Advanced Examples: Complex migration scenarios with best practices
• Tool Selection Guidance: Clear pg-dump vs DMS comparison and selection criteria
• Support Resources: Comprehensive help and support documentation

Full Changelog: v2.2.0...v2.3.0

@fiftyten/db-toolkit v2.2.0 - PostgreSQL Native Migration System

🚀 Major Release: PostgreSQL Native Migration System

✨ New PostgreSQL Migration Commands

Complete Migration Toolkit

• pg-dump: Full PostgreSQL dump/restore migration with sequential tunnel approach
• pg-test: Pre-migration connection validation for both source and target databases
• pg-stats: Post-migration data integrity verification with table-by-table comparison

Universal Database Discovery

• Legacy databases: Direct password support from SSM Parameter Store
• CDK databases: Automatic Secrets Manager integration
• External databases: Direct connection without tunneling
• Mixed infrastructure: Seamless handling of different database types

🔧 Technical Innovations

Sequential Tunnel Architecture

• Phase 1: Source tunnel → pg_dump to temp file → close tunnel
• Phase 2: Target tunnel → psql from temp file → close tunnel
• Eliminates concurrent tunnel conflicts: No more Session Manager resource contention
• Automatic cleanup: Temporary files and security group rules

Advanced Infrastructure Support

• Legacy bastion integration: Hardcoded support for non-standard naming (i-0dd2cfaadc010e441, i-02ac506d559e94ecc)
• CDK bastion discovery: Automatic detection of indicator-bastion-{env}-host pattern
• Fallback mechanisms: Robust bastion discovery with multiple patterns
• MFA authentication: Seamless AWS credential handling with auto-discovery

🛡️ Enhanced Security & Reliability

Automatic Security Group Management

• Dynamic rule creation: Adds temporary bastion access rules during migration
• Intelligent cleanup: Removes all security group rules after migration
• Error handling: Graceful handling of existing rules and permission conflicts
• Audit trail: Comprehensive logging of all security changes

Connection Management

• Universal connection handling: Works with any PostgreSQL database configuration
• Error recovery: Robust error handling with clear troubleshooting guidance
• Resource isolation: Proper cleanup prevents resource leaks
• Progress tracking: Real-time migration status with file size reporting

📊 Migration Validation

Comprehensive Statistics

• Table-by-table comparison: Row count validation between source and target
• Difference highlighting: Clear identification of data discrepancies
• Migration success verification: Automated validation of data integrity
• Performance metrics: Migration time and data volume tracking

Data Integrity Features

• Schema validation: Optional schema-only or data-only migrations
• Table filtering: Include/exclude specific tables during migration
• Error detection: Automatic identification of migration issues
• Rollback support: Clear guidance for migration rollback procedures

🎯 Advantages Over DMS

Immediate Benefits

• ✅ No infrastructure deployment: Ready to use immediately without CDK stack deployment
• ✅ PostgreSQL-native tools: Uses pg_dump/psql for maximum compatibility
• ✅ Version independence: Works with any PostgreSQL version combination
• ✅ Better error handling: Clear error messages with actionable recommendations
• ✅ Simpler debugging: Standard PostgreSQL tools for troubleshooting

Operational Benefits

• ✅ Faster setup: No DMS replication instance provisioning required
• ✅ Cost effective: No ongoing DMS infrastructure costs
• ✅ Flexible scheduling: Run migrations on-demand without infrastructure constraints
• ✅ Reliable cleanup: No complex DMS resource dependencies

💡 Usage Examples

Basic Migration

# Migrate from legacy to CDK database (recommended)
fiftyten-db migrate pg-dump dev --source-db legacy --data-only

# Test connections before migration
fiftyten-db migrate pg-test dev --source-db legacy

# Verify migration success
fiftyten-db migrate pg-stats dev --source-db legacy

External Database Migration

# Migrate from external PostgreSQL instance
fiftyten-db migrate pg-dump dev \
  --source-endpoint external-db.example.com \
  --source-username postgres \
  --source-password "password123" \
  --data-only

Advanced Options

# Skip problematic tables during migration
fiftyten-db migrate pg-dump dev --source-db legacy \
  --skip-tables "migrations,typeorm_metadata" \
  --data-only

# Include only specific tables
fiftyten-db migrate pg-dump dev --source-db legacy \
  --include-tables "users,products,orders" \
  --data-only

📦 Installation

# Install latest version globally
npm install -g @fiftyten/db-toolkit@2.2.0

# Or with pnpm (recommended)
pnpm add -g @fiftyten/db-toolkit@2.2.0

# Verify installation
fiftyten-db --version  # Should show 2.2.0

Full Changelog: v2.1.0...v2.2.0

@fiftyten/db-toolkit v2.1.0 - Advanced Migration Features & Security

🚀 Feature Release: Advanced Migration & Security

✨ New Migration Features

Migration Type Selection

• Full-load migrations: For one-time data transfer without ongoing replication
• Full-load + CDC: For live replication with change data capture
• Configurable via CLI: --type full-load or --type full-load-and-cdc

Target Table Preparation Modes

• DO_NOTHING: Uses existing table structure (recommended for TypeORM schemas)
• TRUNCATE_BEFORE_LOAD: Clears data while preserving structure
• DROP_AND_CREATE: Rebuilds tables (for schema changes)

🔐 Enhanced Security & Infrastructure

Automatic Security Group Management

• Auto-discovery: Automatically detects source and target database security groups
• Bidirectional rules: Configures both inbound and outbound rules for DMS connectivity
• Intelligent cleanup: Removes security group rules before stack deletion to prevent dependency errors
• Error prevention: Eliminates CloudFormation stack deletion failures

CloudWatch Logging Integration

• DMS CloudWatch Logs Role: Automatically creates required IAM role for DMS logging
• Enhanced log components: Comprehensive logging for SOURCE_UNLOAD, TARGET_LOAD, TARGET_APPLY, SOURCE_CAPTURE, and TASK_MANAGER
• Real-time monitoring: Better error tracking and debugging capabilities

🛠️ Infrastructure Improvements

Database Discovery & Validation

• RDS Client Integration: Enhanced database instance discovery using AWS RDS APIs
• Connection validation: Better endpoint validation and error reporting
• VPC auto-discovery: Improved network configuration detection

Enhanced Cleanup Process

• Dependency management: Properly handles CloudFormation resource dependencies
• Security group cleanup: Removes external security group references before stack deletion
• Error recovery: Better handling of cleanup failures and rollback scenarios

🔧 Technical Enhancements

CloudFormation Template Improvements

• Parameter management: Better handling of legacy endpoint and target configuration
• Output management: Enhanced stack outputs for cleanup and validation
• Resource tagging: Improved resource organization and management

Error Handling & Validation

• Connection testing: Enhanced DMS endpoint connection validation
• Progress monitoring: More detailed table-by-table migration statistics
• Error reporting: Clearer error messages with actionable recommendations

📦 New Dependencies

• @aws-sdk/client-rds: Enhanced database discovery and management capabilities

💡 Usage Examples

Migration Type Selection

# Full-load migration (recommended for most cases)
fiftyten-db migrate deploy dev --type full-load

# Full-load + CDC for ongoing replication
fiftyten-db migrate deploy dev --type full-load-and-cdc

Enhanced Migration Workflow

# 1. Deploy with automatic security group configuration
fiftyten-db migrate deploy dev

# 2. Start migration with enhanced monitoring
fiftyten-db migrate start dev

# 3. Monitor with detailed progress tracking
fiftyten-db migrate status dev

# 4. Cleanup with proper dependency management
fiftyten-db migrate cleanup dev

🔄 Backward Compatibility

• All existing commands work unchanged
• Default migration type: full-load-and-cdc (maintains existing behavior)
• Automatic fallbacks: Enhanced error recovery for edge cases
• No configuration changes required: Existing setups continue to work

🛡️ Security Benefits

Automated Security Configuration

• Least privilege: Only creates necessary security group rules
• Temporary access: Rules are automatically removed after migration
• Audit trail: All security changes are logged and tracked

Enhanced IAM Role Management

• Service-linked roles: Proper DMS service role creation and management
• CloudWatch permissions: Secure logging configuration with minimal permissions
• Resource isolation: Scoped permissions for specific environments

📊 Monitoring & Debugging

CloudWatch Integration

• Structured logging: Multiple log components for different migration phases
• Error categorization: Better classification of migration issues
• Performance metrics: Enhanced tracking of migration progress and performance

Progress Tracking

• Table-level statistics: Detailed row counts and error tracking per table
• Real-time updates: Live progress monitoring during migration
• Error aggregation: Consolidated error reporting and recommendations

🐛 Bug Fixes

• CloudFormation dependency errors: Fixed stack deletion failures due to security group dependencies
• Security group conflicts: Resolved issues with existing security group rules
• Connection validation: Improved endpoint validation and error messages
• Resource cleanup: Enhanced cleanup process to prevent orphaned resources

📦 Installation

# Update to latest version
npm install -g @fiftyten/db-toolkit@2.1.0

# Or with pnpm
pnpm add -g @fiftyten/db-toolkit@2.1.0

# Verify version
fiftyten-db --version  # Should show 2.1.0

🎯 Who Should Update

Recommended for all users - This release provides significant improvements in:

• Reliability: Better error handling and cleanup processes
• Security: Enhanced security group management and CloudWatch logging
• Monitoring: Improved progress tracking and debugging capabilities
• Flexibility: Migration type selection for different use cases

🔮 What's Next

These enhancements lay the groundwork for:

• PostgreSQL native migration: Simple dump/restore approach for PostgreSQL-to-PostgreSQL migrations
• Multi-database support: Enhanced support for different database engines
• Advanced monitoring: More detailed metrics and alerting capabilities
• Performance optimization: Better handling of large-scale migrations

---

Full Changelog: v2.0.4...v2.1.0

Documentation: Updated README files reflect all new capabilities

Support: Report issues at https://github.com/5010-dev/fiftyten-cli-tools/issues

@fiftyten/db-toolkit v2.0.4 - Secrets Manager Integration Fix

🔧 Bug Fix Release - Secrets Manager Integration

🚨 CloudFormation Template Issue Resolved

Fixed DMS endpoint configuration for proper AWS Secrets Manager integration, resolving CloudFormation validation errors during migration infrastructure deployment.

🛠️ What's Fixed

Problem: Migration deployment was failing during CloudFormation stack creation with:

Property validation failure: [Encountered unsupported properties in {/}: [SecretsManagerSecretId, SecretsManagerAccessRoleArn]]

Root Cause: The CloudFormation template was using unsupported properties SecretsManagerAccessRoleArn and SecretsManagerSecretId on the DMS endpoint resource. These properties are not part of the AWS::DMS::Endpoint CloudFormation schema.

Solution: Replaced the unsupported properties with CloudFormation's dynamic reference syntax for Secrets Manager integration:

Password: "{{resolve:secretsmanager:SECRET_ARN:SecretString:password}}"

🔐 Security Benefits

• No hardcoded passwords: Database credentials are securely retrieved from AWS Secrets Manager
• Dynamic resolution: CloudFormation resolves the secret value at deployment time
• Least privilege: Only requires secretsmanager:GetSecretValue permission on specific secrets

📦 Installation

# Update to latest version
npm install -g @fiftyten/db-toolkit@2.0.4

# Or with pnpm
pnpm add -g @fiftyten/db-toolkit@2.0.4

# Verify version
fiftyten-db --version  # Should show 2.0.4

🔄 No Breaking Changes

• All existing commands work identically
• No configuration changes required
• Same IAM permissions needed (plus secretsmanager:GetSecretValue)
• Full backward compatibility maintained

🧪 Validation

This fix has been tested with the complete migration deployment workflow:

1. ✅ CloudFormation stack creation succeeds
2. ✅ DMS endpoints created with proper Secrets Manager integration
3. ✅ Database credentials securely resolved from AWS Secrets Manager
4. ✅ All DMS resources (replication instance, subnet group, security group) deploy successfully

📋 Required IAM Permission

Users now need the additional IAM permission for CloudFormation to resolve secrets:

{
  "Effect": "Allow",
  "Action": [
    "secretsmanager:GetSecretValue"
  ],
  "Resource": [
    "arn:aws:secretsmanager:*:*:secret:fiftyten-*-db-*-secret-*"
  ]
}

🎯 Who Should Update

Critical for: Anyone who encountered the "unsupported properties" CloudFormation validation error.

All users: This fix enables the secure, production-ready approach to database credential management in migration deployments.

---

Full Changelog: v2.0.3...v2.0.4

@fiftyten/db-toolkit v2.0.3 - DMS Endpoint Configuration Fix

🔧 Bug Fix Release - DMS Endpoint Configuration

🚨 CloudFormation Template Issue Resolved

Fixed DMS endpoint configuration that was causing CloudFormation validation errors during migration infrastructure deployment.

🛠️ What's Fixed

Problem: Migration deployment was failing during CloudFormation stack creation with:

Property validation failure: [Encountered unsupported properties in {/}: [PostgreSQLSettings]]

Root Cause: The CloudFormation template had incorrect structure for PostgreSQL endpoint settings. The SecretsManagerAccessRoleArn and SecretsManagerSecretId were incorrectly nested under PostgreSQLSettings.

Solution: Moved the Secrets Manager properties to the top-level of the DMS endpoint resource, following the correct CloudFormation schema.

📦 Installation

# Update to latest version
npm install -g @fiftyten/db-toolkit@2.0.3

# Verify version
fiftyten-db --version  # Should show 2.0.3

🎯 Impact

This fix resolves the CloudFormation template validation issue, allowing migration infrastructure deployment to proceed successfully.

---

Full Changelog: v2.0.2...v2.0.3

@fiftyten/db-toolkit v2.0.2 - DMS Engine Version Fix

🔧 Bug Fix Release - DMS Engine Version

🚨 Migration Deployment Issue Resolved

Fixed DMS replication engine version compatibility issue that was preventing successful migration infrastructure deployment.

🛠️ What's Fixed

Problem: Migration deployment was failing during CloudFormation stack creation with:

No replication engine found with version: 3.5.2

Root Cause: DMS engine version 3.5.2 is no longer available in AWS regions, causing the CloudFormation template to fail when creating the replication instance.

Solution: Updated the embedded CloudFormation template to use DMS engine version 3.5.3, which is currently supported and available.

🎯 Impact

Before v2.0.2: Migration deployment would fail at the replication instance creation step, even with all correct permissions and IAM roles configured.

After v2.0.2: Migration deployment proceeds successfully through all CloudFormation resources.

📦 Installation

# Update to latest version
npm install -g @fiftyten/db-toolkit@2.0.2

# Or with pnpm
pnpm add -g @fiftyten/db-toolkit@2.0.2

# Verify version
fiftyten-db --version  # Should show 2.0.2

🔄 No Breaking Changes

• All existing commands work identically
• No configuration changes required
• Same IAM permissions needed
• Full backward compatibility maintained

🧪 Validation

This fix ensures that the migration deployment process completes the infrastructure setup phase successfully, allowing users to proceed with:

1. ✅ Migration infrastructure deployment (fiftyten-db migrate deploy dev)
2. ✅ Migration task startup (fiftyten-db migrate start dev)
3. ✅ Progress monitoring (fiftyten-db migrate status dev)

🎯 Who Should Update

Critical for: Anyone attempting to deploy migration infrastructure who encountered the "No replication engine found" error.

All users: This is a stability fix that ensures the migration feature works reliably across AWS regions.

---

Full Changelog: v2.0.1...v2.0.2

@fiftyten/db-toolkit v2.0.1 - CloudFormation Stack Recovery Fix

🔧 Bug Fix Release

🚨 Migration Deployment Issue Resolved

Fixed a critical issue where migration deployment would fail when previous deployment attempts left CloudFormation stacks in problematic states.

🛠️ What's Fixed

Problem: When migration deployment failed (e.g., due to missing IAM permissions), the CloudFormation stack would be left in ROLLBACK_COMPLETE state. Subsequent deployment attempts would fail with:

Stack is in ROLLBACK_COMPLETE state and can not be updated

Solution: The CLI now automatically detects and handles stacks in problematic states:

• ROLLBACK_COMPLETE - Stack creation failed and rolled back
• ROLLBACK_FAILED - Stack rollback itself failed
• CREATE_FAILED - Stack creation failed without rollback
• DELETE_FAILED - Stack deletion failed

✨ Enhanced Recovery Behavior

When deploying migration infrastructure (fiftyten-db migrate deploy dev), the CLI will now:

1. Detect problematic stacks: Check stack status before attempting deployment
2. Auto-cleanup: Automatically delete stacks in bad states
3. Fresh deployment: Create a clean stack after cleanup
4. User feedback: Clear messaging about recovery actions

🎯 User Experience Improvements

Before v2.0.1:

fiftyten-db migrate deploy dev
# ❌ Error: Stack is in ROLLBACK_COMPLETE state and can not be updated
# Manual cleanup required: fiftyten-db migrate cleanup dev

After v2.0.1:

fiftyten-db migrate deploy dev
# ⚠️  Found stack in ROLLBACK_COMPLETE state - deleting it first...
# 🔄 Now creating fresh stack...
# ✅ Migration infrastructure deployed successfully!

🔄 No Breaking Changes

• All existing commands work identically
• No configuration changes required
• Same IAM permissions needed
• Full backward compatibility maintained

📦 Installation

# Update to latest version
npm install -g @fiftyten/db-toolkit@2.0.1

# Or with pnpm
pnpm add -g @fiftyten/db-toolkit@2.0.1

# Verify version
fiftyten-db --version  # Should show 2.0.1

🧪 Validation

This fix has been tested with the common failure scenario:

1. Migration deployment fails due to missing dms:AddTagsToResource permission
2. Stack left in ROLLBACK_COMPLETE state
3. Adding required permissions and re-running deployment now succeeds automatically

🎯 Who Should Update

Priority users: Anyone who has experienced failed migration deployments or stacks in bad states.

All users: This is a stability improvement that makes the CLI more resilient to deployment failures.

---

Full Changelog: v2.0.0...v2.0.1

@fiftyten/db-toolkit v2.0.0 - Standalone CLI with Embedded Infrastructure

🚀 Major Release: Standalone CLI

⚡ Revolutionary Standalone Architecture

• Complete independence: No longer requires local storage infrastructure repository
• Embedded CloudFormation: Infrastructure templates built directly into CLI
• Direct AWS API calls: Eliminates CDK spawn processes and PATH issues
• Portable deployment: Works on any developer machine with just AWS credentials
• Zero setup required: Install globally and start migrating immediately

🛠️ Technical Transformation

Infrastructure Deployment

• Before: Required local 5010-indicator-storage-infra repository + CDK + npm/pnpm spawn
• After: Embedded CloudFormation templates deployed via direct AWS APIs
• Benefit: No more spawn ENOENT errors or dependency hell

Auto-Discovery Capabilities

• VPC Detection: Automatically discovers VPC and subnet configuration
• Fallback Strategy: Uses default VPC if storage infrastructure not found
• Target Database Discovery: Still discovers target databases from existing stacks

Enhanced CloudFormation Manager

• Real-time Progress: Live CloudFormation event streaming during deployment
• Error Handling: Comprehensive error reporting with actionable messages
• Stack Management: Complete lifecycle management (create/update/delete)

📦 Breaking Changes

🔧 New IAM Requirements

IMPORTANT: Developers now need additional IAM permissions for CloudFormation and DMS operations.

Required Policy: DMSMigrationDeploymentAccess (see documentation)

Key permissions needed:

• cloudformation:* for migration stack operations
• dms:* for replication instance/task management
• ec2:Describe* for VPC auto-discovery
• iam:*Role for DMS service role creation
• logs:*, sns:*, cloudwatch:* for monitoring

📋 Migration Steps from v1.10.1

# 1. Update CLI package
npm install -g @fiftyten/db-toolkit@2.0.0

# 2. Apply new IAM permissions (see policy document)
# Attach DMSMigrationDeploymentAccess policy to your user/group

# 3. Test migration deployment
fiftyten-db migrate targets dev
fiftyten-db migrate deploy dev

✨ All Migration Features Preserved

Same Business Logic

• Migration workflow: Identical full-load + CDC process
• Commands unchanged: All migrate commands work exactly the same
• AWS DMS resources: Same infrastructure deployed (instances, endpoints, tasks)
• Monitoring: Same progress tracking, validation, and error reporting

Same User Experience

• CLI interface: All prompts, outputs, and interactions unchanged
• MFA handling: Same automatic MFA device discovery and authentication
• Database discovery: Same target database auto-discovery from CloudFormation stacks

🔧 Migration Commands (Unchanged)

# Deploy migration infrastructure (now 100% standalone\!)
fiftyten-db migrate deploy dev

# Start full migration (full-load + CDC)
fiftyten-db migrate start dev

# Monitor real-time progress
fiftyten-db migrate status dev

# Comprehensive data validation
fiftyten-db migrate validate dev  

# List available target databases
fiftyten-db migrate targets dev

# Complete cleanup
fiftyten-db migrate cleanup dev

🛡️ Security & Reliability

Enhanced Security

• Principle of least privilege: Scoped CloudFormation and DMS permissions
• Resource isolation: All resources tagged and properly scoped
• MFA support: Seamless MFA authentication flow preserved

Improved Reliability

• No spawn processes: Eliminates process spawning failures
• Direct API calls: More reliable than subprocess management
• Better error messages: Clearer feedback on permission or configuration issues

📈 Performance Improvements

• Faster deployments: Direct API calls reduce overhead
• Real-time feedback: Live CloudFormation event streaming
• Reduced dependencies: Smaller installation footprint
• Better error recovery: More granular error handling and retry logic

📦 Installation & Usage

Fresh Installation

# With npm
npm install -g @fiftyten/db-toolkit@2.0.0

# With pnpm  
pnpm add -g @fiftyten/db-toolkit@2.0.0

Verification

# Check version
fiftyten-db --version  # Should show 2.0.0

# Test standalone deployment (requires new IAM permissions)
fiftyten-db migrate targets dev

⚠️ Important Migration Notes

For Existing Users

1. IAM permissions required: Apply new policy before using migration features
2. Database connections unchanged: All psql, tunnel, connect commands work identically
3. DynamoDB operations unchanged: All DynamoDB features preserved
4. Same configuration: All existing settings and credentials work

For New Users

1. Install CLI: npm install -g @fiftyten/db-toolkit@2.0.0
2. Apply IAM policy: Request DMSMigrationDeploymentAccess permissions
3. Start migrating: No additional setup required!

🐛 Bug Fixes

• Fixed spawn ENOENT errors: Eliminated all subprocess spawning
• Resolved PATH issues: No dependency on local tool installations
• Improved error messages: More actionable error reporting
• TypeScript compliance: Fixed all compiler warnings and errors

🔮 What's Next

This standalone architecture opens up possibilities for:

• CI/CD integration: Run migrations in automated pipelines
• Docker deployments: CLI works in containers without host dependencies
• Cloud environments: Deploy from any cloud instance
• Team scalability: Easier onboarding with fewer setup requirements

---

Full Changelog: v1.10.1...v2.0.0

Documentation: See DMS-Migration-IAM-Policy.md for complete IAM policy requirements

Support: Report issues at https://github.com/5010-dev/fiftyten-cli-tools/issues

@fiftyten/db-toolkit v1.10.1 - ESM Compatibility Fix

🐛 Bug Fixes

ESM/CommonJS Compatibility Issue

• Fixed critical import error: Resolved error when using the CLI
• Replaced inquirer with readline: Migrated from inquirer v9 (ESM-only) to native readline module
• Maintained all functionality: All prompts and user interactions work exactly the same
• Improved reliability: No more module loading conflicts in Node.js environments

🔧 Technical Changes

• MFA Authentication: Converted inquirer prompts to readline for MFA device selection and token input
• Migration Deployment: Simplified configuration prompts using readline
• User Experience: Maintained identical CLI behavior with improved stability

📦 Installation

# Update to latest version
npm install -g @fiftyten/db-toolkit@1.10.1

# Or with pnpm
pnpm add -g @fiftyten/db-toolkit@1.10.1

🎯 Migration from v1.10.0

This is a drop-in replacement - no configuration changes needed:

# Update your installation
pnpm remove -g @fiftyten/db-toolkit
pnpm add -g @fiftyten/db-toolkit@1.10.1

# All commands work exactly the same
fiftyten-db migrate deploy dev
fiftyten-db psql dev -d indicator

⚠️ Important Notes

• Affects all users: Anyone experiencing ERR_REQUIRE_ESM errors should update immediately
• Node.js compatibility: Now works reliably across all supported Node.js versions
• Migration features: All database migration functionality is now fully operational

---

Full Changelog: v1.10.0...v1.10.1