This is a powerful framework for generating variations of images with recurring patterns and structures while preserving their essential characteristics. It's especially effective for pattern-rich images like carpet designs, wallpapers, and tileable textures.

🌟 Features

• Structural Consistency: Generates variations that maintain the essential character of the original pattern
• Controlled Randomness: Intelligently modifies patterns with meaningful variations rather than arbitrary changes
• Latent Space Manipulation: Uses advanced adaptive instance normalization techniques for coherent transformations
• Carpet Specialization: Includes optimizations specifically for textile and carpet pattern generation
• Simple API: Easy-to-use interface for both standard images and pattern-rich designs

💡 How It Works

This uses a modified Stable Diffusion pipeline with specialized attention mechanisms and adaptive instance normalization to create meaningful variations of patterns. The key components include:

1. Custom Attention Mechanism: Modified self-attention that respects structural patterns
2. Enhanced AdaIN: Advanced adaptive instance normalization that preserves motif structures
3. Latent Space Manipulation: Grid-level transformations in latent space for coherent pattern modifications
4. DDIM Inversion: Precise latent representation capturing the essence of the original design

🔧 Advanced Configuration

The behavior of the framework can be extensively customized:

# Detailed configuration
cfgs = {
    "self_attn": {
        "atten_frames": 3,  # Controls attention spread
        "t_align": 700      # Alignment cutoff for detail preservation
    },
    "inference": {
        "invert_step": 100,  # Higher for better inversion quality
        "ddim_step": 100,    # Controls refinement level
        "cfg": 6.5,          # Guidance scale
        "is_null_prompt": True,
        "t_early": 750       # When to start adaptive normalization
    }
}

model.set_config(cfgs)

🧩 Core Components

Modified Attention Mechanism

def new_forward(self, hidden_states, encoder_hidden_states=None, attention_mask=None, temb=None, **cross_attention_kwargs):
    # Custom attention implementation preserving pattern structures
    # while allowing controlled variation
    # ...

Enhanced Adaptive Instance Normalization

def adain_latent(feat, cond_feat, eps=1e-5, detail_preservation=0.65):
    """
    Enhanced Adaptive Instance Normalization to encourage more structural variations
    while maintaining consistency with the original pattern.
    """
    # Calculate spatial-aware statistics
    # This helps capture and transfer motif structures better
    # ...

🎨 Examples Gallery

Original Image and Generated Variations

Original Carpet and Generated Variations

📝 Reference

Real-World Image Variation by Aligning Diffusion Inversion Chain https://arxiv.org/pdf/2305.18729