TaleCraft AI

🎬 TaleCraft AI

AI-Powered Interactive Storytelling Platform - Transform character images and story prompts into rich multimedia experiences with consistent character rendering, voice narration, and professional video generation.

✨ Key Features

🎭 Character DNA System - Upload character images with AI-powered DNA extraction for visual consistency across scenes
🤖 Advanced AI Generation - Powered by Google Gemini AI for high-quality, contextual storytelling
🐍 Free Python SD Service - Local Stable Diffusion service with GPU acceleration (0$ cost, 10x faster, 100% private)
🎨 Smart Storyboards - Automatic storyboard creation using Stable Diffusion models with character consistency
📝 4-Scene Story Architecture - AI-generated multi-scene narratives optimized for visual storytelling
🎙️ Voice Narration - Professional voice synthesis with customizable emotions and languages
🎥 Video Compilation - Automatic video creation combining story, images, motion effects, and narration
📚 Multi-Format Export - Download stories as PDF, video (MP4), or shareable web links
🔄 Real-Time Progress - WebSocket-powered live updates during story generation
📱 Responsive Design - Complete mobile optimization with touch-friendly interface
🌙 Dark Mode - Full dark mode support with system preference detection 💾 Smart Save System - Private/public story saving with instant URL sharing

🏆 Hackathon Features

Character DNA Technology

AI-powered character analysis using CLIP embeddings
Visual consistency across all generated scenes
Support for any character style (cartoon, realistic, fantasy, etc.)

Multi-AI Orchestration

Story Generation: Google Gemini AI for structured narratives
Image Generation: Stable Diffusion for character-consistent visuals
Voice Synthesis: ElevenLabs for professional narration
Video Compilation: FFmpeg for motion effects and scene transitions

Free Python Stable Diffusion Service

Zero Cost: No API fees, unlimited image generation
GPU Acceleration: 10x faster generation with NVIDIA GPUs (5-15s vs 30-60s)
100% Privacy: Images never leave your computer
Smart Fallback: Automatic cloud API fallback when service unavailable
Easy Setup: 5-minute installation with automated scripts

Real-Time Generation Pipeline

WebSocket-powered progress tracking
Live status updates with time estimates
Background processing with job management
Failure recovery and graceful degradation

Professional Export Options

PDF Storybooks: Formatted documents with scenes and images
MP4 Videos: 15-30 second animated stories with voiceover
Web Sharing: Instant shareable links with social media integration
Audio Files: Voice-only narration downloads

User Experience Innovation

One-Click Demo: 4 pre-loaded character archetypes for instant testing
Progressive Upload: Drag-and-drop with real-time preview
Smart Completion: Guided workflow with validation and progress indicators
Mobile Optimized: Touch-friendly interface with responsive design

🚀 Quick Start

Prerequisites

Node.js 18+
Google Gemini API Key
Hugging Face API Token
Supabase Account
ElevenLabs API Key (optional)

Installation

# 1. Clone and install dependencies
git clone <repository-url>
cd talecraft-ai
npm run install:all

# 2. Configure environment (copy and edit)
cp server/.env.example server/.env
# Edit server/.env with your API keys

# 3. Start development servers
npm run dev

🎯 Ready for MVP Demo! Visit http://localhost:5173 and start creating stories.

📖 Need detailed setup? See QUICK_START.md for step-by-step instructions.

🏗️ Architecture

graph TB
    subgraph "Frontend (React + Vite)"
        A[Character Upload] --> B[Story Creation]
        B --> C[AI Generation]
        C --> D[Story Preview]
        D --> E[Export & Share]
    end
    
    subgraph "Backend (Node.js + Express)"
        F[API Routes] --> G[AI Services]
        G --> H[AWS S3 Storage]
        F --> I[WebSocket Updates]
    end
    
    subgraph "AI Services"
        J[Hugging Face] --> K[Story Generation]
        J --> L[Image Generation]
        M[ElevenLabs] --> N[Voice Synthesis]
    end
    
    A --> F
    G --> J
    G --> M

📊 Technology Stack

Frontend

React 18 - Modern UI framework with hooks and concurrent features
Vite - Lightning-fast development and build tool
Tailwind CSS - Utility-first CSS framework for rapid styling
Framer Motion - Smooth animations and transitions
Zustand - Lightweight state management
React Router - Client-side routing

Backend

Node.js - JavaScript runtime for server-side development
Express.js - Minimal web framework for APIs
Multer - File upload handling middleware
WebSocket - Real-time communication for progress updates
Sharp - High-performance image processing

AI & Services

Google Gemini Pro - Advanced story generation and character analysis
Python Stable Diffusion Service - Local, free GPU-accelerated image generation
Hugging Face API - Cloud fallback for story and image generation models
ElevenLabs - Premium voice synthesis
AWS S3 - Scalable cloud storage
FFmpeg - Video processing and compilation

🎯 Hackathon Demo Flow

4-Minute Complete Demonstration

Character Upload (30 seconds)
- Select from 4 demo characters OR upload custom image
- AI processes character DNA with visual feedback
- Character card displays with extracted traits
Story Creation (60 seconds)
- Choose genre: Adventure, Fantasy, Sci-Fi, Mystery
- Set tone: Lighthearted, Serious, Humorous, Dramatic
- Enter story prompt: "What if [character] discovered a magical artifact?"
- Configure voice and video options
AI Generation (2-3 minutes)
- Real-time progress bar with detailed steps:
  - Character DNA Analysis ✓
  - Story Generation (Gemini AI) ✓
  - Scene Image Creation (Stable Diffusion) ✓
  - Voice Narration (ElevenLabs) ✓
  - Video Compilation (FFmpeg) ✓
- Live status updates: "Generating scene 2 of 4..."
Results & Export (30 seconds)
- Tabbed Preview: Full story, scene breakdown, character details
- Instant Download: PDF storybook, MP4 video
- One-Click Sharing: Copy shareable URL or social media
- Save Options: Private or public story library

Key Demo Talking Points

"Upload any character and maintain visual consistency"
"4-scene structured storytelling optimized for engagement"
"Real-time AI coordination across multiple models"
"Professional export quality in under 5 minutes"
"Complete mobile experience with dark mode"

📁 Project Structure

talecraft-ai/
├── client/                 # React frontend
│   ├── src/
│   │   ├── components/     # Reusable UI components
│   │   ├── pages/          # Page components
│   │   ├── stores/         # State management
│   │   └── utils/          # Helper functions
├── server/                 # Node.js backend
│   ├── routes/             # API endpoints
│   ├── services/           # Business logic
│   ├── middleware/         # Express middleware
│   └── utils/              # Server utilities
├── DOCUMENTATION.md        # Comprehensive technical docs
├── QUICK_START.md         # 5-minute setup guide
└── README.md              # This file

🔧 Development

Available Scripts

npm run dev          # Start both frontend and backend
npm run client:dev   # Start only frontend (port 5173)
npm run server:dev   # Start only backend (port 3001)
npm run build        # Build for production
npm start           # Start production server
npm run install:all # Install all dependencies

Environment Configuration

Key environment variables (see server/.env.example):

HUGGING_FACE_TOKEN=your_key_here
AWS_ACCESS_KEY_ID=your_key_here
AWS_SECRET_ACCESS_KEY=your_secret_here
AWS_S3_BUCKET=your_bucket_name
ELEVENLABS_API_KEY=your_key_here

📚 Documentation

📖 DOCUMENTATION.md - Complete technical documentation with architecture diagrams
🚀 QUICK_START.md - 5-minute setup guide for immediate demo
🔧 API Documentation - Detailed API reference
🎨 UI Guide - Component documentation and design system

🌟 Key Differentiators

For Hackathon Judges

Technical Innovation: Multi-AI model orchestration with real-time coordination
Character Consistency: Advanced CLIP embedding system for visual coherence
User Experience: Complete 4-minute workflow from upload to shareable content
Professional Quality: Broadcast-ready video output with motion effects
Scalable Architecture: Production-ready WebSocket infrastructure

Competitive Advantages

Character DNA System: Unique visual consistency technology
4-Scene Story Structure: Optimized narrative format for engagement
Multi-Modal Export: PDF, MP4, and web sharing in one platform
Real-Time Processing: Live progress tracking with time estimates
Mobile-First Design: Complete responsive experience with dark mode
One-Click Demo: Instant testing with pre-loaded characters

Target Market Impact

Content Creators: Social media storytellers, YouTubers, TikTokers
Educators: Interactive lesson creation, student engagement tools
Businesses: Marketing storytelling, brand narratives, training materials
Entertainment: Indie game developers, comic creators, animation studios

🔐 Security & Performance

✅ Input validation and sanitization
✅ File type and size restrictions
✅ Rate limiting on API endpoints
✅ Secure AWS S3 integration
✅ CORS configuration
✅ Environment-based configuration
✅ Optimized image processing
✅ Efficient state management

🚀 Deployment Ready

The application is production-ready with:

Docker containerization support
Environment-based configuration
Health check endpoints
Error monitoring integration
Scalable architecture
CDN-ready static assets

🤝 Contributing

Fork the repository
Create a feature branch (git checkout -b feature/amazing-feature)
Commit your changes (git commit -m 'Add amazing feature')
Push to the branch (git push origin feature/amazing-feature)
Open a Pull Request

📄 License

This project is licensed under the MIT License - see the LICENSE file for details.

🚀 Deployment & Production Ready

Cloud Infrastructure

Cloudflare R2: Scalable object storage with global CDN
WebSocket Scaling: Real-time communication with load balancing
Docker Support: Containerized deployment for easy scaling
Environment Config: Production-ready configuration management

Performance Optimizations

Image Processing: Sharp-powered optimization with automatic resizing
Caching Strategy: Smart caching for characters and generated content
Background Jobs: Queue-based processing for heavy AI operations
Error Handling: Graceful degradation with fallback mechanisms

Security Features

JWT Authentication: Secure user sessions with token-based auth
File Validation: Comprehensive image upload security
Rate Limiting: API protection against abuse
Data Privacy: User data encryption and secure storage

🎯 Hackathon Submission Status: ✅ COMPLETE

Your TaleCraft AI application is fully functional and ready for demonstration!

Submission Checklist:

✅ Demo Ready: 4-minute complete workflow demonstration
✅ Technical Innovation: Multi-AI orchestration with character consistency
✅ User Experience: Mobile-responsive with dark mode support
✅ Professional Output: PDF, MP4, and web sharing capabilities
✅ Documentation: Comprehensive setup and API documentation
✅ Production Code: Clean, scalable, and well-structured codebase

Next Steps:

Set up environment variables (see QUICK_START.md)
Run npm run dev to start development servers
Test complete workflow with demo characters
Prepare hackathon presentation and live demo

Built with ❤️ for the future of AI-powered interactive storytelling

🚀 Features

Character DNA System: Upload images with auto-processing and CLIP embeddings
AI Story Engine: 4-scene story generation with branching narratives
Visual Generation: Consistent character rendering with style transfer
Video Synthesis: 15-30s animated videos with motion effects
Multi-Format Export: MP4, PDF, and shareable web links
Voice Narration: Emotion-based voice synthesis in multiple languages

🏗️ Architecture

talecraft-ai/
├── client/          # React frontend (Vite)
├── server/          # Node.js/Express backend
├── shared/          # Shared types and utilities
└── docs/           # Documentation

📋 Requirements

Node.js 18+
NPM or Yarn
Environment variables (see .env.example)

🛠️ Development Setup

Clone and install dependencies:

git clone <repository-url>
cd talecraft-ai
npm run install:all

Set up environment variables:

cp server/.env.example server/.env
# Edit server/.env with your API keys

Start development servers:
```
npm run dev
```
This starts:
- Frontend: http://localhost:5173
- Backend: http://localhost:3001

🔧 Environment Variables

Create server/.env with:

HUGGING_FACE_TOKEN=your_hf_token
ELEVENLABS_API_KEY=your_elevenlabs_key
CLOUDFLARE_R2_ACCESS_KEY=your_r2_access_key
CLOUDFLARE_R2_SECRET_KEY=your_r2_secret_key
CLOUDFLARE_R2_BUCKET=your_bucket_name
CLOUDFLARE_R2_ENDPOINT=your_r2_endpoint

🎯 API Endpoints

Endpoint	Method	Description
`/api/upload`	POST	Upload character image
`/api/generate`	POST	Generate story and scenes
`/api/status/:id`	GET	Check generation status
`/api/export/pdf`	POST	Export as PDF
`/api/voices`	GET	Get available voices

🎬 Demo Timeline

Day 1: Character upload + DNA system
Day 2: Story engine with Mistral 7B
Day 3: Video pipeline with Wan-AI
Day 4: Voice narration + polish
Day 5: Demo preparation

Name		Name	Last commit message	Last commit date
Latest commit History 25 Commits
.vscode		.vscode
client		client
database		database
python-sd-service		python-sd-service
server		server
.env.example		.env.example
.gitignore		.gitignore
CHARACTER_CONSISTENCY.md		CHARACTER_CONSISTENCY.md
CLIP_TOKEN_OPTIMIZATION.md		CLIP_TOKEN_OPTIMIZATION.md
DOCUMENTATION.md		DOCUMENTATION.md
LICENSE		LICENSE
QUICK_START.md		QUICK_START.md
README.md		README.md
debug_parsing.js		debug_parsing.js
package-lock.json		package-lock.json
package.json		package.json
test_clip_optimization.js		test_clip_optimization.js

License

notedwin-dev/talecraft-ai

Folders and files

Latest commit

History

Repository files navigation

TaleCraft AI

🎬 TaleCraft AI

✨ Key Features

🏆 Hackathon Features

Character DNA Technology

Multi-AI Orchestration

Free Python Stable Diffusion Service

Real-Time Generation Pipeline

Professional Export Options

User Experience Innovation

🚀 Quick Start

Prerequisites

Installation

🏗️ Architecture

📊 Technology Stack

Frontend

Backend

AI & Services

🎯 Hackathon Demo Flow

4-Minute Complete Demonstration

Key Demo Talking Points

📁 Project Structure

🔧 Development

Available Scripts

Environment Configuration

📚 Documentation

🌟 Key Differentiators

For Hackathon Judges

Competitive Advantages

Target Market Impact

🔐 Security & Performance

🚀 Deployment Ready

🤝 Contributing

📄 License

🚀 Deployment & Production Ready

Cloud Infrastructure

Performance Optimizations

Security Features

🎯 Hackathon Submission Status: ✅ COMPLETE

Submission Checklist:

Next Steps:

🚀 Features

🏗️ Architecture

📋 Requirements

🛠️ Development Setup

🔧 Environment Variables

🎯 API Endpoints

🎬 Demo Timeline

About

Resources

License

Uh oh!

Stars

Watchers

Forks

Releases

Packages 0

Contributors 2

Uh oh!

Languages

Packages