Skip to content
This repository was archived by the owner on Jul 11, 2025. It is now read-only.

Latest commit

 

History

History
1360 lines (1112 loc) · 39.7 KB

String-Guide.md

File metadata and controls

1360 lines (1112 loc) · 39.7 KB

String: A Real-World Guide

Table of Contents

  1. Introduction & Core Concepts
  2. Basic Usage Patterns
  3. Real-World Examples
  4. Advanced Features Deep Dive
  5. Practical Patterns & Best Practices
  6. Integration Examples

Introduction & Core Concepts

The Problem String Solves

Traditional JavaScript string manipulation often leads to verbose, error-prone code with scattered utility functions across different libraries and inconsistent error handling.

// Traditional approach - scattered functions and inconsistent patterns
const processUserInput = (input: string) => {
  // Multiple libraries, inconsistent APIs
  const trimmed = input.trim()
  const normalized = trimmed.toLowerCase()
  const words = trimmed.split(' ').filter(word => word.length > 0)
  
  // Manual validation with unclear error handling
  if (words.length === 0) {
    throw new Error('Empty input')
  }
  
  // Case conversion scattered across different utilities
  const camelCase = words.map((word, index) => 
    index === 0 ? word : word.charAt(0).toUpperCase() + word.slice(1)
  ).join('')
  
  return camelCase
}

// Inconsistent null/undefined handling
const formatName = (first?: string, last?: string) => {
  if (!first && !last) return 'Anonymous'
  if (!first) return last!.toUpperCase()
  if (!last) return first.toUpperCase()
  return `${first.charAt(0).toUpperCase()}${first.slice(1)} ${last.toUpperCase()}`
}

This approach leads to:

  • Inconsistent APIs - Different string libraries have different function signatures
  • Poor Composability - Hard to chain operations cleanly
  • Manual Error Handling - Each validation requires custom error logic
  • Type Unsafety - Many operations can return undefined or throw exceptions

The String Solution

Effect's String module provides a unified, composable API for string operations with consistent error handling and excellent type safety.

import { String, Array as Arr, Effect } from "effect"

// Clean, composable string processing
const processUserInput = (input: string) => Effect.gen(function* () {
  const trimmed = String.trim(input)
  const normalized = String.toLowerCase(trimmed)
  const words = Arr.filter(String.split(' ')(normalized), String.isNonEmpty)
  
  if (Arr.isEmptyArray(words)) {
    return yield* Effect.fail(new Error('Empty input'))
  }
  
  const camelCase = Arr.mapWithIndex(words, (word, index) => 
    index === 0 ? word : String.capitalize(word)
  ).pipe(Arr.join(''))
  
  return camelCase
})

// Type-safe name formatting with consistent patterns
const formatName = (first: string, last: string) => Effect.gen(function* () {
  const names = Arr.filter([first, last], String.isNonEmpty)
  
  if (Arr.isEmptyArray(names)) {
    return 'Anonymous'
  }
  
  return Arr.map(names, String.capitalize).pipe(Arr.join(' '))
})

Key Concepts

Functional Composition: All String functions are designed to work seamlessly with pipe, enabling clean data transformation pipelines.

Type Safety: Functions preserve and enhance TypeScript types, with refinement types like NonEmptyString where appropriate.

Consistent API: All functions follow the same curried signature pattern, making them highly composable and predictable.

Basic Usage Patterns

Pattern 1: Basic String Operations

import { String } from "effect"

// String creation and basic operations
const text = "  Hello World  "

const processed = String.trim(text).pipe(
  String.toLowerCase,
  (s) => String.replace(s, ' ', '_'),
  String.capitalize
) // "Hello_world"

// String inspection
const analysis = {
  isEmpty: String.isEmpty(text),           // false
  isNonEmpty: String.isNonEmpty(text),     // true
  length: String.length(text),             // 15
  includes: String.includes(text, "Hello"), // true
  startsWith: String.startsWith(text, "  "), // true
  endsWith: String.endsWith(text, "  ")    // true
}

Pattern 2: String Transformation Pipelines

import { String, Array as Arr } from "effect"

// Complex transformation pipeline
const transformText = (input: string) => Effect.gen(function* () {
  const trimmed = String.trim(input)
  const normalized = String.normalize(trimmed)
  const words = Arr.filter(String.split(normalized, /\s+/), String.isNonEmpty)
  const lowercased = Arr.map(words, String.toLowerCase)
  const joined = Arr.join(lowercased, '-')
  const sliced = String.slice(joined, 0, 50)
  
  return String.trimEnd(sliced)
})

// Case conversion utilities
const toCamelCase = (input: string) => 
  String.split(input, '_').pipe(
    Arr.mapWithIndex((word, index) => 
      index === 0 ? word : String.capitalize(word)
    ),
    Arr.join('')
  )

const toKebabCase = (input: string) => 
  String.replace(input, /([A-Z])/g, '-$1').pipe(
    String.toLowerCase,
    (s) => String.replace(s, /^-/, '')
  )

Pattern 3: String Parsing and Pattern Matching

import { String, Option, Array as Arr, Effect } from "effect"

// Pattern matching with regular expressions
const extractEmail = (text: string) => Effect.gen(function* () {
  const emailRegex = /([a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+\.[a-zA-Z]{2,})/
  const match = String.match(text, emailRegex)
  
  if (Option.isNone(match)) {
    return yield* Effect.fail('No email found')
  }
  
  return match.value[1]
})

// Multi-line string processing
const processMultilineText = (text: string) => Effect.gen(function* () {
  const lines = String.split(text, '\n')
  const trimmed = Arr.map(lines, String.trim)
  const filtered = Arr.filter(trimmed, String.isNonEmpty)
  
  return Arr.join(filtered, '\n')
})

// Advanced pattern matching
const parseLogEntry = (line: string) => Effect.gen(function* () {
  const logRegex = /^\[(\d{4}-\d{2}-\d{2})\] (\w+): (.+)$/
  const match = String.match(line, logRegex)
  
  if (Option.isNone(match)) {
    return yield* Effect.fail('Invalid log format')
  }
  
  const [, date, level, message] = match.value
  
  return {
    date,
    level,
    message
  }
})

Real-World Examples

Example 1: User Input Sanitization and Validation

Processing user form input with comprehensive validation and normalization.

import { String, Array as Arr, Effect, Option } from "effect"

// Define custom error types
class ValidationError extends Error {
  readonly _tag = 'ValidationError'
  constructor(readonly field: string, readonly reason: string) {
    super(`Invalid ${field}: ${reason}`)
  }
}

// Username sanitization pipeline
const sanitizeUsername = (input: string) => Effect.gen(function* () {
  const trimmed = String.trim(input)
  
  if (String.isEmpty(trimmed)) {
    return yield* Effect.fail(new ValidationError('username', 'cannot be empty'))
  }
  
  const normalized = String.toLowerCase(trimmed).pipe(
    (s) => String.replace(s, /[^a-z0-9_-]/g, ''),
    (s) => String.slice(s, 0, 20)
  )
  
  if (String.length(normalized) < 3) {
    return yield* Effect.fail(new ValidationError('username', 'must be at least 3 characters'))
  }
  
  return normalized
})

// Email validation with domain checking
const validateEmail = (input: string) => Effect.gen(function* () {
  const trimmed = String.trim(input)
  const emailRegex = /^[^\s@]+@[^\s@]+\.[^\s@]+$/
  
  if (!String.match(trimmed, emailRegex)) {
    return yield* Effect.fail(new ValidationError('email', 'invalid format'))
  }
  
  const domain = Arr.get(String.split(trimmed, '@'), 1).pipe(
    Option.getOrElse(() => '')
  )
  
  const allowedDomains = ['gmail.com', 'yahoo.com', 'outlook.com', 'company.com']
  if (!Arr.contains(allowedDomains, domain)) {
    return yield* Effect.fail(new ValidationError('email', 'domain not allowed'))
  }
  
  return String.toLowerCase(trimmed)
})

// Phone number sanitization
const sanitizePhone = (input: string) => Effect.gen(function* () {
  const digitsOnly = String.replace(input, /[^\d]/g, '')
  
  if (String.length(digitsOnly) !== 10) {
    return yield* Effect.fail(new ValidationError('phone', 'must be 10 digits'))
  }
  
  return digitsOnly
})

// Complete user registration form processing
const processUserRegistration = (formData: {
  username: string
  email: string
  fullName: string
  phone: string
}) => Effect.gen(function* () {
  const username = yield* sanitizeUsername(formData.username)
  const email = yield* validateEmail(formData.email)
  const phone = yield* sanitizePhone(formData.phone)
  
  const fullName = String.trim(formData.fullName).pipe(
    (s) => String.replace(s, /\s+/g, ' '),
    (s) => String.split(s, ' '),
    (words) => Arr.filter(words, String.isNonEmpty),
    (words) => Arr.map(words, String.capitalize),
    (words) => Arr.join(words, ' ')
  )
  
  return {
    username,
    email,
    fullName,
    phone
  }
}).pipe(
  Effect.catchTag('ValidationError', (error) =>
    Effect.fail(`Registration failed: ${error.message}`)
  ),
  Effect.withSpan('user.registration')
)

Example 2: Content Management System

Building a content management system with text processing capabilities.

import { String, Array as Arr, Effect, HashMap, Option } from "effect"

// Text analysis utilities
const analyzeText = (content: string) => Effect.gen(function* () {
  const cleaned = String.toLowerCase(content).pipe(
    (s) => String.replace(s, /[^\w\s]/g, ''),
    (s) => String.split(s, /\s+/),
    (words) => Arr.filter(words, String.isNonEmpty)
  )
  
  const wordCount = Arr.length(cleaned)
  const uniqueWords = Arr.dedupeWith(cleaned, String.Equivalence)
  
  const totalLength = Arr.reduce(
    Arr.map(cleaned, String.length),
    0,
    (acc, len) => acc + len
  )
  
  const avgWordLength = wordCount > 0 ? Math.round(totalLength / wordCount) : 0
  
  return {
    wordCount,
    uniqueWordCount: Arr.length(uniqueWords),
    avgWordLength,
    readingTime: Math.ceil(wordCount / 200) // Assume 200 WPM
  }
})

// SEO-friendly slug generation
const generateSlug = (title: string) => Effect.gen(function* () {
  const slug = String.trim(title).pipe(
    String.toLowerCase,
    String.normalize,
    (s) => String.replace(s, /[^\w\s-]/g, ''),
    (s) => String.replace(s, /\s+/g, '-'),
    (s) => String.replace(s, /-+/g, '-'),
    String.trimStart,
    String.trimEnd,
    (s) => String.slice(s, 0, 60)
  )
  
  if (String.isEmpty(slug)) {
    return yield* Effect.fail('Cannot generate slug from title')
  }
  
  return slug
})

// Content excerpt generation
const generateExcerpt = (content: string, maxLength: number = 150) => Effect.gen(function* () {
  const cleaned = String.replace(content, /<[^>]*>/g, '').pipe(
    (s) => String.replace(s, /\s+/g, ' '),
    String.trim
  )
  
  if (String.length(cleaned) <= maxLength) {
    return cleaned
  }
  
  const truncated = String.slice(cleaned, 0, maxLength)
  const lastSpaceIndex = String.lastIndexOf(truncated, ' ')
  
  const excerpt = lastSpaceIndex > 0 
    ? String.slice(truncated, 0, lastSpaceIndex)
    : truncated
  
  return `${excerpt}...`
})

// Keyword extraction for content tagging
const extractKeywords = (content: string, minLength: number = 4) => Effect.gen(function* () {
  const words = String.toLowerCase(content).pipe(
    (s) => String.replace(s, /[^\w\s]/g, ''),
    (s) => String.split(s, /\s+/),
    (words) => Arr.filter(words, word => String.length(word) >= minLength),
    (words) => Arr.filter(words, String.isNonEmpty)
  )
  
  // Count word frequencies
  const frequencies = Arr.reduce(
    words,
    HashMap.empty<string, number>(),
    (acc, word) => HashMap.modify(acc, word, (count) => 
      Option.getOrElse(count, () => 0) + 1
    )
  )
  
  // Get top keywords
  const topKeywords = HashMap.toEntries(frequencies).pipe(
    Arr.sort(([, a], [, b]) => b - a),
    Arr.take(10),
    Arr.map(([word]) => word)
  )
  
  return topKeywords
})

// Complete blog post processing
const processBlogPost = (post: {
  title: string
  content: string
  author: string
}) => Effect.gen(function* () {
  const slug = yield* generateSlug(post.title)
  const excerpt = yield* generateExcerpt(post.content)
  const keywords = yield* extractKeywords(post.content)
  const analysis = yield* analyzeText(post.content)
  
  const authorSlug = String.replace(post.author, /\s+/g, '-').pipe(
    String.toLowerCase
  )
  
  return {
    ...post,
    slug,
    excerpt,
    keywords,
    authorSlug,
    ...analysis,
    publishedAt: new Date().toISOString()
  }
}).pipe(
  Effect.withSpan('blog.process', {
    attributes: { 'blog.title': post.title }
  })
)

Example 3: Configuration Parser and Template Engine

Building a configuration parser and template engine for application settings.

import { String, Array as Arr, Effect, Option, Record } from "effect"

// Configuration value parsing with type coercion
const parseConfigValue = (key: string, value: string) => Effect.gen(function* () {
  const trimmed = String.trim(value)
  
  // Boolean parsing
  const lowerTrimmed = String.toLowerCase(trimmed)
  if (lowerTrimmed === 'true' || lowerTrimmed === 'false') {
    return lowerTrimmed === 'true'
  }
  
  // Number parsing
  const numberMatch = String.match(trimmed, /^-?\d+(\.\d+)?$/)
  if (Option.isSome(numberMatch)) {
    return parseFloat(trimmed)
  }
  
  // Array parsing (comma-separated)
  if (String.includes(trimmed, ',')) {
    return String.split(trimmed, ',').pipe(
      Arr.map(String.trim),
      Arr.filter(String.isNonEmpty)
    )
  }
  
  // URL validation
  if (String.startsWith(trimmed, 'http')) {
    try {
      new URL(trimmed)
      return trimmed
    } catch {
      return yield* Effect.fail(`Invalid URL for ${key}: ${trimmed}`)
    }
  }
  
  // Environment variable expansion
  const envVarRegex = /\$\{([^}]+)\}/g
  const envMatches = String.matchAll(trimmed, envVarRegex)
  let expanded = trimmed
  
  for (const match of envMatches) {
    const varName = String.slice(match[0], 2, -1)
    const envValue = process.env[varName] || ''
    expanded = String.replace(expanded, match[0], envValue)
  }
  
  return expanded
})

// Template engine with variable substitution
const processTemplate = (template: string, variables: Record<string, string>) => Effect.gen(function* () {
  let result = template
  const templateRegex = /\{\{([^}]+)\}\}/g
  const matches = String.matchAll(template, templateRegex)
  
  for (const match of matches) {
    const fullMatch = match[0]
    const variableName = String.trim(match[1])
    
    // Support dot notation for nested access
    const value = String.split(variableName, '.').pipe(
      Arr.reduce(variables as Record<string, any>, (obj, key) => obj?.[key] || ''),
      String
    )
    
    if (String.isEmpty(value)) {
      return yield* Effect.fail(`Template variable not found: ${variableName}`)
    }
    
    result = String.replaceAll(result, fullMatch, value)
  }
  
  return result
})

// Configuration file processing
const parseConfigFile = (content: string) => Effect.gen(function* () {
  const config: Record<string, unknown> = {}
  
  const lines = String.split(content, '\n').pipe(
    Arr.map(String.trim),
    Arr.filter(String.isNonEmpty),
    Arr.filter(line => !String.startsWith(line, '#'))
  )
  
  for (const line of lines) {
    const equalIndex = String.indexOf(line, '=')
    
    if (equalIndex === -1) {
      continue // Skip invalid lines
    }
    
    const key = String.trim(String.slice(line, 0, equalIndex))
    const value = String.slice(line, equalIndex + 1).pipe(
      String.trim,
      String.trimStart,
      String.trimEnd
    )
    
    const parsedValue = yield* parseConfigValue(key, value)
    config[key] = parsedValue
  }
  
  return config
}).pipe(
  Effect.withSpan('config.parse')
)

// Email template processor
const processEmailTemplate = (template: string, data: {
  user: { name: string; email: string }
  order: { id: string; total: number; items: Array<{ name: string; price: number }> }
}) => Effect.gen(function* () {
  // Prepare template variables
  const templateVars = {
    'user.name': data.user.name,
    'user.email': data.user.email,
    'order.id': data.order.id,
    'order.total': data.order.total.toString(),
    'order.itemCount': data.order.items.length.toString(),
    'order.itemList': Arr.map(
      data.order.items, 
      item => `${item.name} - $${item.price}`
    ).pipe(Arr.join('\n'))
  }
  
  const processed = yield* processTemplate(template, templateVars)
  
  // Post-processing for email formatting
  const formatted = String.replace(processed, /\n\s*\n/g, '\n\n').pipe(
    String.trim
  )
  
  return formatted
}).pipe(
  Effect.withSpan('email.template.process')
)

Advanced Features Deep Dive

Feature 1: Unicode and Internationalization Support

Effect's String module provides robust Unicode handling for international applications.

Basic Unicode Operations

import { String } from "effect"

// Unicode normalization for consistent text processing
const normalizeText = (input: string) => 
  String.trim(String.normalize(input))

// Example with different Unicode forms
const text1 = "café" // é as single character
const text2 = "cafe\u0301" // e + combining accent

const normalized1 = normalizeText(text1) // "café"
const normalized2 = normalizeText(text2) // "café" (same result)

Real-World Unicode Example

import { String, Array as Arr, Effect } from "effect"

// International name processing
const processInternationalName = (name: string) => Effect.gen(function* () {
  // Normalize Unicode for consistent storage/comparison
  const normalized = String.normalize(name)
  
  // Handle right-to-left languages
  const direction = /[\u0590-\u05FF\u0600-\u06FF\u0750-\u077F]/.test(normalized) 
    ? 'rtl' : 'ltr'
  
  // Extract character categories for validation
  const hasLatinChars = /[A-Za-z]/.test(normalized)
  const hasArabicChars = /[\u0600-\u06FF]/.test(normalized)
  const hasCyrillicChars = /[\u0400-\u04FF]/.test(normalized)
  
  // Length calculation considering grapheme clusters
  const visualLength = Arr.length(Array.from(normalized))
  
  if (visualLength < 2) {
    return yield* Effect.fail('Name too short')
  }
  
  return {
    original: name,
    normalized,
    direction,
    visualLength,
    scripts: { hasLatinChars, hasArabicChars, hasCyrillicChars }
  }
}).pipe(
  Effect.withSpan('name.process.international')
)

// Multi-language search functionality
const createSearchIndex = (texts: Array<string>) => Effect.gen(function* () {
  return Arr.map(texts, text => ({
    original: text,
    normalized: String.normalize(text).pipe(
      String.toLowerCase,
      (s) => String.replace(s, /[^\p{L}\p{N}\s]/gu, ''),
      String.trim
    ),
    searchTerms: String.normalize(text).pipe(
      String.toLowerCase,
      (s) => String.split(s, /\s+/),
      (terms) => Arr.filter(terms, String.isNonEmpty)
    )
  }))
})

Advanced Unicode: Emoji and Special Characters

import { String, Array as Arr, Effect } from "effect"

// Emoji-aware text processing
const processTextWithEmoji = (text: string) => Effect.gen(function* () {
  // Count actual characters vs code points
  const codePointLength = String.length(text)
  const characterLength = Arr.length(Array.from(text))
  
  // Extract emojis
  const emojiRegex = /[\u{1F600}-\u{1F64F}]|[\u{1F300}-\u{1F5FF}]|[\u{1F680}-\u{1F6FF}]|[\u{1F1E0}-\u{1F1FF}]/gu
  const emojis = String.matchAll(text, emojiRegex)
  
  // Remove emojis for text analysis
  const textOnly = String.replace(text, emojiRegex, '').pipe(
    String.trim,
    (s) => String.replace(s, /\s+/g, ' ')
  )
  
  return {
    originalText: text,
    textOnly,
    codePointLength,
    characterLength,
    emojiCount: Arr.length(emojis),
    emojis: Arr.map(emojis, match => match[0])
  }
})

// Social media username validation with Unicode support
const validateSocialUsername = (username: string) => Effect.gen(function* () {
  const normalized = String.normalize(username)
  
  // Check length (visual characters, not code points)
  const visualLength = Arr.length(Array.from(normalized))
  if (visualLength < 3 || visualLength > 30) {
    return yield* Effect.fail('Username must be 3-30 characters')
  }
  
  // Allow letters, numbers, underscore, hyphen from any language
  const validPattern = /^[\p{L}\p{N}_-]+$/u
  if (!validPattern.test(normalized)) {
    return yield* Effect.fail('Username contains invalid characters')
  }
  
  // Prevent confusing character combinations
  const confusables = /[il1|oO0]/g
  const confusableMatches = String.matchAll(normalized, confusables)
  if (Arr.length(confusableMatches) > 2) {
    return yield* Effect.fail('Username contains too many similar-looking characters')
  }
  
  return String.toLowerCase(normalized)
}).pipe(
  Effect.withSpan('username.validate.social')
)

Feature 2: Advanced Pattern Matching and Text Parsing

Complex text parsing capabilities for structured data extraction.

Regex Pattern Builder

import { String, Array as Arr, Effect, Option } from "effect"

// Composable regex pattern builder
const RegexPatterns = {
  email: /[a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+\.[a-zA-Z]{2,}/,
  phone: /(\+\d{1,3}[-.\s]?)?\(?\d{3}\)?[-.\s]?\d{3}[-.\s]?\d{4}/,
  url: /https?:\/\/(www\.)?[-a-zA-Z0-9@:%._\+~#=]{1,256}\.[a-zA-Z0-9()]{1,6}\b([-a-zA-Z0-9()@:%_\+.~#?&//=]*)/,
  creditCard: /\d{4}[-\s]?\d{4}[-\s]?\d{4}[-\s]?\d{4}/,
  ipAddress: /\b(?:[0-9]{1,3}\.){3}[0-9]{1,3}\b/,
  macAddress: /([0-9A-Fa-f]{2}[:-]){5}([0-9A-Fa-f]{2})/
}

// Smart contact information extractor
const extractContactInfo = (text: string) => Effect.gen(function* () {
  const emails = String.matchAll(text, RegexPatterns.email).pipe(
    Arr.map(match => match[0])
  )
  
  const phones = String.matchAll(text, RegexPatterns.phone).pipe(
    Arr.map(match => String.replace(match[0], /[^\d+]/g, '')),
    Arr.map(phone => phone.startsWith('+') ? phone : `+1${phone}`)
  )
  
  const urls = String.matchAll(text, RegexPatterns.url).pipe(
    Arr.map(match => match[0])
  )
  
  return {
    emails: Arr.dedupeWith(emails, String.Equivalence),
    phones: Arr.dedupeWith(phones, String.Equivalence),
    urls: Arr.dedupeWith(urls, String.Equivalence)
  }
}).pipe(
  Effect.withSpan('contact.extract')
)

Advanced Log Parser

import { String, Array as Arr, Effect, Option } from "effect"

// Structured log entry parser
interface LogEntry {
  timestamp: string
  level: 'ERROR' | 'WARN' | 'INFO' | 'DEBUG'
  service: string
  message: string
  metadata?: Record<string, string>
}

const parseLogEntry = (line: string) => Effect.gen(function* () {
  // Support multiple log formats
  const formats = [
    // ISO timestamp format: 2023-10-15T10:30:00Z [INFO] service-name: message
    /^(\d{4}-\d{2}-\d{2}T\d{2}:\d{2}:\d{2}(?:\.\d{3})?Z?)\s+\[(\w+)\]\s+([^:]+):\s*(.+)$/,
    // Simple format: [2023-10-15 10:30:00] INFO service-name: message
    /^\[(\d{4}-\d{2}-\d{2}\s\d{2}:\d{2}:\d{2})\]\s+(\w+)\s+([^:]+):\s*(.+)$/,
    // Syslog format: Oct 15 10:30:00 service-name[INFO]: message
    /^(\w{3}\s+\d{1,2}\s+\d{2}:\d{2}:\d{2})\s+([^[]+)\[(\w+)\]:\s*(.+)$/
  ]
  
  for (const format of formats) {
    const match = String.match(line, format)
    if (Option.isSome(match)) {
      const [, timestamp, level, service, message] = match.value
      
      // Extract metadata from message if present
      const metadataMatch = String.match(message, /\{([^}]+)\}$/)
      const metadata = Option.isSome(metadataMatch) 
        ? String.split(metadataMatch.value[1], ',').pipe(
            Arr.map(pair => String.split(pair, '=')),
            Arr.filter(parts => Arr.length(parts) === 2),
            Arr.reduce({} as Record<string, string>, (acc, [key, value]) => ({
              ...acc,
              [String.trim(key)]: String.trim(value)
            }))
          )
        : undefined
      
      const cleanMessage = Option.isSome(metadataMatch)
        ? String.replace(message, metadataMatch.value[0], '')
        : message
      
      return {
        timestamp: String.trim(timestamp),
        level: String.toUpperCase(String.trim(level)) as LogEntry['level'],
        service: String.trim(service),
        message: String.trim(cleanMessage),
        metadata
      } satisfies LogEntry
    }
  }
  
  return yield* Effect.fail(`Unable to parse log line: ${line}`)
}).pipe(
  Effect.withSpan('log.parse.entry')
)

// Batch log processing with error recovery
const processLogFile = (content: string) => Effect.gen(function* () {
  const lines = String.split(content, '\n').pipe(
    Arr.map(String.trim),
    Arr.filter(String.isNonEmpty)
  )
  
  const results = yield* Effect.all(
    Arr.map(lines, line => 
      parseLogEntry(line).pipe(
        Effect.catchAll(() => Effect.succeed(null))
      )
    )
  )
  
  const validEntries = Arr.filter(results, (entry): entry is LogEntry => entry !== null)
  const errorCount = Arr.length(results) - Arr.length(validEntries)
  
  // Group by service and level for analysis
  const byService = Arr.groupBy(validEntries, entry => entry.service)
  const errorEntries = Arr.filter(validEntries, entry => entry.level === 'ERROR')
  
  return {
    totalLines: Arr.length(lines),
    parsedEntries: Arr.length(validEntries),
    errorCount,
    byService,
    errors: errorEntries,
    summary: {
      services: Object.keys(byService).length,
      errorRate: errorCount / Arr.length(lines)
    }
  }
}).pipe(
  Effect.withSpan('log.process.file')
)

Feature 3: Performance-Optimized String Operations

Efficient string processing for high-performance applications.

Streaming String Processing

import { String, Array as Arr, Effect, Stream, Chunk } from "effect"

// Memory-efficient large text processing
const processLargeText = (textStream: Stream.Stream<string>) =>
  textStream.pipe(
    Stream.map(String.trim),
    Stream.filter(String.isNonEmpty),
    Stream.mapChunks(chunk => 
      Chunk.map(chunk, line => ({
        line,
        wordCount: String.split(line, /\s+/).pipe(
          Arr.filter(String.isNonEmpty),
          Arr.length
        ),
        charCount: String.length(line)
      }))
    ),
    Stream.runReduce(
      { totalLines: 0, totalWords: 0, totalChars: 0 },
      (acc, chunk) => 
        Chunk.reduce(chunk, acc, (sum, item) => ({
          totalLines: sum.totalLines + 1,
          totalWords: sum.totalWords + item.wordCount,
          totalChars: sum.totalChars + item.charCount
        }))
    )
  )

// Efficient string deduplication for large datasets
const deduplicateStrings = (strings: Array<string>) => Effect.gen(function* () {
  const seen = new Set<string>()
  const unique: Array<string> = []
  
  for (const str of strings) {
    const normalized = String.trim(str).pipe(
      String.toLowerCase,
      String.normalize
    )
    
    if (!seen.has(normalized)) {
      seen.add(normalized)
      unique.push(str)
    }
  }
  
  return {
    original: strings,
    unique,
    duplicateCount: Arr.length(strings) - unique.length,
    deduplicationRate: 1 - (unique.length / Arr.length(strings))
  }
}).pipe(
  Effect.withSpan('string.deduplicate')
)

Practical Patterns & Best Practices

Pattern 1: Safe String Transformation Pipelines

import { String, Array as Arr, Effect, Option } from "effect"

// Helper for safe string operations with fallbacks
const createSafeStringProcessor = <E>(
  operations: Array<(s: string) => Effect.Effect<string, E>>,
  fallback: string = ''
) =>
  (input: string) => Effect.gen(function* () {
    let result = input
    
    for (const operation of operations) {
      const processed = yield* operation(result).pipe(
        Effect.catchAll(() => Effect.succeed(fallback))
      )
      result = processed
    }
    
    return result
  })

// Reusable validation utilities
const StringValidators = {
  nonEmpty: (input: string) =>
    String.isNonEmpty(input) 
      ? Effect.succeed(input)
      : Effect.fail('String cannot be empty'),
      
  maxLength: (max: number) => (input: string) =>
    String.length(input) <= max
      ? Effect.succeed(input)
      : Effect.fail(`String exceeds maximum length of ${max}`),
      
  minLength: (min: number) => (input: string) =>
    String.length(input) >= min
      ? Effect.succeed(input)
      : Effect.fail(`String must be at least ${min} characters`),
      
  pattern: (regex: RegExp, message: string) => (input: string) =>
    regex.test(input)
      ? Effect.succeed(input)
      : Effect.fail(message),
      
  noSpecialChars: (input: string) =>
    /^[a-zA-Z0-9\s]*$/.test(input)
      ? Effect.succeed(input)
      : Effect.fail('String contains special characters')
}

// Composable string cleaning pipeline
const cleanString = createSafeStringProcessor([
  (s) => Effect.succeed(String.trim(s)),
  (s) => Effect.succeed(String.normalize(s)),
  (s) => Effect.succeed(String.replace(s, /\s+/g, ' ')),
  StringValidators.nonEmpty,
  StringValidators.maxLength(1000)
])

Pattern 2: Internationalization-Ready Text Processing

import { String, Array as Arr, Effect } from "effect"

// Locale-aware string utilities
const createI18nStringUtils = (locale: string = 'en-US') => ({
  // Locale-specific case conversion
  toLocaleLowerCase: (input: string) => String.toLocaleLowerCase(input, locale),
  toLocaleUpperCase: (input: string) => String.toLocaleUpperCase(input, locale),
  
  // Locale-aware comparison
  compare: (a: string, b: string) => String.localeCompare(a, b, locale),
  
  // Smart truncation respecting word boundaries
  truncate: (maxLength: number) => (input: string) => Effect.gen(function* () {
    if (String.length(input) <= maxLength) return input
    
    // Find word boundary
    const truncated = String.slice(input, 0, maxLength)
    const lastSpace = String.lastIndexOf(truncated, ' ')
    
    const result = lastSpace > maxLength * 0.8 
      ? String.slice(truncated, 0, lastSpace)
      : truncated
    
    return `${result}…`
  }),
  
  // Extract initials respecting cultural conventions
  getInitials: (fullName: string) => 
    String.trim(fullName).pipe(
      (s) => String.split(s, /\s+/),
      (names) => Arr.filter(names, String.isNonEmpty),
      (names) => Arr.map(names, name => 
        String.charAt(name, 0).pipe(
          char => String.toLocaleUpperCase(char, locale)
        )
      ),
      (initials) => Arr.take(initials, 3),
      (initials) => Arr.join(initials, '')
    )
})

// Multi-language content processor
const processMultiLanguageContent = (content: {
  [language: string]: string
}, defaultLang: string = 'en') => Effect.gen(function* () {
  const processed: Record<string, {
    text: string
    wordCount: number
    charCount: number
    slug: string
  }> = {}
  
  for (const [lang, text] of Object.entries(content)) {
    const utils = createI18nStringUtils(lang)
    
    const cleaned = String.trim(text).pipe(
      String.normalize,
      (s) => String.replace(s, /\s+/g, ' ')
    )
    
    const wordCount = String.split(cleaned, /\s+/).pipe(
      Arr.filter(String.isNonEmpty),
      Arr.length
    )
    
    // Generate language-appropriate slug
    const slug = utils.toLocaleLowerCase(text).pipe(
      (s) => String.replace(s, /[^\p{L}\p{N}\s]/gu, ''),
      (s) => String.replace(s, /\s+/g, '-'),
      (s) => String.slice(s, 0, 50)
    )
    
    processed[lang] = {
      text: cleaned,
      wordCount,
      charCount: String.length(cleaned),
      slug
    }
  }
  
  return processed
}).pipe(
  Effect.withSpan('content.process.multilang')
)

Integration Examples

Integration with Schema Validation

import { String, Array as Arr, Effect, Schema } from "effect"

// Schema-based string validation with custom transformations
const UserSchema = Schema.Struct({
  username: Schema.String.pipe(
    Schema.transform(
      Schema.String,
      {
        decode: (input) => String.trim(input).pipe(
          String.toLowerCase,
          (s) => String.replace(s, /[^a-z0-9_-]/g, ''),
          (s) => String.slice(s, 0, 20)
        ),
        encode: (processed) => processed
      }
    ),
    Schema.filter((s) => String.length(s) >= 3, {
      message: () => "Username must be at least 3 characters"
    })
  ),
  email: Schema.String.pipe(
    Schema.transform(
      Schema.String,
      {
        decode: (input) => String.toLowerCase(String.trim(input)),
        encode: (processed) => processed
      }
    ),
    Schema.filter((s) => /^[^\s@]+@[^\s@]+\.[^\s@]+$/.test(s), {
      message: () => "Invalid email format"
    })
  ),
  displayName: Schema.String.pipe(
    Schema.transform(
      Schema.String,
      {
        decode: (input) => String.trim(input).pipe(
          (s) => String.replace(s, /\s+/g, ' '),
          (s) => String.split(s, ' '),
          (words) => Arr.map(words, String.capitalize),
          (words) => Arr.join(words, ' ')
        ),
        encode: (processed) => processed
      }
    )
  )
})

// Usage with validation
const processUserInput = (rawData: unknown) => Effect.gen(function* () {
  const validated = yield* Schema.decodeUnknown(UserSchema)(rawData)
  
  // Additional processing after schema validation
  const enhancedUser = {
    ...validated,
    usernameHash: String.split(validated.username, '').pipe(
      Arr.map(char => String.charCodeAt(char, 0)),
      Arr.reduce(0, (acc, code) => acc + code),
      String
    ),
    initials: String.split(validated.displayName, ' ').pipe(
      Arr.map(name => String.charAt(name, 0)),
      Arr.join('')
    )
  }
  
  return enhancedUser
}).pipe(
  Effect.withSpan('user.input.process')
)

Integration with HTTP APIs

import { String, Array as Arr, Effect, HttpClient, HttpClientRequest } from "@effect/platform"

// Content-aware HTTP client with string processing
const createContentProcessor = (baseUrl: string) => {
  const client = HttpClient.HttpClient
  
  return {
    // Process and submit text content
    submitText: (content: string, contentType: 'plain' | 'markdown' | 'html' = 'plain') =>
      Effect.gen(function* () {
        // Pre-process content based on type
        const processedContent = (() => {
          switch (contentType) {
            case 'markdown':
              return String.replace(content, /^\s*#\s+/gm, '# ').pipe(
                (s) => String.replace(s, /\n\s*\n\s*\n/g, '\n\n')
              )
            case 'html':
              return String.trim(String.replace(content, />\s+</g, '><'))
            default:
              return String.trim(content)
          }
        })()
        
        const wordCount = String.replace(processedContent, /<[^>]*>/g, '').pipe(
          (s) => String.split(s, /\s+/),
          (words) => Arr.filter(words, String.isNonEmpty),
          Arr.length
        )
        
        const request = HttpClientRequest.post(`${baseUrl}/content`).pipe(
          HttpClientRequest.jsonBody({
            content: processedContent,
            type: contentType,
            metadata: {
              wordCount,
              charCount: String.length(processedContent)
            }
          })
        )
        
        const response = yield* client.execute(request)
        return yield* Effect.tryPromise(() => response.json())
      }).pipe(
        Effect.withSpan('content.submit', { 
          attributes: { 'content.type': contentType } 
        })
      ),
    
    // Fetch and process remote content
    fetchAndProcess: (url: string) => Effect.gen(function* () {
      const request = HttpClientRequest.get(url)
      const response = yield* client.execute(request)
      const content = yield* Effect.tryPromise(() => response.text())
      
      // Process based on content type
      const contentType = response.headers['content-type'] ?? ''
      
      if (String.includes(contentType, 'application/json')) {
        return {
          type: 'json' as const,
          content: yield* Effect.tryPromise(() => JSON.parse(content))
        }
      }
      
      if (String.includes(contentType, 'text/html')) {
        return {
          type: 'html' as const,
          content: String.replace(content, /<script[^>]*>[\s\S]*?<\/script>/gi, '').pipe(
            (s) => String.replace(s, /<style[^>]*>[\s\S]*?<\/style>/gi, ''),
            (s) => String.replace(s, /<[^>]*>/g, ' '),
            (s) => String.replace(s, /\s+/g, ' '),
            String.trim
          )
        }
      }
      
      return {
        type: 'text' as const,
        content: String.trim(content)
      }
    }).pipe(
      Effect.withSpan('content.fetch', { 
        attributes: { 'content.url': url } 
      })
    )
  }
}

Testing Strategies

import { String, Array as Arr, Effect } from "effect"
import { describe, it, expect } from '@effect/vitest'

// Property-based testing for string transformations
describe('String transformations', () => {
  it('should preserve identity through round-trip transformations', () =>
    Effect.gen(function* () {
      const testStrings = [
        'hello world',
        'HELLO WORLD',
        'hElLo WoRlD',
        '  hello world  ',
        'hello_world',
        'HelloWorld'
      ]
      
      for (const input of testStrings) {
        // Test case conversion round-trips
        const upperLower = String.toLowerCase(String.toUpperCase(input))
        const lowerUpper = String.toUpperCase(String.toLowerCase(input))
        
        // Verify normalization is idempotent
        const normalized1 = String.normalize(input)
        const normalized2 = String.normalize(normalized1)
        
        expect(normalized1).toBe(normalized2)
        
        // Test trim idempotency
        const trimmed1 = String.trim(input)
        const trimmed2 = String.trim(trimmed1)
        
        expect(trimmed1).toBe(trimmed2)
      }
    })
  )
  
  it('should handle edge cases consistently', () =>
    Effect.gen(function* () {
      const edgeCases = ['', ' ', '\n', '\t', '\u0000', '🚀', 'café']
      
      for (const input of edgeCases) {
        // Test that operations don't throw
        const result = String.trim(input).pipe(
          String.normalize,
          String.toLowerCase,
          String.toUpperCase
        )
        
        expect(typeof result).toBe('string')
        
        // Test length calculations
        const codePointLength = String.length(input)
        const characterLength = Arr.length(Array.from(input))
        
        expect(characterLength).toBeLessThanOrEqual(codePointLength)
      }
    })
  )
})

// Mock string processing service for testing
const createMockStringService = () => ({
  process: (input: string) => Effect.succeed(
    String.trim(input).pipe(
      String.toLowerCase,
      (s) => String.replace(s, /\s+/g, '-')
    )
  ),
  
  validate: (input: string) => 
    String.isNonEmpty(input)
      ? Effect.succeed(input)
      : Effect.fail(new Error('Empty string'))
})

// Integration test with mock service
describe('String service integration', () => {
  it('should process strings through service pipeline', () =>
    Effect.gen(function* () {
      const service = createMockStringService()
      
      const testInputs = [
        'Hello World',
        '  HELLO WORLD  ',
        'hello   world'
      ]
      
      for (const input of testInputs) {
        const result = yield* service.process(input)
        expect(result).toBe('hello-world')
      }
    })
  )
})

Conclusion

The Effect String module provides a comprehensive, type-safe, and composable solution for string manipulation in TypeScript applications. It addresses common pain points of traditional string processing while maintaining excellent performance and developer ergonomics.

Key benefits:

  • Composability: All functions work seamlessly with pipe for clean transformation pipelines
  • Type Safety: Leverages TypeScript's type system to prevent common string-related errors
  • Internationalization: Built-in Unicode support and locale-aware operations for global applications
  • Performance: Optimized implementations that handle large strings and high-throughput scenarios
  • Consistency: Uniform API design across all string operations

Effect's String module is ideal for applications requiring robust text processing, from simple form validation to complex content management systems and internationalized applications.