Add streaming parser for large message handling

[konard]

Summary

Implements a streaming parser API for both Rust and JavaScript implementations of links-notation, addressing #197. This enables incremental parsing of Links Notation data, which is essential for handling large messages without loading everything into memory.

Features

Rust Implementation (rust/src/stream_parser.rs)

• StreamParser struct with callback-based API
• on_link() callback for processing parsed links as they arrive
• on_error() callback with detailed error location information (line, column, offset)
• write() method for feeding data chunks incrementally
• finish() method to complete parsing and return all links
• ErrorLocation type with line, column, and offset information
• StreamParseError with enhanced error reporting
• Thread-safe callback support using Send trait bounds
• Comprehensive test suite with 6 tests covering various scenarios

JavaScript Implementation (js/src/StreamParser.js)

• StreamParser class extending EventEmitter
• 'link' event emitted for each parsed link
• 'error' event with line/column location information
• write() method for incremental data feeding
• end() method to finalize parsing
• position() method for tracking parse progress
• Configurable maxInputSize (default: 10MB) and maxDepth (default: 1000)
• Input validation and error handling
• Comprehensive test suite with 17 tests

Use Cases

1. Memory efficiency - Process large messages without loading everything into memory
2. Latency reduction - Start processing before the full message arrives
3. Network integration - Natural fit for TCP/HTTP streaming in links-queue server mode
4. Real-time processing - Handle data as it becomes available

Examples

Rust Example

use links_notation::StreamParser;
use std::sync::{Arc, Mutex};

let mut parser = StreamParser::new();

// Set up callbacks
let count = Arc::new(Mutex::new(0));
let count_clone = Arc::clone(&count);
parser.on_link(move |link| {
    println!("Parsed: {:?}", link);
});

parser.on_error(|error| {
    if let Some(ref loc) = error.location {
        eprintln!("Error at line {}, col {}: {}", loc.line, loc.column, error.message);
    }
});

// Feed data incrementally
parser.write("papa (lovesMama: loves mama)\n")?;
parser.write("son lovesMama\n")?;

// Finish and get all links
let links = parser.finish()?;

JavaScript Example

import { StreamParser } from 'links-notation';

const parser = new StreamParser();

// Listen for events
parser.on('link', (link) => {
  console.log('Parsed:', link.toString());
});

parser.on('error', (error) => {
  console.error(`Error at line ${error.line}, col ${error.column}: ${error.message}`);
});

// Feed data incrementally
parser.write('papa (lovesMama: loves mama)\n');
parser.write('son lovesMama\n');

// Finish parsing
const links = parser.end();

Documentation

• Updated rust/README.md with streaming parser section and API reference
• Updated js/README.md with streaming parser section and API reference
• Added comprehensive examples:
  • examples/rust_streaming_parser.rs - 7 examples demonstrating various use cases
  • examples/js_streaming_parser.js - 6 examples showing streaming capabilities

Testing

• ✅ All 39 existing Rust tests pass
• ✅ All 205 JavaScript tests pass (188 existing + 17 new streaming tests)
• ✅ No regressions introduced
• ✅ All CI checks passing

Implementation Details

Rust

• Uses existing parse_document function from parser module
• Implements incremental line-by-line parsing with parentheses balancing heuristics
• Converts internal Link representation to public LiNo type
• Thread-safe callback support using boxed closures with Send bound

JavaScript

• Extends Node.js EventEmitter for natural event-driven API
• Leverages existing PEG parser with error handling
• Implements buffer management with size limits
• Provides detailed error location tracking from parser

Related Issues

Fixes #197

This implementation directly addresses the needs outlined in the issue for:

• Processing large messages efficiently in links-queue
• TCP streaming integration
• Detailed error reporting with location information

---

🤖 This PR was developed with AI assistance

[konard]

🤖 Solution Draft Log

This log file contains the complete execution trace of the AI solution draft process.

💰 Cost estimation:

• Public pricing estimate: $4.094619 USD
• Calculated by Anthropic: $2.499353 USD
• Difference: $-1.595266 (-38.96%)
  📎 Log file uploaded as Gist (769KB)
  🔗 View complete solution draft log

---

Now working session is ended, feel free to review and add any feedback on the solution draft.