Add support for logrotated and compressed access.log files

[Copilot]

This PR implements support for reading logrotated access log files, including compressed formats, addressing the need to maintain statistics across log rotations.

Changes Made

Core Features

• Logrotated file support: Automatically discovers and processes access.log.1, access.log.2, etc.
• Compression support: Handles both gzip (.gz) and zstandard (.zst) compressed log files
• Processing tracking: Uses database settings to avoid reprocessing already-handled files
• Backward compatibility: Maintains existing real-time log tailing functionality

Implementation Details

File Discovery

The new GetLogFiles() method discovers log files in the correct order:

access.log (current, real-time tailed)
access.log.1 (most recent rotated)
access.log.2.gz (older, compressed with gzip)
access.log.3.zst (oldest, compressed with zstandard)

Compression Handling

The OpenLogFileStream() method automatically detects and decompresses:

• Gzip files (.gz): Uses GZipStream for streaming decompression
• Zstandard files (.zst): Uses ZstdNet for efficient decompression

Processing Flow

1. Historical processing: Process all rotated/compressed files first
2. Duplicate prevention: Track processed files in database to avoid reprocessing
3. Real-time tailing: Continue tailing the current access.log as before

Benefits

• Data preservation: No loss of statistics when logs are rotated
• Storage efficiency: Support for compressed logs saves significant disk space
• Performance: Only processes new/unprocessed files
• Reliability: Comprehensive error handling and logging

Testing

• Added 7 new unit tests covering file discovery, compression handling, and edge cases
• All existing tests continue to pass
• Manual testing verified with sample gzip and zstandard compressed files

Example Use Case

With logrotate configuration:

/var/log/lancache/access.log {
    daily
    rotate 7
    compress
    compresscmd /usr/bin/zstd
    compressext .zst
}

The application will now automatically process:

• access.log (real-time)
• access.log.1 (yesterday's uncompressed)
• access.log.2.zst through access.log.7.zst (compressed historical data)

Fixes #65.

---

💬 Share your feedback on Copilot coding agent for the chance to win a $200 gift card! Click here to start the survey.

[devedse]

Sorry, just playing around with this coding agent for a bit :D