redaction performance optimization

[YchampionOP]

Summary

Describe what changed and why.

Changes

Validation

• python -m pytest -q
• Added/updated tests for behavior changes
• Updated docs for command or UX changes

Risk and Rollback

Risk level:

• Low
• Medium
• High

Rollback plan:

[gemini-code-assist]

Summary of Changes

Hello @YchampionOP, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request focuses on enhancing the performance of text redaction functionalities within the codeclaw module. The changes introduce more efficient algorithms for identifying and replacing sensitive information, particularly when dealing with multiple custom patterns or extensive text modifications. The primary goal is to reduce processing time during export operations without altering the core redaction logic.

Highlights

• Redaction Performance Optimization: Implemented significant performance improvements in the redaction process, particularly for custom string redaction and general text redaction, resulting in up to a ~4x speedup.
• Combined Regex for Custom Strings: The redact_custom_strings function was refactored to combine multiple custom string patterns into a single regular expression. This allows for a single pass over the text, drastically reducing the computational complexity from O(N*M) to O(N) where N is text length and M is number of patterns.
• Optimized String Building: The redact_text function now uses a list-based string joining approach (''.join(parts)) instead of repeated string concatenation, which is a standard Python optimization for building strings efficiently.
• Early Exit for Character Type Check: The _has_mixed_char_types utility function was optimized to iterate over the string only once and exit early as soon as all required character types (uppercase, lowercase, digit) are found.
• Documentation of Learning: A new entry was added to the .jules/bolt.md file, documenting the learning and action taken regarding the redaction performance optimization.

Changelog

• .jules/bolt.md
  • Added a new bolt entry detailing the performance optimization for redaction, including the learning and action taken.
• codeclaw/secrets.py
  • Refactored _has_mixed_char_types to use a single loop with early exit for improved efficiency.
  • Updated redact_text to use list-based string construction for better performance during multiple replacements.
  • Rewrote redact_custom_strings to combine all custom patterns into a single regex for a more efficient, single-pass redaction.

Ignored Files

• Ignored by pattern: .github/workflows/** (1)
  • .github/workflows/python-publish.yml

Activity

• No human activity has been recorded on this pull request yet.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature	Command	Description
Code Review	/gemini review	Performs a code review for the current pull request in its current state.
Pull Request Summary	/gemini summary	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help	/gemini help	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

[gemini-code-assist]

Code Review

This pull request introduces significant performance optimizations to the text redaction logic. The changes in _has_mixed_char_types, redact_text, and redact_custom_strings are well-aligned with this goal. Specifically, _has_mixed_char_types is updated to use a single pass over the string, redact_text now uses an efficient join operation for string construction, and redact_custom_strings combines multiple patterns into a single regex for a one-pass scan. My review includes one suggestion for redact_custom_strings to handle cases with overlapping patterns correctly, which is a potential bug that could be addressed.

[gemini-code-assist]

When using a combined regex with |, the order of patterns matters. If a shorter string is a substring of a longer one (e.g., "foo" and "foobar"), the regex engine might match the shorter one first, leading to incorrect redaction like "[REDACTED]bar". To prevent this, you should process the strings in descending order of their length. This ensures that longer, more specific strings are matched before their shorter substrings.

Suggested change

	for target in strings:
	for target in sorted(strings, key=len, reverse=True):