Skip to content

Combine files in a folder structure into a single text file suitable for use by LLMs, inspired by gpt-repository-loader

License

Notifications You must be signed in to change notification settings

HenriquesLab/folder2md4llms

folder2md4llms

Tests Release PyPI version Python 3.11+ License: MIT Downloads

folder2md4llms is a configurable tool that converts any folder structure and its contents into a single, LLM-friendly Markdown file. It supports various file formats and provides options for content condensing and filtering.

✨ Key Features

  • Smart Condensing: Automatically condenses code to fit within a specified token or character limit without crude truncation.
  • Document Conversion: Converts PDF, DOCX, XLSX, and other document formats into text.
  • Binary File Analysis: Provides intelligent descriptions for images, archives, and other binary files.
  • Highly Configurable: Use a folder2md.yaml file or command-line options to customize the output.
  • Parallel Processing: Uses multi-threading for processing multiple files concurrently.
  • Advanced Filtering: Uses .gitignore-style patterns to exclude files and directories.

🚀 Installation

Requires Python 3.11+

# Using pipx (recommended)
pipx install folder2md4llms

# Or using pip
pip install folder2md4llms

Package name is folder2md4llms, command is folder2md

Basic Usage

# Process the current directory and save to output.md
folder2md .

# Process a specific directory and set a token limit
folder2md /path/to/repo --limit 80000t

# Copy the output to the clipboard
folder2md /path/to/repo --clipboard

# Generate a .folder2md_ignore file
folder2md --init-ignore

# Upgrade to the latest version
folder2md --upgrade

# Check for available updates
folder2md --upgrade-check

For a full list of commands and options, see the CLI Reference or run folder2md --help.

🚨 Troubleshooting

Command not found?

  • Ensure pipx is installed: pip install pipx
  • Or use: python -m folder2md4llms .

Need Help?

🔧 Configuration

You can configure folder2md4llms by creating a folder2md.yaml file in your repository's root directory. This allows you to set advanced options and define custom behavior.

For more details, see the CLI Reference.

🛠️ Development

Interested in contributing? Get started with these simple steps:

# Clone the repository
git clone https://github.com/henriqueslab/folder2md4llms.git
cd folder2md4llms

# Set up the development environment
just setup

# See all available commands
just --list

🤝 Contributing

Contributions are welcome! Please feel free to submit a Pull Request. For more information, see the Contributing Guidelines.

📖 Documentation

📦 Distribution Channels

📄 License

This project is licensed under the MIT License - see the LICENSE file for details.

About

Combine files in a folder structure into a single text file suitable for use by LLMs, inspired by gpt-repository-loader

Topics

Resources

License

Code of conduct

Contributing

Stars

Watchers

Forks

Packages

No packages published

Contributors 2

  •  
  •