A multi-model deliberation engine for LM Studio
Running a single model gives you one perspective. Running a council gives you… perspective squared.
PolyCouncil is an open-source application that orchestrates multiple local Large Language Models (LLMs) running in LM Studio. Instead of querying a single model, PolyCouncil runs multiple models in parallel, has them evaluate each other's responses using a shared scoring rubric, and then conducts a weighted voting process to determine the best answer through consensus.
This creates a more reliable, research-friendly way to:
- Test and compare different local models
- Improve answer quality through ensemble methods
- Explore emergent behavior from model deliberation
- Benchmark models against each other systematically
- Parallel Execution: Multiple models answer the same question simultaneously
- Cross-Evaluation: Each model scores every other model's response using a shared rubric
- Weighted Voting: Models vote on which answer is best, with votes weighted by their rubric scores
- Consensus Result: The winning answer is determined through democratic deliberation
- Parallel Model Execution: Run multiple local LLMs simultaneously and collect answers in parallel
- Rubric-Based Scoring: Each model evaluates every other response across customizable criteria (accuracy, clarity, completeness, etc.)
- Consensus Voting System: Final answer chosen through weighted voting based on rubric scores
- Leaderboard Tracking: Track which models perform best over time with persistent statistics
- Custom Personas: Create and assign different personas to different models for varied perspectives
- Built-in Personas: Includes recommended personas:
- Meticulous fact-checker
- Pragmatic engineer
- Cautious risk assessor
- Clear teacher
- Data analyst
- Systems thinker
- Per-Model Assignment: Apply different personas to different models simultaneously to test how perspectives affect responses
- Single-Voter (Judge) Mode: Use one model as the "ultimate judge" for all votes while all models still answer
- Adjustable Concurrency: Control how many models run simultaneously (1-8 concurrent jobs)
- Debug Mode: Toggle verbose logging to see detailed scoring, voting, and deliberation processes
- Auto-Discovery: Automatically detect available models from LM Studio
- Centralized Settings: Easy-to-use settings dialog for configuration
- Debug Log Dock: Dockable panel for detailed analysis of the deliberation process
- Modern UI: Clean, intuitive interface built with PySide6 with automatic dark/light theme support
- Persistent Settings: All preferences saved automatically
- Download the latest
PolyCouncil.exefrom Releases - Start LM Studio and load at least one model
- Launch
PolyCouncil.exe(no installation required!) - Connect to LM Studio by clicking "Connect"
- Select models you want to use
- Ask questions and watch the council deliberate!
# Clone the repository
git clone https://github.com/TrentPierce/PolyCouncil.git
cd PolyCouncil
# Install dependencies
pip install -r requirements.txt
# Run the application
python council.py- Windows 10/11 (64-bit)
- LM Studio installed and running
- At least one model loaded in LM Studio
- ~50 MB disk space for the executable
- Python 3.10+
- LM Studio installed and running
- pip (Python package manager)
- Dependencies listed in
requirements.txt:- PySide6 (GUI framework)
- aiohttp (async HTTP client)
- requests (HTTP library)
- qdarktheme (optional, for theme support)
- Launch LM Studio and ensure at least one model is loaded
- Start PolyCouncil (either EXE or
python council.py) - Configure Base URL (default:
http://localhost:1234) if LM Studio uses a different port - Click "Connect" to discover available models
- Select models by checking the boxes next to model names
Personas allow you to give different models different perspectives or roles:
- Open Settings (top-right button)
- Check "Enable personas"
- Click the "Persona" button next to each model
- Select a persona from the menu
- Each model will now approach questions from that persona's perspective
Example: Assign "Meticulous fact-checker" to one model and "Pragmatic engineer" to another to see how different perspectives affect their answers.
Use one model as the judge while all models answer:
- Check "Single-voter" in the main window
- Select which model should act as the judge from the dropdown
- All models will answer the question
- Only the selected judge model will vote on which answer is best
This is useful for testing how different models evaluate responses.
Enable detailed logging to understand the deliberation process:
- Open Settings
- Check "Enable debug logs"
- A dockable log panel will appear showing:
- Individual model responses
- Scoring details
- Voting ballots
- Final consensus calculation
Control how many models run simultaneously:
- Adjust the "Max concurrent jobs" spinner (1-8)
- Higher values = faster but more resource-intensive
- Lower values = slower but more stable on modest hardware
Compare how different models perform on the same questions. The leaderboard tracks which models win most often.
Explore emergent behavior from model voting and deliberation. See how ensemble methods improve reliability.
Get more reliable answers by combining multiple model perspectives. The consensus approach often produces better results than any single model.
Understand how ensemble methods, voting systems, and multi-agent systems work in practice.
Test how different personas affect model responses. Useful for understanding bias, perspective, and role-playing in LLMs.
PolyCouncil automatically saves your preferences, including:
- LM Studio base URL
- Selected models
- Persona assignments
- Concurrency settings
- Debug mode preference
- Single-voter settings
Settings are stored in council_settings.json in the application directory.
- Ensure LM Studio is running
- Verify at least one model is loaded in LM Studio
- Check that the base URL is correct (default:
http://localhost:1234) - Click "Refresh Models" if needed
- Reduce "Max concurrent jobs" to 1-2
- Close other resource-intensive applications
- Ensure your models fit in available RAM
- Windows may cache icons - restart File Explorer
- Move the EXE to a different location to force refresh
- Verify LM Studio is running
- Check firewall settings
- Ensure LM Studio API is enabled (default port 1234)
Contributions are welcome! Whether it's:
- Fixing bugs
- Improving the GUI
- Optimizing performance
- Adding new features
- Improving documentation
Please feel free to open an issue or submit a pull request. All contributions help the council grow smarter!
git clone https://github.com/TrentPierce/PolyCouncil.git
cd PolyCouncil
pip install -r requirements.txt
python council.pyThis project is licensed under the Polyform Noncommercial License 1.0.0.
- ✅ Free for personal and research use
- ✅ Free to modify and share
- ❌ Commercial use is not permitted
See LICENSE for full details.
Future enhancements planned:
- Web UI version
- More advanced judge agents
- Plugin system for custom scoring rules
- Model personality profiles / bias testing
- Automatic answer explanation generator
- Graph-based deliberation visualization
- Export results to various formats
- Model performance analytics dashboard
Built for model tinkerers, curious minds, and anyone who enjoys watching artificial brains argue politely.
Special thanks to:
- The LM Studio team for creating an amazing local LLM platform
- The open-source community for inspiration and tools
- All contributors and users who help improve PolyCouncil
- Found a bug? Open an issue
- Have a feature request? Open an issue
- Need help? Check existing issues or open a new one
- Like the project? Star it on GitHub!
Enjoy exploring the power of multi-model deliberation!
Made with care for the local LLM community