Open datasets, collection scripts, and methodology behind our published research.
Website · All Research · Security Tools · License
AppSec Santa is an independent review and comparison platform covering 129+ application security tools across 10 categories including SAST, SCA, DAST, IaC Security, and more.
This repository contains everything needed to verify, reproduce, or build upon our published research — raw datasets, collection scripts, and aggregation code.
6 LLMs · 89 prompts · 534 code samples · 6 SAST tools · 1,173 findings triaged
GPT-5.2 · Claude Opus 4.6 · Gemini 2.5 Pro · DeepSeek V3 · Llama 4 Maverick · Grok 4
Documentation · Published Article
65 tools · 5 health dimensions · GitHub + npm + PyPI + Docker Hub data
Recency · Activity · Releases · Community · Responsiveness
Documentation · Published Article
10,000 websites · Mozilla Observatory scoring · A+ to F grading
CSP · HSTS · X-Frame-Options · Referrer-Policy · X-Content-Type-Options · Redirection · X-XSS-Protection
Documentation · Published Article
Each study follows a three-stage pipeline — collect raw data from public sources, aggregate into scored datasets, and publish findings with full reproducibility.
┌─────────────────────────────────────────────┐
│ Data Collection │
│ │
Source APIs ──────────►│ GitHub API · npm · PyPI · Docker Hub │
LLM APIs ────────────►│ OpenRouter · SAST tool scans │
Target sites ────────►│ HTTP HEAD requests · DNS queries │
│ │
└──────────────────┬──────────────────────────┘
│
▼
┌─────────────────────────────────────────────┐
│ Aggregation & Scoring │
│ │
│ Merge datasets · Compute health scores │
│ Validate findings · Generate distributions │
│ │
└──────────────────┬──────────────────────────┘
│
▼
┌─────────────────────────────────────────────┐
│ Output │
│ │
│ Final JSON dataset · Published article │
│ │
└─────────────────────────────────────────────┘
- Python 3.10+
- Node.js 18+
Study-specific dependencies are listed in each study's README.
Looking for the security scanning tools used in our research? Check out appsecsanta/security-tools — 4 open-source security scanners (HTTP headers, DNS, SSL/TLS, subdomains) you can self-host on Cloudflare Workers.
Found an issue with our data or methodology? Open an issue and we'll look into it.
This project is licensed under the MIT License.
Built by AppSec Santa — curated application security tools comparison.