Agent & Coding-Agent Security
Scanning, sandboxing, and runtime enforcement for AI agents, coding assistants, MCP servers, and the skills/plugins they load.
garak
active Modular LLM vulnerability scanner with dozens of probes for jailbreaks, prompt injection, data leakage, and toxic output; doubles as an agent-facing red-team harness.
LLM Guard
archived on GitHub Pluggable set of input/output scanners for LLM applications covering PII leakage, prompt injection, toxicity, and secrets detection.
mcp-scan
active Static and runtime scanner for MCP server configurations that flags tool-poisoning, prompt-injection, and unsafe tool definitions before an agent connects to them.
Rebuff
archived on GitHub Layered prompt-injection detector combining heuristics, a dedicated classifier, vector-store similarity search, and canary tokens.
agentic-radar
slowing down CLI that maps the tool and data-flow graph of agent frameworks such as LangGraph and CrewAI, then flags overly broad tool permissions and untrusted data paths.
AgentDojo
active Benchmark environment for measuring how well tool-using agents resist prompt-injection attacks embedded in the documents and API responses they read.
Vigil
dormant Self-hostable library and REST service that scores prompts and completions for injection, jailbreak, and canary-leak signals.
Model & AI/ML Supply-Chain Security
Tools that inspect model artifacts, serialized weights, and ML pipeline dependencies for malicious or unsafe content.
ai-exploits
dormant Collection of real-world exploits for MLOps platforms (MLflow, Ray, H2O, BentoML) packaged as Metasploit modules and Nuclei templates for authorized testing.
ModelScan
active Scans serialized model files (pickle, PyTorch, Keras, TensorFlow) for unsafe deserialization operators and embedded executable code before they're loaded.
Fickling
active Python pickle decompiler and static analyzer from Trail of Bits that can trace, rewrite, and flag malicious opcodes hidden inside pickled model checkpoints.
picklescan
active Lightweight CLI and library that walks pickle byte-streams looking for suspicious global imports and calls, used by several model hubs as a first-pass gate.
asamm (Agentic SAMM)
active An OWASP SAMM extension proposing a maturity model and control set for securing AI-driven development pipelines rather than a single artifact.
Pentest & Red-Team Agents
Autonomous or semi-autonomous LLM-driven agents that perform reconnaissance, exploitation, and attack-path discovery.
Strix
active Autonomous testing agents that execute code and validate findings with runnable proof-of-concepts rather than just describing a hypothetical vulnerability.
PentAGI
active Multi-agent autonomous pentest framework that runs each engagement inside sandboxed containers and coordinates specialized sub-agents for recon and exploitation.
PentestGPT
active The original academic LLM pentesting assistant that breaks an engagement into reasoning, generation, and parsing modules to guide a human or automated operator.
HexStrike AI
active MCP server that exposes a large catalog of existing offensive tools (nmap, gobuster, nuclei, and more) so any MCP-capable agent can drive them directly.
CAI (Cybersecurity AI)
active Modular bug-bounty-oriented agent framework supporting a wide range of LLM backends; the research build is MIT-licensed with a separate commercial track for production use.
hackingBuddyGPT
active Minimal academic framework for studying LLM-driven Linux privilege escalation and web-app testing, designed to be easy to read and extend rather than production-ready.
Counterfit
dormant Microsoft's automation layer for adversarial ML testing, wrapping attack libraries like ART and TextAttack behind a single command-line interface.
AI-Driven Fuzzing
Two flavors live here: LLMs that write fuzz harnesses for traditional targets, and fuzzers built to stress-test LLMs themselves.
FuzzyAI
active Automated fuzzing framework for discovering jailbreaks and prompt-injection payloads against hosted and self-hosted LLM endpoints.
OSS-Fuzz-Gen
active Google's pipeline for generating and refining OSS-Fuzz harnesses with LLMs; credited with surfacing real memory-safety bugs in widely used C/C++ libraries.
ChatAFL
dormant Extends the AFLNet protocol fuzzer with LLM-suggested message sequences and state transitions to reach deeper protocol states.
PromptFuzz
active Uses LLM-mutated prompts to synthesize fuzz drivers for C/C++ libraries that lack existing harnesses.
Fuzz4All
slowing down General-purpose LLM-based fuzzer that targets many languages and compilers from a single prompt-driven generation loop instead of a hand-written grammar.
SIEM / SOC Alert Triage
LLM-assisted correlation, summarization, and prioritization of security alerts, logs, and incident data.
Wazuh
active Full open-source SIEM/XDR platform; not AI-native, but the most common base that AI triage layers and copilots are built on top of.
TheHive
archived on GitHub Case-management and incident-response platform frequently paired with LLM enrichment scripts for automatic triage notes and severity scoring.
AttackGen
active Generates tabletop incident-response scenarios mapped to MITRE ATT&CK and ATLAS techniques, useful for SOC training as well as detection-gap analysis.
TI-Mindmap-GPT
active Streamlit app that turns raw threat reports into summaries, mind maps, and MITRE ATT&CK Navigator layers for faster analyst triage.
Reverse Engineering
LLM copilots for binary analysis, decompilation, and traffic inspection inside established RE toolchains.
GhidraMCP
dormant MCP server that exposes Ghidra's decompilation and analysis primitives, letting any MCP-capable model drive a live RE session.
angr
active Binary analysis framework (symbolic execution, CFG recovery) that underlies many AI-assisted vulnerability-discovery and exploit-generation research projects.
Gepetto
active IDA Pro plugin that asks an LLM to rename variables and add comments to decompiled functions, cutting down manual annotation time.
burpgpt
dormant Burp Suite extension that pipes HTTP traffic through an LLM during passive scanning to flag anomalies a signature-based scanner would miss.
ghidra_tools (G-3PO)
dormant Tenable's Ghidra plugin for AI-assisted review of decompiled output, aimed at speeding up first-pass triage of unfamiliar binaries.
LLM Red-Teaming & Guardrails
Frameworks for attacking LLM applications on purpose, and the guardrail libraries built to withstand that attention.
promptfoo
active LLM evaluation CLI with a dedicated red-teaming mode offering dozens of attack plugins for jailbreaks, injection, and PII extraction.
NeMo Guardrails
active Programmable rails (input, output, dialog, retrieval) that let developers constrain what an LLM application is allowed to say or do.
PurpleLlama
active Meta's collection of safety tooling — Llama Guard classifiers, CodeShield, and the CyberSecEval benchmark suite — for evaluating and constraining model output.
PyRIT
active Microsoft's Python Risk Identification Tool for generative AI; an extensible red-teaming harness used across hundreds of internal GenAI assessments.
DeepTeam
active Red-teaming framework for LLM systems covering jailbreaks, prompt injection, bias, and data-leakage vulnerability classes with pytest-style assertions.
EasyJailbreak
active Framework for constructing and evaluating adversarial jailbreak prompts across a common set of attack recipes and target models.
Moonshot
active Modular benchmarking and red-teaming tool for LLM applications with pluggable connectors, attack recipes, and evaluation cookbooks.
Threat Intelligence & Deception
AI-assisted IOC/TTP extraction from reports, plus LLM-driven honeypots that simulate convincing fake services.
Beelzebub
active Low-code honeypot framework that uses an LLM to generate realistic SSH, HTTP, and MCP service responses for attacker deception.
shelLM
active Research SSH honeypot that has an LLM improvise a full fake shell session convincingly enough to hold an attacker's attention.
trs
dormant Summarizes threat-intel reports and extracts MITRE ATT&CK TTPs and IOCs using an LLM paired with a local vector store.
CTF / Exploit Benchmarks
Environments and leaderboards used to measure how capable an AI agent is at finding and exploiting real vulnerabilities.
SWE-agent (EnIGMA mode)
active General coding-agent framework whose EnIGMA branch adds an offensive-CTF mode that performs strongly on several public CTF benchmarks.
Cybench
active A set of professional-grade CTF tasks used by multiple AI safety evaluators to benchmark autonomous offensive capability.
InterCode
dormant Interactive-coding benchmark suite that includes an InterCode-CTF track for evaluating agentic exploitation in a sandboxed shell.
Watchlist
Entries here need a maintainer's eye before you rely on them — either flagged manually, or auto-detected because GitHub hasn't seen a push in over a year.
- CAI (Cybersecurity AI) — Dual-licensed (research MIT / commercial). Re-check license terms before recommending for production use.
- Fickling — GPL-3.0 — confirm this fits your project's license posture before embedding it in a larger toolchain.
- Rebuff — Archived upstream; still widely cited, but no longer receiving maintenance. Consider LLM Guard or Vigil instead.
- burpgpt — Requires sending traffic to a third-party LLM API — confirm this fits your data-handling requirements before use on client engagements.
- LLM Guard — auto-detected as dormant (no push in 365+ days)
- Vigil — auto-detected as dormant (no push in 365+ days)
- ai-exploits — auto-detected as dormant (no push in 365+ days)
- Counterfit — auto-detected as dormant (no push in 365+ days)
- ChatAFL — auto-detected as dormant (no push in 365+ days)
- TheHive — auto-detected as dormant (no push in 365+ days)
- GhidraMCP — auto-detected as dormant (no push in 365+ days)
- ghidra_tools (G-3PO) — auto-detected as dormant (no push in 365+ days)
- trs — auto-detected as dormant (no push in 365+ days)
- InterCode — auto-detected as dormant (no push in 365+ days)