Arcanum AI Sec Resource Hub

with ❤️ from www.arcanum-sec.com

23 Active Labs
5 Competitions
4 Bug Bounties
7 Security Tools
3 Text Resources

Hack The Agent

All Levels • Online Platform

Interactive AI security challenge platform with progressive difficulty levels. Test your skills in prompt injection, jailbreaking, and AI manipulation techniques.

  • Multiple challenge categories
  • Real-time scoring system
  • Community leaderboard
  • Educational walkthroughs
Start Challenges →

Broken LLM Integration App

Beginner-Intermediate Level

Comprehensive vulnerable LLM application demonstrating common integration flaws. Based on the open-source project exploring real-world AI security vulnerabilities in web applications.

  • Direct Prompt Injection
  • Indirect Prompt Injection
  • Prompt Leaking
  • P2SQL Injection
  • LLM2Shell
  • 📂 View Source Code
Get Source Code →

Gandalf - Lakera AI

Beginner-Advanced Level

Popular online prompt injection challenge by Lakera AI. Progressive levels teaching fundamental to advanced prompt engineering and security bypass techniques.

  • Progressive difficulty levels
  • Real-time feedback
  • Community leaderboards
  • Fundamental prompt injection
  • External hosted challenge
Start Challenge →

Gandalf Agent Breaker - Lakera AI

Intermediate-Advanced Level

Advanced agent-based prompt injection challenge by Lakera AI. Test your skills against AI agents that can use tools and take actions. Push the boundaries of agentic AI security and discover vulnerabilities in multi-step AI workflows.

  • Agentic AI security testing
  • Tool-using AI agent targets
  • Multi-step workflow exploitation
  • Advanced prompt injection techniques
  • Real-world agent vulnerabilities
  • Progressive difficulty challenges
  • Interactive feedback system
Break the Agent →

Gandalf Adventures - Lakera AI

Beginner-Advanced Level

Complete collection of Gandalf adventure challenges by Lakera AI. Seven unique scenarios testing different prompt injection techniques and AI security concepts, from basic password extraction to advanced jailbreaking methods.

  • 7 unique adventure scenarios
  • Progressive difficulty across adventures
  • Diverse prompt injection techniques
  • Scenario-based learning approach
  • Creative storytelling elements
  • Beginner to advanced skill levels
  • Interactive challenge format
  • Real-time feedback system
Start Adventures →

HackMerlin

Beginner-Intermediate Level

The alternative wizard challenge for prompt injection!

  • Advanced prompt injection
  • Interactive challenge format
  • External hosted platform
Start Challenge →

Immersive Labs Prompt Injection

Beginner-Intermediate Level

Professional prompt injection training platform by Immersive Labs with 10 progressive levels. Learn to hack AI chatbots by extracting secret passwords through increasingly sophisticated injection techniques.

  • 10 levels with increasing difficulty
  • Secret password extraction challenges
  • DLP and word filtering bypasses
  • Professional cybersecurity training
  • Hands-on AI vulnerability exploration
  • Free access challenge platform
Start Challenge →

GPT Prompt Attack

Beginner Level

Interactive web-based game challenging users to craft the shortest possible prompt that can trick an AI assistant into revealing its system prompt's secret key. Perfect introduction to prompt injection concepts.

  • Minimalist prompt crafting challenge
  • Secret key extraction goal
  • Multiple difficulty levels
  • Hall of Fame leaderboard
  • Gamified prompt engineering
  • System prompt vulnerability exploration
Start Game →

TensorTrust

Intermediate Level

⚠️ Buggy

UC Berkeley research platform combining attack and defense scenarios in a gamified environment. Players create defense prompts to protect assets and craft attack prompts to gain unauthorized access through prompt injection.

  • Attack vs Defense game mechanics
  • 126K+ prompt injection dataset
  • Research benchmark contributions
  • Leaderboard competition system
  • Open-source research project
  • UC Berkeley developed platform
Join Research →

PromptMe - OWASP LLM Top 10

Intermediate-Advanced Level • Self-Hosted

🚧 Under Construction ⚠️ Buggy

Educational CTF-style platform with 10 hands-on challenges based on OWASP LLM Top 10 vulnerabilities. Runs locally using Python and Ollama framework with open-source models like Mistral and Llama3.

  • 10 OWASP LLM Top 10 challenges
  • CTF-style interactive format
  • Local setup with no API keys required
  • Hints and solutions provided
  • Real-world LLM vulnerability scenarios
  • 📂 View Source Code
Get Source Code →

SecOps Group - HackAI Mock Lab

Intermediate Level

Advanced AI/ML penetration testing mock lab by The SecOps Group. Hands-on practical exercises covering AI/ML vulnerabilities including prompt injection, model attacks, and certification preparation scenarios.

  • AI/ML penetration testing scenarios
  • Certified AI/ML Pentester prep
  • Advanced prompt injection challenges
  • Model vulnerability assessment
  • Professional certification training
  • Hands-on practical exercises
Start Lab →

OWASP FinBot CTF Demo

Intermediate Level

Agentic AI security CTF simulating goal manipulation attacks against AI-powered financial systems. The "Juice Shop for Agentic AI" - manipulate FinBot to approve fraudulent invoices without triggering detection.

  • Goal manipulation vulnerabilities
  • AI-powered invoice processing attacks
  • Prompt injection detection evasion
  • Easy, Intermediate, and Hard levels
  • Financial services AI simulation
  • 📂 View Source Code
Start CTF →

HackingLLMs - JARVIS Protocol

Intermediate Level

JARVIS-themed cybersecurity challenge platform by Tyson0x0 featuring 4 distinct prompt injection challenges. Navigate through high-tech protocols with an Iron Man-inspired interface and competitive leaderboard system.

  • 4 JARVIS-themed prompt injection challenges
  • Iron Man-inspired cyberpunk interface
  • Operator ranking leaderboard
  • High-tech challenge protocols
  • Interactive sci-fi narrative framework
  • Created by Tyson0x0
Access Protocols →

PortSwigger LLM Labs

Intermediate-Advanced Level

Comprehensive series of 4 LLM security labs by PortSwigger. Covers indirect prompt injection, data exfiltration, cross-user data leakage, and authentication bypass techniques.

  • 4 distinct LLM attack scenarios
  • Indirect prompt injection
  • Data exfiltration techniques
  • Cross-user data leakage
  • Authentication bypass methods
  • Industry-standard lab environment
Start Lab Series →

MyLLMBank

Intermediate-Advanced Level

Advanced prompt injection challenge against chained AI agents that perform data transformation. Explore sophisticated attacks targeting multi-agent banking systems.

  • Chained agent architectures
  • Data transformation used in protection like some guardrails and classifiers
  • Multi-agent prompt injection
  • Banking workflow exploitation
  • Agent-to-agent attack chains
Start Banking Challenge →

MyLLMDoc

Intermediate-Advanced Level

Document-focused AI security challenge exploring vulnerabilities in document processing and analysis systems. Practice attacks against LLM-powered document handling applications.

  • Document processing attacks
  • File upload vulnerabilities
  • Content extraction bypasses
  • Document analysis exploitation
  • RAG system vulnerabilities
Start Document Challenge →

Auto Parts CTF

Intermediate Level

Chained LLM-powered auto parts system with multiple vulnerability types. Features real-time WebSocket communication and API endpoints.

  • Prompt injection attacks
  • IDOR vulnerabilities
  • WebSocket exploitation
  • API security testing
Get Source Code →

PwnGPT CTF

Intermediate Level • Self-Hosted

Agentic LLM CTF with vector searching and OpenAI LLMs. Features 10+ progressive levels teaching prompt injection, information retrieval, and LLM security vulnerabilities.

  • 10+ progressive difficulty levels
  • RAG with llamaindex integration
  • Agentic abilities with react agents
  • Multiple model support (GPT-3.5/4, Llama)
  • Flag-based CTF progression
  • 📂 View Source Code
Get Source Code →

Professional-Secure-AI-Bot

Intermediate Level • Self-Hosted

Multi-feature secure AI platform demonstrating proper security implementations. Includes RAG, web assistance, and security demos.

  • RAG chatbot implementation
  • Text management system
  • Web assistant functionality
  • Security-focused design
  • XSS prevention demos
  • 📂 View Source Code
Get Source Code →

Prompt Airlines

Intermediate Level

AI security CTF by Wiz featuring 5 progressive challenges where participants manipulate a customer service chatbot to earn a free airline ticket. Learn prompt injection through hands-on practice.

  • 5 progressive prompt injection challenges
  • AI chatbot manipulation techniques
  • System prompt leaking
  • Authentication bypass methods
  • Coupon code extraction
  • Real-world AI vulnerability scenarios
Start Flight →

Dreadnode Crucible

Specialized Platform

🔥 Advanced

Professional ML/AI security platform with 80+ challenges covering prompt injection, adversarial attacks, model inversion, and data poisoning. Features challenges from DEFCON, Black Hat, and GovTech competitions.

  • 80+ ML/AI security challenges
  • Prompt injection & LLM escapes
  • Adversarial image/audio attacks
  • Model inversion & fingerprinting
  • Data poisoning scenarios
  • DEFCON/Black Hat competition challenges
  • Interactive Jupyter notebooks
  • Professional cloud platform
Access Platform →

LMQL Demo Environment

Intermediate Level • Self-Hosted

Language Model Query Language demonstrations with specialized bots for different use cases.

  • General Chat Interface
  • Summary Bot
  • Analyst Bot
  • LMQL syntax exploration
Get Source Code →

ℹ️ Competition Status

These competitions remain available for practice and skill development. However, please note that the monetary prize periods have concluded. You can still participate to test your skills, learn new techniques, and compete on leaderboards where available.

Pangea AI Escape Room

Intermediate-Advanced Level

Interactive AI escape room challenge where participants use prompt injection techniques to outsmart AI chatbot supervisors and reveal secret passcodes. Features $10K competition with global leaderboard.

  • Virtual escape room scenarios
  • AI chatbot prompt injection
  • Multi-level difficulty progression
  • Global leaderboard competition
  • Real-world prompt engineering
  • Professional security platform
Start Challenge →

RedTeam Arena

Intermediate Level

Open-source community-driven LLM red-teaming platform featuring gamified AI security challenges. Players have 60 seconds to convince models to say target words using jailbreaking techniques.

  • Gamified LLM red-teaming
  • 60-second jailbreak challenges
  • Community-driven platform
  • Elo scoring system
  • No registration required
  • Open-source codebase
Start Game →

HackAPrompt 2.0

Advanced Competition

World's largest AI red-teaming competition with $100,000+ prize pool by Learn Prompting & OpenAI. Multiple tracks for discovering AI vulnerabilities through jailbreaking and prompt engineering attacks.

  • $100,000+ total prize pool
  • CBRNE, Agents, and Classic GenAI tracks
  • Shortest jailbreak challenges
  • 2-month competition duration
  • Educational tutorials included
  • OpenAI partnership
Join Competition →

Gray Swan AI Arena

Intermediate Competition

Competitive AI safety and alignment arena featuring prompt injection challenges, model evaluation, and red-teaming competitions. Test your skills against various AI models in structured competitive scenarios.

  • Model evaluation challenges
  • Prompt injection competitions
  • AI safety testing scenarios
  • Structured competitive format
  • Multiple AI model targets
  • Community-driven platform
Enter Arena →

LLM Hacker Challenge

Intermediate Competition • All About AI

Interactive LLM hacking challenge created by All About AI. Test your prompt engineering and jailbreaking skills through progressively difficult levels designed to push the boundaries of AI security and model manipulation.

  • Progressive difficulty challenges
  • Prompt injection techniques
  • Jailbreak scenarios
  • Real-time feedback system
  • Community leaderboard
  • Educational content creator challenge
  • Browser-based interface
Start Challenge →

Anthropic Bug Bounty

Professional Bounty

Official Anthropic bug bounty program for reporting security vulnerabilities in Claude AI systems and infrastructure. Submit security findings through their responsible disclosure process.

  • Official Anthropic program
  • Claude AI system vulnerabilities
  • Infrastructure security issues
  • Responsible disclosure process
  • Professional security research
  • Direct impact on AI safety
Submit Report →

OpenAI Bug Bounty

Professional Bounty

OpenAI's bug bounty program hosted on Bugcrowd for discovering security vulnerabilities in ChatGPT, GPT API, and related OpenAI services and infrastructure.

  • Bugcrowd hosted program
  • ChatGPT & GPT API scope
  • OpenAI infrastructure targets
  • Professional bounty rewards
  • Security research community
  • Industry-leading AI security
Join Program →

Google Gemini Bug Bounty

Professional Bounty

Google's Abuse Vulnerability Reward Program for Gemini AI models and services. Part of Google's Bug Hunters program focusing on AI safety and security vulnerabilities.

  • Google Bug Hunters program
  • Gemini AI model vulnerabilities
  • Abuse & safety vulnerability focus
  • Professional reward structure
  • Google AI services scope
  • Industry-standard disclosure process
Join Program →

0din.ai - GenAI Bug Bounty

Professional Bounty

Mozilla's 0-Day Investigative Network GenAI bug bounty program targeting vulnerabilities in large language models and generative AI systems. Rewards up to $15,000 for critical discoveries.

  • $500 - $15,000 reward range
  • Guardrail jailbreak research
  • Prompt injection vulnerabilities
  • Training data leakage detection
  • Mozilla partnership program
  • GenAI security focus
Submit Vulnerabilities →

P4RS3LT0NGV3 - Extended Edition

Prompt Injection Tool • Self-Hosted

Extended version of P4RS3LT0NGV3 with additional payload generation techniques. Advanced prompt injection payload generator with 30+ text transformation techniques for LLM security testing and red teaming.

  • 30+ text transformation methods
  • Extended payload generation techniques
  • Unicode and encoding obfuscation
  • Mathematical and linguistic transformations
  • Real-time payload preview
  • Copy-to-clipboard functionality
  • Additional custom transformations
Access Instance →

P4RS3LT0NGV3 - Original

Prompt Injection Tool • Online

Original P4RS3LT0NGV3 by Elder Plinius. A prompt injection payload generator that creates obfuscated prompts using various text transformation techniques to test LLM security controls and filters.

  • 20+ text transformation techniques
  • Leetspeak and character substitution
  • ROT13 and Caesar cipher encoding
  • Reverse text and word scrambling
  • Homoglyph and lookalike characters
  • Morse code and phonetic spelling
  • 📂 View Source Code
Access Online Tool →

Pyrit - Python Risk Identification Tool

Red Team Framework • Microsoft

Microsoft's Python Risk Identification Tool for Generative AI (PyRIT). An open-source automation framework designed to empower security professionals and ML engineers to proactively identify risks in AI systems through automated red teaming.

  • Automated AI red teaming framework
  • Multi-turn attack orchestration
  • Prompt injection automation
  • Jailbreak technique library
  • Risk assessment scoring
  • Extensible attack strategies
  • Integration with Azure AI services
  • Comprehensive documentation & examples
  • 📂 View Source Code
Get Framework →

Garak - LLM Vulnerability Scanner

Security Scanner • NVIDIA

NVIDIA's comprehensive LLM vulnerability scanner that probes for hallucination, data leakage, prompt injection, misinformation, toxicity generation, jailbreaks, and many other weaknesses in Large Language Models. Think of it as "nmap for LLMs".

  • 40+ probe modules for different vulnerabilities
  • Supports multiple LLM providers (OpenAI, Hugging Face, etc.)
  • Automated vulnerability scanning
  • Jailbreak detection and testing
  • Toxicity and bias evaluation
  • Data leakage detection
  • Hallucination testing
  • JSON and HTML reporting
  • Extensible plugin architecture
  • 📂 View Source Code
Get Scanner →

Promptfoo - LLM Testing & Red Teaming

Testing Framework • Open Source

Open-source LLM testing and red teaming framework for evaluating prompt quality, catching regressions, and identifying vulnerabilities. Test your prompts, agents, and RAG applications for security, quality, and performance issues.

  • Automated red teaming with 15+ attack types
  • OWASP LLM Top 10 vulnerability testing
  • Prompt injection & jailbreak detection
  • PII leakage and data security testing
  • Hallucination and factuality checking
  • Custom security policy enforcement
  • Side-by-side model comparison
  • CI/CD integration support
  • Web UI for visual test management
  • 📂 View Source Code
Get Started →

Spikeé - AI Security Analysis Platform

Security Platform • Arcanum

Advanced AI security analysis platform for comprehensive testing of LLM applications. Provides automated vulnerability assessment, prompt injection testing, and security posture evaluation for AI systems with enterprise-grade reporting.

  • Automated security assessment workflows
  • Comprehensive prompt injection testing
  • Jailbreak attempt detection
  • Data exfiltration prevention testing
  • Model behavior analysis
  • Security compliance checking
  • Real-time threat monitoring
  • Enterprise reporting dashboard
  • API security testing
Access Platform →

PyRIT SHIP - Burp Suite Extension

Burp Extension • Microsoft

PyRIT-Ship is a prototype project that extends Microsoft's PyRIT (Python Risk Identification Toolkit) by providing API integration capabilities for security testing tools. Features a Python Flask server and Burp Suite Intruder extension for AI safety testing.

  • PyRIT API server with Flask implementation
  • Burp Suite Intruder extension integration
  • Prompt generation and scoring capabilities
  • Experimental AI vulnerability testing
  • Microsoft AI Red Team project
  • Community-driven development
  • Flexible integration for security tools
  • Prototype approach to AI safety testing
  • Open-source passion project
  • 📂 View Source Code
Get Extension →

Arcanum Prompt Injection Taxonomy

Research Resource • Arcanum Security

Comprehensive taxonomy and classification system for prompt injection attacks developed by Arcanum Security. A structured framework for understanding, categorizing, and analyzing different types of prompt injection vulnerabilities and attack vectors.

  • Systematic attack classification framework
  • Comprehensive vulnerability taxonomy
  • Research-based methodology
  • Attack vector categorization
  • Security assessment guidelines
  • Educational reference material
  • Open-source research contribution
  • 📂 View Repository
Explore Taxonomy →

AI Pentest Questionnaire

Assessment Guide • Arcanum Security

Comprehensive penetration testing questionnaire for AI systems developed by Arcanum Security. A structured assessment guide covering security evaluation criteria, attack vectors, and vulnerability assessment methodologies for AI/LLM applications.

  • Structured pentest assessment framework
  • AI-specific security evaluation criteria
  • Comprehensive vulnerability checklist
  • Attack vector identification guide
  • Security assessment methodology
  • Professional pentesting resource
  • Industry best practices
  • 📂 View Questionnaire
Access Guide →

AI Security Ecosystem

Research Collection • Arcanum Security

Enterprise AI deployment ecosystem mapping project by Arcanum Security. Maps applications and components in the orbit of enterprise AI deployments to help AI pentesters identify and include all relevant components in their security testing scope.

  • Enterprise AI deployment mapping
  • Component identification for pentesters
  • AI infrastructure ecosystem analysis
  • Testing scope expansion guidance
  • Adjacent system identification
  • Enterprise architecture components
  • AI supply chain mapping
  • Comprehensive testing coverage
  • 📂 View Ecosystem
Explore Ecosystem →