AI Fundamentals and AI Hacking 101
The AI Fundamentals and AI Hacking 101 ILT teaches students the fundamentals of how AI works under the hood and then how to break it. The first day of the course focuses on the fundamentals of how AI works. Students will learn and perform labs on topics such as: How do neural networks function Training of neural networks The progression of AI for natural language processing Recurrent neural networks (RNN) Large Language Models and Attention Self-Hosting LLMs and interacting with them programmatically The hacking portion of the course focuses on penetration testing AI/LLM based applications such as customer facing chatbots by demonstrating how to detect and exploit common AI vulnerabilities such as: Prompt Injection Sensitive Information Disclosure Improper Output Handling System Prompt Leakage Misinformation Excessive Agency Not only will students learn about these core topics and exploits, but they will also spend hands-on time in a custom-built environment training their own neural networks, tweaking LLMs, exploiting and uncovering vulnerabilities and much more. The online lab features the TCM Vulnerable Chatbot, a customer service chatbot that can interact with customers' tickets and improve its responses via Retrieval Augmented Generation (RAG) using the company's knowledge base.
Description
1 - Intro to neural networks
- Learn how neural networks function, the math behind them and how they are trained.
2 - Neural Network Lab
- Train a neural network to perform basic image recognition of numerals and tweak the neural network design to see how this changes its behavior and effectiveness.
3 - Intro to Natural Language Processing
- Learn about what NLP is and how it works, explore how words can be represented as vectors and a word2vec lab and visualization.
4 - Neural Network Bigrams and Trigrams
- Learn about some of the first text predictive models: bigrams and trigrams and how they can be implemented with basic neural networks.
5 - Recurrent Neural Networks
- Learn about the first attempts to add context to neural networks using recurrent neural networks.
6 – Intro to LLMs
- Learn about the evolution of natural language processing to the LLM and the transformer decoder architecture.
7 – LLM Attention
- Learn how LLM attention works and explore how the attention mechanism adds context in an interactive lab.
8 – Self-Hosting LLMs
- Learn how to self-host your own open source LLMs using Ollama and how to interact with them.
9 – Scripting Examples
- Learn how to write your own basic chatbot and interact programmatically with Ollama or other AI APIs.
10 - AI Fundamentals Review
- A quick review of some of the fundamentals of AI such as how they operate and standard terms such as model parameters, temperature, top-p, inference, training, LLMs.
11 – AI Threat Model
- Discuss the threat actors, assets, adversary goals and attack surfaces for modern AI applications and the specific AI application used in the course
12 - Reconnaissance, Model Mapping and Baseline Behavior and Fingerprinting
- Demonstrate techniques for performing reconnaissance of AI applications with a specific focus on fingerprinting underlying AI models and their settings.
13 - Prompt Injection and Jailbreaking
- Demonstrate common techniques for prompt injection and jail breaking
14 - Prompt Injection Tools and Resources
- Show common tools and repositories of prompts used for prompt injection and jailbreaking
15 - Bypassing Common Protections
- Showcase how to bypass common protections for prompt injection such as input/output filtering
16 - Testing for harmful output/hate speech/misinformation/off-topic content and resource drainage
- Demonstrate tests for verifying the model responds correctly to requests for generating harmful or Off-topic content or attempts to waste resources.
17 - Data Exfiltration
- Demonstrate how retrieval augmented generation works and vulnerabilities associated with it such as leakage of confidential material and PII.
18 - RAG and Vector DB Attacks
- Demonstrate attacks the focus on the retrieval of documents and the ticket base, showcase vector poisoning attacks.
19 – Excessive Agency
- Demonstrate how excessive agency in applications can be exploited and tested for.
Prerequisites
Target Audience
- Penetration testers looking to add AI/LLM pen testing to their tool kit
- Developers working with AI and LLM applications
- Defenders looking to understand AI risks and how they can impact their organizations
- Anyone interested in AI and its risks and dangers