--- title: AI Safety & Alignment description: AI safety studies how to make advanced AI systems behave as intended. You will learn alignment problems, interpretability, evaluations, governance debates and the main research agendas and their critiques. category: programming-tech subcategory: artificial-intelligence difficulty: beginner, intermediate, advanced url: /subject/ai-safety-and-alignment --- # AI Safety & Alignment AI safety studies how to make advanced AI systems behave as intended. You will learn alignment problems, interpretability, evaluations, governance debates and the main research agendas and their critiques. ## Available Resources 1 Books • 2 Courses • 3 Websites • 2 Papers ## Books ### 1. Introduction to AI Safety, Ethics, and Society **Author:** Dan Hendrycks Dan Hendrycks's course textbook, free to read online, spanning catastrophic risk taxonomies, single-agent safety, safety engineering, complex systems, machine ethics, collective action problems and governance. No machine learning background is assumed; appendices supply the needed technical and philosophical grounding. **Difficulty:** Beginner | **Language:** English | **Price:** Paid **Link:** https://www.amazon.com/dp/1032917229?tag=edmonddante07-20 **Tags:** ai-safety, ai-ethics, catastrophic-risk, ai-governance, machine-ethics ## Papers ### 1. AI as Normal Technology **Author:** Arvind Narayanan, Sayash Kapoor Princeton computer scientists Arvind Narayanan and Sayash Kapoor argue against the superintelligence framing, separating AI methods from applications and adoption and proposing resilience-focused policy. The strongest articulated counterposition to catastrophic-risk arguments, and the essay to stress-test them against. **Difficulty:** Intermediate | **Language:** English | **Price:** Free **Link:** https://knightcolumbia.org/content/ai-as-normal-technology **Tags:** ai-policy, ai-risk, ai-governance, technology-adoption ### 2. International AI Safety Report 2026 **Author:** Yoshua Bengio (Chair) and 91 co-authors The second edition of the expert panel report chaired by Yoshua Bengio, written by over 100 researchers and backed by 30-plus governments and the UN, OECD and EU. It synthesises evidence on general-purpose AI capabilities, risks and safeguards. **Difficulty:** Intermediate | **Language:** English | **Price:** Free **Link:** https://arxiv.org/abs/2602.21012 **Tags:** ai-safety, ai-risk, general-purpose-ai, ai-governance, ai-policy ## Websites ### 1. How To Become A Mechanistic Interpretability Researcher **Author:** Neel Nanda Neel Nanda's September 2025 roadmap for entering mechanistic interpretability: what to learn in the first month, how to run one-to-five day throwaway mini-projects, and how to build toward publishable research sprints. Gives a concrete self-directed research plan rather than a reading list. **Difficulty:** Advanced | **Language:** English | **Price:** Free **Link:** https://www.alignmentforum.org/posts/jP9KDyMkchuv6tHwm/how-to-become-a-mechanistic-interpretability-researcher **Tags:** mechanistic-interpretability, ai-alignment, research-careers, neural-networks ### 2. AI Safety Atlas **Author:** Markov Grey, Charbel-Raphael Segerie An open textbook in eight chapters, written by researchers at the French Center for AI Safety and updated quarterly, covering capabilities, threat models, evaluations, interpretability, oversight and governance. Readers finish able to place any safety agenda within a shared conceptual map. **Difficulty:** Intermediate | **Language:** English | **Price:** Free **Link:** https://ai-safety-atlas.com/ **Tags:** ai-safety, threat-models, ai-evaluations, interpretability, scalable-oversight, ai-governance ### 3. Machine Intelligence Research Institute (MIRI) Site of the Berkeley research nonprofit that pioneered technical AI alignment work, hosting papers on agent foundations, decision theory, logical uncertainty and corrigibility alongside essays and policy writing on existential risk. Explains the case that advanced AI could be catastrophically misaligned. **Difficulty:** Intermediate | **Language:** English | **Price:** Free **Link:** https://intelligence.org **Tags:** ai-alignment, ai-safety, decision-theory, existential-risk, agent-foundations ## Courses ### 1. ARENA: Alignment Research Engineer Accelerator Curriculum **Author:** Callum McDougall Five chapters of PyTorch exercises with solutions: deep learning foundations, transformer interpretability with TransformerLens, reinforcement learning, LLM evaluations and alignment science. Working through them leaves you able to replicate interpretability papers and build evaluation harnesses yourself. **Difficulty:** Advanced | **Language:** English | **Price:** Free **Link:** https://learn.arena.education/ **Tags:** ai-alignment, mechanistic-interpretability, transformerlens, pytorch, reinforcement-learning, llm-evaluations ### 2. Technical AI Safety Course (AI Safety Fundamentals) A facilitated cohort course covering alignment and RLHF, mechanistic interpretability, evaluations and red-teaming, AI control and scalable oversight, run over six weeks part-time or six intensive days with expert-led discussion groups. Participants leave able to critique current technical agendas. **Difficulty:** Intermediate | **Language:** English | **Price:** Free **Link:** https://bluedot.org/courses/technical-ai-safety **Tags:** ai-alignment, rlhf, mechanistic-interpretability, red-teaming, scalable-oversight, ai-control --- *This content is part of Dantes.io - Your Treasure Map to Knowledge* *Curated by humans at Dantes.io. Personal study use welcome; republishing this curation requires permission (team@dantes.io).* View this page online: https://dantes.io/subject/ai-safety-and-alignment