--- title: Neural Networks description: Understand the building blocks of deep learning. Study perceptrons, activation functions, backpropagation, optimization algorithms, and network architectures from basics to advanced concepts. category: programming-tech subcategory: artificial-intelligence difficulty: beginner, intermediate, advanced url: /subject/neural-networks --- # Neural Networks Understand the building blocks of deep learning. Study perceptrons, activation functions, backpropagation, optimization algorithms, and network architectures from basics to advanced concepts. ## Available Resources 2 Videos • 1 Books • 3 Courses • 1 Websites • 2 Papers ## Courses ### 1. Neural Networks: Zero to Hero **Author:** Andrej Karpathy Free video lecture series building neural networks from scratch in Python: the micrograd autograd engine, character-level language models, MLPs, activations and batch normalization, manual backpropagation, WaveNet, and a GPT with its tokenizer. Requires solid Python and introductory calculus; learners can implement and train transformers themselves. **Difficulty:** Intermediate | **Price:** Free **Link:** https://karpathy.ai/zero-to-hero.html **Tags:** neural-networks, backpropagation, language-models, transformers, pytorch ### 2. Introduction to Deep Learning (MIT 6.S191) **Author:** Alexander Amini, Ava Amini Annual one-week bootcamp on deep learning: neural network foundations, sequence models and transformers, convolutional networks for vision, generative models, and reinforcement learning. Recorded lectures, slides and software labs in TensorFlow and PyTorch let learners build and train working models. **Difficulty:** Intermediate | **Language:** English | **Price:** Free **Link:** https://introtodeeplearning.com/ **Tags:** deep-learning, neural-networks, transformers, convolutional-neural-networks, generative-models ### 3. Andrew Ng's Deep Learning Specialization This specialization is one of the most popular and comprehensive introductions to deep learning. It covers foundational concepts of neural networks, including how to build and train them, and delves into convolutional networks, recurrent networks, and more. While the certificate requires payment, auditing the courses (watching lectures, accessing many materials) is typically free. **Difficulty:** Beginner | **Language:** English | **Price:** Free **Link:** https://www.coursera.org/specializations/deep-learning **Tags:** deep-learning, neural-networks, convolutional-neural-networks, recurrent-neural-networks, hyperparameter-tuning ## Websites ### 1. Distill.pub Peer-reviewed web journal of machine learning explanations, publishing interactive articles on topics like feature visualisation, attention and neural network interpretability. Archive remains readable, though the journal went on indefinite hiatus in 2021 and no longer publishes. **Difficulty:** Intermediate | **Price:** Free **Link:** https://distill.pub/ **Tags:** deep-learning, neural-networks, interpretability, interactive-visualization, machine-learning ## Papers ### 1. ImageNet Classification with Deep Convolutional Neural Networks **Author:** Alex Krizhevsky, Ilya Sutskever, Geoffrey E. Hinton This paper introduced AlexNet, a groundbreaking convolutional neural network that significantly outperformed previous methods in the ImageNet Large Scale Visual Recognition Challenge (ILSVRC) 2012. It marked a turning point for deep learning's resurgence and practical application. **Difficulty:** Advanced | **Language:** English | **Price:** Free **Link:** https://proceedings.neurips.cc/paper/2012/file/c399862d3b4b6b014381395edb6697af-Paper.pdf **Tags:** convolutional-neural-networks, imagenet, computer-vision, deep-learning, alexnet ### 2. Deep Learning **Author:** Yann LeCun, Yoshua Bengio, Geoffrey Hinton A seminal review paper by three pioneers of deep learning, providing an overview of the field, its history, key concepts, and future directions. It's an excellent read for understanding the landscape of deep learning. **Difficulty:** Advanced | **Language:** English | **Price:** Free **Link:** https://www.nature.com/articles/nature14539 **Tags:** deep-learning, neural-networks, representation-learning, backpropagation, review-paper ## Videos ### 1. StatQuest with Josh Starmer - Neural Networks videos Josh Starmer's StatQuest provides clear, concise, and often humorous explanations of statistical and machine learning concepts. His videos on neural networks break down complex ideas into easily digestible 'quests.' **Difficulty:** Beginner | **Language:** English | **Price:** Free **Link:** https://www.youtube.com/playlist?list=PLXe8IwAUB2B-Ews9ADFJBHE7nVAZAfV05 **Tags:** neural-networks, backpropagation, machine-learning, statistics, video-series ### 2. 3Blue1Brown - Neural Networks series Grant Sanderson's 3Blue1Brown channel offers visually intuitive and mathematically rigorous explanations of complex topics. His series on neural networks is particularly praised for making the core concepts, like backpropagation, understandable through animated visuals. **Difficulty:** Beginner | **Language:** English | **Price:** Free **Link:** https://www.youtube.com/playlist?list=PLW_xnxm7jEso **Tags:** neural-networks, backpropagation, gradient-descent, deep-learning, visual-explanation ## Books ### 1. Neural Networks and Deep Learning **Author:** Michael Nielsen A foundational and widely acclaimed online book that explains neural networks and deep learning concepts from scratch, with clear explanations and interactive examples. It's an excellent resource for building a strong theoretical understanding. **Difficulty:** Beginner | **Language:** English | **Price:** Free **Link:** https://neuralnetworksanddeeplearning.com/ **Tags:** neural-networks, backpropagation, gradient-descent, deep-learning, free-textbook --- *This content is part of Dantes.io - Your Treasure Map to Knowledge* *Curated by humans at Dantes.io. Personal study use welcome; republishing this curation requires permission (team@dantes.io).* View this page online: https://dantes.io/subject/neural-networks