Skip to main content
Dantes.io Logo
Dantes.io

Best resources to learnRunning LLMs Locally

Study sheet

Running open-weight language models on your own hardware keeps data private and costs predictable. You will learn model formats and quantization, memory and GPU requirements, tools such as llama.cpp and Ollama, and how to choose a model for a task.

8 resources · · How we curate

🌐Websites(8)

scroll for more →
website

LM Studio Documentation

The pending set has the low-level engine (llama.cpp) and a CLI runner FAQ (Ollama) but no GUI on-ramp for a non-terminal beginner, and nothing covering Apple Silicon's MLX path. LM Studio covers both in one place, including concrete RAM/VRAM requirements. Official docs for LM Studio, a free desktop app for downloading and running open-weight models offline through llama.cpp (GGUF) or Apple MLX engines. Covers system requirements, model downloads, chatting with documents, and serving models on an OpenAI-compatible local API.

beginner🇬🇧
website

LLM Inference Handbook

Fills the biggest gap in the pending set: hardware and memory sizing (weights x precision + KV cache + overhead, with worked examples and a calculator) plus model-type and GPU selection. Free open-source handbook on how LLM inference works and how to size it: CPU vs GPU, estimating GPU memory from parameters, precision and KV cache, choosing a GPU, choosing an open-weight model, and quantization, with interactive memory calculators.

intermediate🇬🇧
website

Optimizing Your LLM in Production

Works through LLM memory from first principles: the rule that X billion parameters need about 2X GB in 16-bit, 8-bit and 4-bit quantization, Flash Attention, how the key-value cache grows with context, and how multi-query and grouped-query attention reduce it. Teaches how to estimate whether a model and context length will fit on your hardware, which tool docs assume you already know.

advanced🇬🇧
website

Ollama FAQ

The Ollama quickstart now mixes in cloud models and coding-agent setup. Official answers to the practical questions of running models with Ollama: setting context length, checking CPU/GPU placement with ollama ps, keep-alive and memory release, KV-cache quantization, flash attention, and exposing the server on a network.

beginner🇬🇧
website

GGUF (Hugging Face Hub Documentation)

The best short primary source on model file formats. Its quant-type table explains the file names learners see when they download models. Hugging Face documentation for the GGUF model format. Explains how GGUF differs from safetensors, how to find and inspect GGUF files on the Hub, and how they load in llama.cpp, Ollama and LM Studio, with a reference table of quantization types and bits per weight.

beginner🇬🇧
website

A Visual Guide to Quantization

The clearest free explanation of how quantization works, and a good first read before choosing quant files. It dates from 2024, but the concepts have not changed. An illustrated explanation of why LLMs use so much memory and how quantization shrinks them. Covers FP16, BF16 and INT8, symmetric and asymmetric mapping, and post-training methods including GPTQ and GGUF's block-wise scheme, plus quantization-aware training and BitNet.

beginner🇬🇧
website

llama.cpp Quantize Tool: Quantization Types and Trade-offs

Answers the practical question of which quant to download using measured sizes and speeds. Maintainer documentation for llama-quantize. Measured tables for Llama-3.1-8B and 70B show file size, bits per weight and throughput for each quantization type from IQ1_S to F16, and explain importance-matrix (imatrix) quantization and memory needs.

intermediate🇬🇧
website

llama.cpp

Official repository and documentation for llama.cpp, the C/C++ inference engine behind GGUF, Ollama and LM Studio. Covers building for CUDA, Metal, Vulkan and CPU, running models from Hugging Face, and the OpenAI-compatible llama-server. This is the primary source for the engine that almost every local-LLM workflow depends on. Its build, server and model docs are kept current by the maintainers.

intermediate🇬🇧
Share this topic

More in Artificial Intelligence

See all 28 topics in Artificial Intelligence →

Frequently Asked Questions

What are the best free resources to learn Running LLMs Locally?

Dantes has curated 8 resources for Running LLMs Locally, including 8 websites. All resources are hand-picked for quality — no algorithmic filler. Browse the full list above to find the format that works best for you.

Is Running LLMs Locally hard to learn?

Running LLMs Locally is approachable at the beginner level — there are resources here specifically for those starting from scratch. As you progress, intermediate and advanced material is also available to take your skills further.

What types of Running LLMs Locally learning resources are available on Dantes?

For Running LLMs Locally, Dantes has curated 8 websites. Each resource type serves a different learning style: videos and YouTube for visual learners, books for depth, courses for structured progression, and websites for quick reference.

How does Dantes select Running LLMs Locally resources?

Dantes is an algorithm-free learning directory. Resources are hand-curated based on quality, accuracy, and usefulness — not engagement metrics or paid placements. The goal is to surface the best learning material for Running LLMs Locally, whether it's a free YouTube series, a classic textbook, or an open courseware from a top university.

Go deeper

Test your understanding of Running LLMs Locally

Explain it out loud. An AI tutor listens and asks questions that expose gaps you didn't know you had.