Local AI Quest

Tracks

1. 01🏠Why Local AI

   0/5 lessons

   Privacy, cost, latency, offline, freedom

   When local-first wins over cloud-only — and what you give up to get there. Hardware, quantization, formats, and the engine ecosystem you'll work in for the rest of the quest.

   Lesson list (5)

   • 01
     The Local Proposition~22 min · local, strategy, privacy, cost
   • 02
     Hardware Reality~22 min · hardware, vram, apple-silicon
   • 03
     Quantization Tradeoffs~22 min · quantization, gguf
   • 04
     GGUF and MLX Formats~22 min · gguf, mlx, formats
   • 05
     Engines and Ecosystem~20 min · ollama, llamacpp, vllm, lm-studio

   ✓Quiz · 5 questions→

2. 02🦙Ollama Runtime

   0/5 lessons

   Install, pull, run, manage

   Ollama is the simplest correct path from nothing installed to a local model behind a stable HTTP API. Master the CLI, the model library, and the Modelfile system before touching the API.

   Lesson list (5)

   • 01
     Install and Serve~18 min · ollama, install
   • 02
     Pull and Run Models~22 min · ollama, models
   • 03
     The 2026 Model Library~20 min · models, library
   • 04
     Management Commands~16 min · ollama, ops
   • 05
     Modelfile Configuration~24 min · ollama, modelfile

   ✓Quiz · 5 questions→

3. 03🔌Ollama HTTP API

   0/5 lessons

   Endpoints, options, structured outputs

   The HTTP surface you'll talk to from every adapter. /api/chat for conversation, /api/generate for FIM, model management endpoints for ops, and the format field for typed JSON outputs.

   Lesson list (5)

   • 01
     API Surface~18 min · api, endpoints
   • 02
     POST /api/chat — The Conversational Endpoint~26 min · api, chat, messages
   • 03
     POST /api/generate — Raw Completion~16 min · api, completion, fim
   • 04
     Model Management Endpoints~16 min · api, management
   • 05
     Options & Structured Outputs~22 min · api, options, json-schema

   ✓Quiz · 5 questions→

4. 04🌊Streaming the Local Stream

   0/4 lessons

   NDJSON, not SSE

   Ollama streams over NDJSON, not SSE. Get streaming right in Python and TypeScript, then build the optional NDJSON→SSE proxy when you need OpenAI-shaped frontend compatibility.

   Lesson list (4)

   • 01
     NDJSON, Not SSE~18 min · streaming, ndjson
   • 02
     Python Streaming with httpx~22 min · streaming, python, httpx
   • 03
     TypeScript Streaming with fetch~22 min · streaming, typescript
   • 04
     NDJSON → SSE Proxy~20 min · streaming, sse, proxy

   ✓Quiz · 4 questions→

5. 05🔧Local Tool Use

   0/4 lessons

   Function calling that actually works

   OpenAI-format tool definitions, multi-turn tool loops, defensive dispatch with timeouts and error JSON, and the production patterns that keep an agentic loop bounded.

   Lesson list (4)

   • 01
     Local Tool Calling 101~18 min · tools, function-calling
   • 02
     Defining and Calling Tools~22 min · tools, schema
   • 03
     The Multi-Turn Tool Loop~24 min · tools, loop, agent
   • 04
     Tool Calling Best Practices~18 min · tools, practices

   ✓Quiz · 4 questions→

6. 06👁️Local Vision Models

   0/4 lessons

   OCR, charts, screenshots — locally

   Vision-capable local models (Llama 3.2-Vision, Gemma 3, Qwen 2.5-VL), the Ollama image format, multi-image use cases, and where local vision wins versus cloud.

   Lesson list (4)

   • 01
     Vision-Capable Local Models~18 min · vision, multimodal
   • 02
     Sending Images~20 min · vision, images
   • 03
     Multiple Images and Use Cases~18 min · vision, multi-image, ocr
   • 04
     Local vs Cloud Vision~14 min · vision, tradeoffs

   ✓Quiz · 3 questions→

7. 07🏗️Beyond Ollama — Other Servers

   0/4 lessons

   llama.cpp, vLLM, TGI

   Engine map for when you outgrow Ollama. llama.cpp server for raw GGUF, vLLM for concurrent throughput, TGI for HuggingFace-native production, and a compatibility test harness so you can swap engines safely.

   Lesson list (4)

   • 01
     Beyond Ollama — Engine Map~18 min · serving, engines
   • 02
     llama.cpp Server~22 min · serving, llamacpp, gguf
   • 03
     vLLM for Throughput~22 min · serving, vllm, throughput
   • 04
     OpenAI Compatibility Tests~18 min · serving, compatibility, openai

   ✓Quiz · 3 questions→

8. 08🔄The Adapter Pattern

   0/5 lessons

   Make local an interchangeable peer of cloud

   One narrow interface, one universal stream chunk, mini-mode for small models, stateless sessions, and a multi-vessel orchestrator with cloud→local fallback.

   Lesson list (5)

   • 01
     Common AI Adapter Interface~20 min · adapters, abstraction
   • 02
     OllamaAdapter Implementation~26 min · adapters, ollama
   • 03
     Model Selection & Mini Mode~18 min · adapters, selection
   • 04
     Stateless Sessions and Remote Ollama~18 min · adapters, session, remote
   • 05
     Multi-Vessel Fallback~22 min · adapters, fallback, orchestration

   ✓Quiz · 4 questions→

9. 09⚙️Production Operations

   0/5 lessons

   Tune, benchmark, update, ship

   Performance env vars, benchmark discipline, the MLX boundary this quest deliberately stays behind, update hygiene, and shipping a small cloud-first / local-fallback service end-to-end.

   Lesson list (5)

   • 01
     Performance Tuning~22 min · ops, performance
   • 02
     Benchmark Locally~18 min · ops, benchmark
   • 03
     MLX Boundary~14 min · ops, mlx, boundary
   • 04
     Update Discipline~14 min · ops, updates
   • 05
     Ship a Local AI Fallback~22 min · ops, production, fallback

   ✓Quiz · 4 questions→