Language models never see text. They see integers. The layer that turns a string into a list of token IDs is byte-pair encoding, a 1994 compression trick repurposed to carve words into subword units. It quietly decides your context window, your bill, and why the model cannot spell.
Llm
-
Tokenization and Byte-Pair Encoding: How Text Becomes Numbers -
Claude Sonnet 5 and Fable 5: The New Generation, Priced and Positioned Anthropic shipped Claude Sonnet 5 on June 30 at $2/$10 per million tokens, closing most of the capability gap to Opus 4.8 at a third of the price — while Fable 5 spent 19 days pulled from the market under US export controls before returning July 1 with a stricter cybersecurity classifier. What actually changed, what the benchmark table hides, and which tier real workloads should run on now.
-
AI-Assisted Coding: The Complete Developer's Guide to Tools, Agents, and Local Models A deep dive into the AI coding ecosystem — Claude Code, Google Antigravity, MCP servers, Cursor, Windsurf, Continue.dev, Aider, and running powerful models locally with Ollama and LM Studio.
-
KV Cache Engineering: PagedAttention, Continuous Batching, Attention Variants, and the Bandwidth Wall The KV cache is the single biggest lever for LLM inference cost and throughput. How it works, why naive allocation wastes 60–80% of GPU memory, how PagedAttention and continuous batching fix that, what prefix caching actually saves, how FP8/INT8 quantization and GQA/MLA shrink the cache, and why long context is fundamentally a memory-bandwidth problem.
-
LLM Quantization Compared: GGUF, AWQ, GPTQ, FP8, and the int4 Cliff A practical comparison of the quantization formats that decide whether a model fits your GPU: weight-only versus activation quantization, what calibration data actually buys you, the accuracy cost of int4, and which of GGUF, GPTQ, AWQ, and FP8 makes sense for which workload.
-
Speculative Decoding Explained: Trading Compute for Bandwidth to Decode Faster How draft-model speculative sampling makes LLM inference 2-3x faster without changing the output: why decode is bandwidth-bound, the draft-verify-accept loop, the rejection-sampling step that keeps it lossless, the acceptance-rate math that decides the speedup, EAGLE and Medusa as learned drafters, and the batch-size regime where speculation quietly makes things slower.
-
Mixture of Experts, Honestly: Why Every Frontier Model Went Sparse and What It Actually Costs The honest accounting on Mixture of Experts: why every frontier model in 2026 is sparse, what the "X total / Y active" parameter math actually buys you, the routing problems the marketing skips, and the all-to-all communication tax that decides whether a MoE model ships or stalls.
-
RAG Beyond Toy Demos: Chunking, Reranking, and the Evaluation Problem Nobody Solved The gap between a notebook RAG demo and a system that ships: why chunking is the retrieval you never tuned, query rewriting and HyDE, the retrieve-wide rank-narrow pipeline, citation faithfulness and grounding, and the evaluation problem that quietly decides whether any of it works.
-
Pi: The Agentic Coding Harness That Refuses to Have Opinions Pi is a deliberately minimal terminal coding agent — four tools, a unified multi-provider LLM API, and an extension system instead of a feature list. How its four-tool core, four run modes, tree sessions, and bring-your-own-model design differ from Claude Code, and the honest trade-offs of a harness you have to assemble yourself.
-
DiffusionGemma: When the LLM Stops Writing Left to Right DiffusionGemma is Google's first open-weight diffusion language model — text generated by denoising blocks in parallel instead of one token at a time. How it differs from autoregressive Gemma, the real speed win and the honest benchmark tax, and exactly what it takes to run the thing on your own GPU.
-
Claude Fable 5 vs Opus: A Realistic Look at Whether You Need the Mythos-Class Model Anthropic just released Claude Fable 5, the first public Mythos-class model, at double the price of Opus 4.8. A realistic breakdown of where the capability gap actually shows up, where it does not, the safety-routing wrinkle, and a decision framework for when to pay for Fable versus staying on Opus or Sonnet.
-
Guardrails for Production LLM Applications A defense-in-depth playbook for the AI systems you actually ship: input and output filtering, system-prompt hardening and instruction hierarchy, sandboxing and least-privilege for tool-using agents, human-in-the-loop gates, structured-output and allow-list constraints, PII redaction, injection detection with heuristics and classifier models, rate limiting and spend caps, and red-teaming your own app. The tooling — NeMo Guardrails, Llama Guard, Guardrails AI, LLM Guard, Presidio — and the honest trade-offs of each layer.
-
The OWASP LLM Top 10 and Prompt Injection The threat model for applications built on language models: direct and indirect prompt injection, jailbreaks, system-prompt and training-data leakage, insecure output handling, and excessive agency in tool-using agents. Real exploit patterns from 2025 — zero-click exfiltration, confused-deputy tool calls, denial-of-wallet — and the uncomfortable reason classic input validation does not save you.
-
AI in the Terminal A practical guide to AI-assisted development and automation in the terminal — Claude Code for agentic coding, aider for pair programming, the llm CLI for pipelines, fabric for pattern-based text processing, and how to integrate LLM output into shell workflows without creating new problems.
-
Building an AI-Augmented Knowledge Work Workflow A practical guide to integrating LLMs into knowledge work without creating new problems — prompt patterns that work, task-to-tool matching, managing hallucination risk, context window strategies, and honest trade-offs for writing, research, and code review workflows.
-
Local LLMs with Ollama Running large language models locally with Ollama — VRAM requirements by tier, quantization formats explained, model selection for different use cases, Open WebUI setup, modelfiles for custom system prompts, and an honest comparison of when local beats cloud and when it doesn't.
-
Building a Local RAG Pipeline: From Documents to Answers A practical guide to building a fully local Retrieval-Augmented Generation pipeline: document ingestion, chunking strategies, embedding model selection, vector store options (ChromaDB, Qdrant, FAISS), reranking, and wiring it all together with LangChain and Ollama — no cloud APIs required.
-
AI Code Review Bots: Building One Engineers Don't Mute What separates a review bot engineers actually act on from one they silence on day three: context selection, taste calibration, blocking vs. advisory mode, and the metrics that tell you which you've built.
-
Constrained Generation: Outlines, JSON Mode, and Structured Output That Works How regex-constrained sampling and grammar-guided decoding actually work at the token level, why prompt-only JSON mode fails 5–20% of the time in production, and the full toolchain for guaranteed structured output: Outlines, XGrammar, vLLM guided decoding, and Instructor.
-
LiteLLM and Model Routing: The Proxy Pattern for Multi-Provider LLM Apps One OpenAI-compatible endpoint, many backends. How LiteLLM's proxy pattern works, how to configure routing, fallbacks, cost controls, and observability, and when a self-hosted gateway beats managed alternatives.
-
LLM Evals: Testing Your AI Application Like Real Software Writing evaluations that catch regressions before they reach users: golden datasets, LLM-as-judge pitfalls, CI integration, and the tools that make a sustainable eval pipeline — Braintrust, Langfuse, Promptfoo, Inspect, and DeepEval compared.
-
MCP Deep Dive: Servers, Resources, and Tools A thorough technical walkthrough of the Model Context Protocol: how JSON-RPC over stdio and HTTP works, building servers with FastMCP and the TypeScript SDK, implementing tools and resources, the sampling primitive, security threat model, and production deployment patterns.
-
Mixture of Experts Internals: Routing, Expert Parallelism, and Load Balancing MoE is now the default architecture for top-tier open models. This is how it actually works: router design, top-k token dispatch, auxiliary loss for load balancing, expert parallelism across GPUs, and what it means for inference serving — with DeepSeek-V3 and Mixtral as concrete case studies.
-
Quantization Deep Dive: GPTQ, AWQ, GGUF, AQLM, and MLX A rigorous look at every major LLM quantization format: how GPTQ's Hessian-guided rounding works, why AWQ's activation-aware scaling beats naive per-channel approaches, when GGUF k-quants are the right choice, where AQLM wins at sub-3-bit, and how MLX handles Apple Silicon. Includes conversion workflows, a perplexity shootout, and a decision guide.
-
Semantic Caching for LLM Applications: Cutting Cost and Latency How embedding-based semantic caching works, why threshold tuning is the hardest part, cache invalidation patterns that prevent staleness, and practical implementations with GPTCache, LiteLLM, and a from-scratch Redis + FastAPI setup.
-
Speculative Decoding: Draft Models, EAGLE, and How to Actually Use It How speculative decoding achieves 2–4x inference speedup without changing model outputs, covering the rejection-sampling proof, EAGLE and EAGLE2 draft strategies, ngram lookahead, and production configuration for vLLM and llama.cpp.
-
vLLM vs SGLang vs TensorRT-LLM: Inference Engine Shootout 2026 A thorough comparison of the three dominant LLM inference engines: how each one works internally, where each wins on benchmarks, and which to reach for given your hardware, workload, and operational tolerance.
-
Llama 4 and Gemma 4: The 2026 Self-Hosted Model Landscape Meta shipped Llama 4 Scout and Maverick and Google shipped four Gemma 4 variants under Apache 2.0 — all within a few weeks of each other. This is a practical walkthrough of what each model is, what hardware you actually need, how to deploy them with Ollama, vLLM, and llama.cpp, and which one to pick for which job.
-
Building AI Agents That Actually Work A practical engineering guide to building reliable AI agents — tool use, ReAct loops, structured output, memory patterns, multi-agent systems, and the failure modes nobody warns you about.
-
Fine-Tuning LLMs on Your Own Hardware LoRA and QLoRA explained, unsloth for efficient fine-tuning, dataset preparation, evaluation, merging adapters, and when fine-tuning actually beats RAG — a practical guide to training language models on consumer and prosumer hardware.
-
Google's TurboQuant: What It Means for Home AI Enthusiasts Google Research's TurboQuant compresses LLM KV cache memory by 4–5x with minimal accuracy loss. Two weeks in: real benchmarks, active framework integrations, controversies, and what you can actually run today.
-
Self-Hosted AI Inference: vLLM, llama.cpp, and Running Your Own OpenAI-Compatible API A comprehensive guide to running your own LLM inference servers — covering vLLM, llama.cpp, OpenAI-compatible APIs, batching strategies, quantization, and benchmarking throughput so you can make informed hardware and software decisions.
-
Fine-Tuning Small Models: When and Why to Fine-Tune vs. Prompt Engineer A practical guide to fine-tuning small language models — understanding when fine-tuning beats prompt engineering, how LoRA and QLoRA work, training a model on your own data with Unsloth or Axolotl, and deploying the result with Ollama.
-
Local LLM Deep Dive: Ollama, Quantization, and Running AI on Your Own Hardware A comprehensive technical guide to running large language models on your own hardware — covering Ollama setup, quantization formats, hardware selection, Apple Silicon and NVIDIA GPU configuration, and building a private local coding assistant.
-
RAG from Scratch: Building a Retrieval-Augmented Chatbot Over Your Own Documents A complete technical walkthrough of building a retrieval-augmented generation (RAG) system from scratch — covering embeddings, vector databases, chunking strategy, retrieval quality, and a fully runnable Python implementation using Ollama and ChromaDB.
-
Local LLM Inference for Coding: The Complete 2025/2026 Guide Everything you need to run powerful AI coding assistants locally — model benchmarks, Ollama, LM Studio, llama.cpp, hardware requirements, and editor integration with Continue.dev and Aider.