Recent discussion on r/LocalLLaMA focuses on practical tips for running local LLaMA models well, particularly for always-on agent workflows.

On performance, community members recommend persistent KV caches or SSD-backed checkpoints to avoid re-evaluating repeated context — one report cites CachyLLama adding persistent SSD-backed KV checkpoints plus a system-prompt cache. Speculative decoding is highlighted as a major win: a test of llama.cpp's methods on Qwen 3.6 27B measured roughly 2.7x speedup with MTP, 3.7x with DFlash, and about 6x with an n-gram stack on real coding workloads. Linux is preferred over Windows, with one comparison showing 5:25 versus 12:57 wall-clock time for the same task.

On hardware, the advice is to match VRAM to the chosen quant and context length, since overflowing into system RAM causes slowdowns. For multi-GPU across machines, 10GbE helps cold loads while RDMA gives the biggest gains in large spanning setups; one user reported roughly 2000-2800 prompt-processing tokens per second and 88 tokens per second generation over pooled all-GPU RPC, tripling decode because keeping every expert in VRAM outweighs the network penalty. Consumer builds rarely exceed four cards per chassis anyway.

For model and runtime choices, users suggest quants that fit VRAM for the target context (e.g. Qwen3.6-27B-MTP at Q4_K_XL used with VS Code Copilot), q8_0 KV caches, mmproj offload, and --cache-ram (48GB reportedly sufficed for one Qwen3.6 27B workload). Flags like --flash-attn, --cache-type-k q8_0 and --model-draft are missed when using LM Studio's pinned older llama.cpp build.

Operationally, the thread stresses starting small with agent integrations, noting that API integration (e.g. G-suite) is often the real bottleneck rather than the LLM. Local hosting is framed as a tradeoff chosen for privacy, control, or cost — one user reported about $80/month in extra electricity for two old Tesla cards running 24/7 — and not as a drop-in replacement for frontier cloud models.