The honest DIY guide
Here is the entire job, accurately, with the trap in every step — the same traps in our research files. If reading this list sounds like a good month, you should genuinely DIY, and this page will save you real time. If it sounds like someone else's job: that is the job we sell.
Read licences, not just leaderboards: one of this summer's hottest video models excludes the EU, UK and US by name. Then dodge the MoE trap — 'active parameters' is a compute figure, so a 2.8T model with 104B active still needs ~1.4TB resident. The scoreboard favourites mostly do not fit.
The trap: Vendor benchmark charts routinely include API-only models that have no weights to download at all.
days of reading, or minutes with a verified library
For the flagship alone there are 38 GGUF builds from 7 publishers. Quality-per-GB data below Q4 is mostly unpublished; labels drift between publishers (one popular analysis mislabels a 104GB IQ3 as an IQ2); and the wrong pick either refuses to load or quietly degrades.
The trap: The optional 22GB speculative-decode module pushes an otherwise-fitting quant past the usable pool — it OOMs only under load.
an evening of forum archaeology per model
Hugging Face auth tokens, resume-safe transfers, checksums, and a disk layout that survives the next model. At 100GB+ per model a failed download at 94% is a rite of passage.
The trap: The default download location fills the OS partition, not the 4TB data volume.
hours, bandwidth-dependent
GB10 is sm_121a, newer than most toolchains expect. llama.cpp wants CUDA build flags set for the exact architecture. vLLM's stock pip install fails outright — bundled PyTorch compiles through sm_120 only — so it is containers with pinned digests, and FlashInfer JIT needs CUDA ≥ 12.9 or it dies at runtime. SGLang support is a workaround container with an open tracking issue.
The trap: Everything installs cleanly on x86 tutorials and fails here — the guides you will find were written for other machines.
a weekend the first time
128GB on the box, ~119–121GB actually allocatable after OS and driver reserve. Weights + KV cache + framework overhead must fit inside it: a 104GB quant leaves ~15GB for context, which is the difference between 131K tokens and an OOM crash mid-conversation. Some models need --no-mmap on unified memory to load at all.
The trap: It loads fine, runs fine, and falls over an hour later when the KV cache grows into the reserve.
iterative — each model again
Agents need structured tool calls; local models often emit them as raw JSON or XML text instead — the agent framework's own docs say so. The fix is per-model chat-template surgery, and until it is done your agents narrate what they would have done rather than doing it.
The trap: Chat works perfectly, so everything looks fine — until the first agent run produces prose instead of actions.
hours per model, once you know it is the template
Each app has its own local-endpoint dialect: one needs OLLAMA_CONTEXT_LENGTH raised past its too-small default, another wants an Anthropic-compatible server of a minimum version, a third needs the context window declared by hand. Multiply by every tool your team uses.
The trap: Defaults are tuned for cloud APIs — locally they truncate context silently and answers just get mysteriously worse.
an afternoon per app
Model server, agent runtime and chat app as supervised services: restart policies, logs somewhere findable, load order, update strategy that will not brick a working setup. The difference between a demo and infrastructure.
The trap: It ran for three weeks, the power blipped, and nothing comes back because nothing was a service.
a day to do properly
Multi-user auth on the chat app, the API bound away from 0.0.0.0, remote access via your own VPN rather than a port forward. The machine's whole point is privacy — an open port defeats it from the LAN side.
The trap: Default binds listen on all interfaces; scanning for open inference endpoints is already a hobby.
half a day
Sending the right people away is why the rest trust us.
If you have built CUDA toolchains before, enjoy this kind of work, and the month is yours to spend — DIY is real and the ecosystem is open. Start with NVIDIA’s official DGX Spark playbooks for llama.cpp, Open WebUI and ComfyUI, and Unsloth’s quant documentation. Everything we install is open source; there is no lock-in to us, ever — which is also true when we do it for you.
€2,000 per box buys every step on this page — models chosen and quantised for your work, the server built for this silicon, tool calling fixed, apps wired, services supervised, security closed, your team walked through it. The machine arrives working.
