The Private AI Lab

I packed The Private AI Lab into a suitcase and took it to NVIDIA GTC Europe in Berlin. Six conversations about what it takes to run AI on your own hardware.You'll hear:– GTC highlights with ITQ colleague Florian Casse (01:34)– SUSE AI Factory, quantum-ready AI and secure agents with Sanjeet Singh (05:50)– SLES for NVIDIA with Miguel Pérez Colino (16:44)– AI security on the show floor with Luke Wignall (23:23)– Building an agent with wooden blocks: Nemotron 3.5 Bolt and Switchyard on a DGX Spark, with Luke Wignall & Matthew Feinberg (33:52)– Post-training Nemotron for your own domain, with Luke Wignall & Amelia (43:34)– NemoClaw & OpenShell at Build-a-Claw with Mark Heaps (51:27)Read more: https://johan.mlBlog on OpenClaw, NemoClaw & OpenShell: https://johan.ml/from-yolo-ops-to-governed-agents-what-openclaw-nemoclaw-and-openshell-mean-for-the-rest-of-us/The Private AI Lab: experimenting, exploring and debating the future of AI behind the firewall. Hosted by Johan van Amersfoort (ITQ).

NVIDIA just shipped a model with four times the context window of what's been running my home agent for months. So I swapped it in, broke my own setup within the first hour, and then threw 700,000 words at it to see if it would lie to me.No guest this episode, just me and my home lab. Jarvis, my home agent, has been running on Qwen3.6-35B-A3B since the start, served through vLLM on the DGX Spark's 128GB of unified memory. Then NVIDIA released Nemotron 3.5 Lightning 30B-A3B in NVFP4, and one number stopped me mid-scroll: a validated 1 million token context window, four times what Qwen gives you at 262,000.I don't trust a spec sheet claim until I've broken it myself, so I swapped the model into Jarvis, found a memory reservation bug in NVIDIA's own deployment recipe within the first hour, fixed it, and then buried a fabricated secret code word 90% of the way through a 700,000 word document to see if the model could actually recall it. I also dug into DSpark, the speculative decoding checkpoint built specifically for this hardware, and measured what it actually does for throughput.This episode is the honest version: what's proven on my own hardware, what's still sitting on NVIDIA's spec sheet, and what I still don't know about whether Nemotron makes Jarvis a better agent day to day.Key topics covered:Why Jarvis's setup (Qwen3.6-35B-A3B on vLLM) needed an indirection layer before any of this matteredNVIDIA Nemotron 3.5 Lightning 30B-A3B-NVFP4's validated 1 million token context window, tested against a 700,000-word documentThe memory reservation bug in NVIDIA's DGX Spark deployment recipe, and the one-line fixDSpark speculative decoding: how it works, and the acceptance rate that made it worth usingNemotron's hybrid Mamba-2, attention, and mixture-of-experts architecture, and why it matters for running 1M tokens on a single boxWhat's proven on real hardware versus what's still just NVIDIA's own benchmark numbersWhy Nemotron isn't confirmed as the better agent brain for Jarvis yetChapters00:00 Introduction — what a 1M-token context window actually changes00:37 Welcome back to The Private AI Lab (no guest this episode)01:19 Recap: Jarvis running on Qwen3.6-35B-A3B via vLLM on the DGX Spark01:58 NVIDIA Nemotron 3.5 Lightning lands with a validated 1M-token window02:49 Deployment: why the Jarvis-LLM alias made the model swap painless03:28 The memory reservation bug NVIDIA's own recipe doesn't warn about04:53 The test: a 700,000-word document with one hidden code word05:51 DSpark speculative decoding: free speed, fully verified06:47 The reasoning quirk that quietly eats your token budget07:31 Nemotron vs Qwen: what's proven versus what's still a spec sheet claim08:01 The hybrid architecture behind why 1M tokens fits on a DGX Spark09:24 NVIDIA's own SWE-bench Verified numbers, and their limits09:56 Closing thoughts, and what's coming in the next episodeLinks mentioned in this episode:NVIDIA Nemotron 3.5 Lightning 30B-A3B-NVFP4 on Hugging Face: https://huggingface.co/nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-NVFP4NVIDIA Nemotron 3.5 Lightning 30B-A3B-NVFP4-DSpark (speculative decoding checkpoint): https://huggingface.co/nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-NVFP4-DSparkQwen3.5-35B-A3B on Hugging Face: https://huggingface.co/Qwen/Qwen3.5-35B-A3BNVIDIA DGX Spark: https://www.nvidia.com/en-us/products/workstations/dgx-spark/vLLM: https://github.com/vllm-project/vllm

Introducing the NVIDIA Nemotron 3 Embed Family

Most enterprise AI projects succeed as pilots. They fail on the way to production. In this episode, Rhys Oxenham, VP and General Manager of AI at SUSE, joins me to break down what SUSE announced at SUSECon 2026: SUSE AI Factory with NVIDIA. We go deep on what it actually is, what problem it solves, and what happens when AI starts managing the infrastructure itself. We cover the assembly line concept, validated blueprints, why this isn't just for large enterprises, unified support across the full stack, digital sovereignty as resilience, and the very real governance challenge when agents start acting autonomously in your cluster, what I've been calling "YOLO ops." Release is July 2026. This is worth understanding before then.🕐 CHAPTERS 00:00 — Intro01:15 — Who is Rhys Oxenham and what happened at SUSECon 202603:19 — What is SUSE AI Factory with NVIDIA?07:16 — The assembly line: what moves through it and where it starts and ends10:24 — Blueprints explained: RAG, digital assistants, research agents12:03 — ClickOps vs GitOps: who is each mode for?15:10 — Scale: from single node to thousands of GPUs17:29 — DGX Spark compatibility (teaser)19:34 — Open source + enterprise support: one throat to choke21:34 — Day-two operations: what does running it actually look like?25:54 — Digital sovereignty: open source + NVIDIA — where does the openness live?30:14 — MCP servers, agentic AI, and integrations with N8N and others34:00 — YOLO ops: how do you govern autonomous agents in your cluster?38:32 — Model selection: which model for which use case?44:16 — AI ops blueprints: what's coming after launch46:03 — Release date and what to expect in July 202648:56 — Wrap up and where to follow Rhys 🔗 LINKS SUSE AI Factory with NVIDIA → https://www.suse.comRhys Oxenham on LinkedIn → https://www.linkedin.com/in/rhys-oxenham/Johan on LinkedIn → https://www.linkedin.com/in/hojanThe Private AI Lab newsletter → https://www.linkedin.com/newsletters/the-private-ai-lab-7381951883810111489 🎙️ ABOUT THE GUEST Rhys Oxenham is VP and General Manager of AI at SUSE, where he leads all AI product strategy and execution. He came up through solution architecture and field engineering before running SUSE's Edge and Telco Engineering groups, deploying infrastructure in air-gapped, industrial, and tactical environments. He keynoted SUSECon 2026 in Prague to announce the SUSE AI Factory with NVIDIA partnership. 🔔 SUBSCRIBE for weekly episodes on private AI, self-hosted infrastructure, and what it actually takes to run AI inside your organisation. 📨 Follow The Private AI Lab newsletter on LinkedIn for episode breakdowns and experiment write-ups.

Johan breaks down NVIDIA’s ComputeEx 2026 announcement of Nemotron 3 Ultra 550B-A 55B, an open-weights mixture-of-experts model with 550B total parameters and 55B active, positioned as an orchestration “agent brain” for multi-step tasks behind the firewall. He reviews NVIDIA’s benchmarks versus GLM 5.1, Kimi K 2.6, and Qwen 3.5, highlighting best-in-class instruction following (82%), long-context performance (95%) with a 1M-token window, strong agent productivity (91%), and weaker coding results on TerminalBench versus Kimi. Johan emphasizes reported advantages in speed (~300 tokens/sec, ~5x faster), cost (up to ~30% cheaper on SWE-bench tests), and deployability via a unified NVFP4 checkpoint optimized for H100 and B200 GPUs, plus NemoClaw as the agent blueprint. He closes with an early-access demo comparing two agents researching Netherlands’ 2026 World Cup odds, showing Nemotron’s more granular path analysis and a 5.8% win estimate.00:00 Private AI Lab Intro01:19 Nemotron Ultra Explained02:22 Agent Brain Focus03:07 Benchmark Reality Check05:14 Speed And Cost Edge06:11 Training And Precision08:02 NeMo Claw Agents08:58 World Cup Agent Demo12:22 Why This Matters13:17 Wrap Up And Links