System Prompt

You've seen what goes into an AI agent. Now what does it take to actually configure one and give it a job?In Episode 31 of System Prompt, Peter and Val move from the architecture of Hermes Agent to a hands-on walkthrough of its Windows desktop interface. With Hermes already installed and connected to DeepSeek V4.1 Flash running locally across two NVIDIA DGX Sparks, they explore the settings that shape an agent's behavior, access, memory, and execution.They work through model selection and fallbacks, chat personalities, workspace configuration, approvals, file checkpoints, context compression, providers, tool integrations, and plugins. Not every feature needs changing, and not every experiment works as expected. That's part of the demonstration: no prearranged happy path.Then they give Hermes a real task: create a scheduled 8 a.m. AI news briefing, add sentiment analysis, inspect sources and output, and request a more polished PDF format. The conversation expands into what makes an automation useful—from daily operational reporting and ad analytics to finding RFPs and preparing proposals. The question isn't just what an agent can do. It's which steps in your existing workflow are worth handing off.• Configuring Hermes through the Windows GUI• Primary models, reasoning settings, fallbacks, and auxiliary models• Personalities, attachments, interface settings, and workspaces• Command approvals, secret redaction, file checkpoints, and browser access• Persistent memory and context compression• Agent step limits, subagents, and local versus cloud providers• Search tools, custom integrations, MCP, skills, and plugins• Testing web access with a locally hosted model• Creating an automated AI news brief with sentiment analysis• Applying agents to real business workflowsKEY TAKEAWAYSCONFIGURATION IS PART OF AGENT ENGINEERINGChoosing a model is only the beginning. Workspace access, approval rules, memory, tools, and execution limits determine how an agent operates.LOCAL DOESN'T MEAN UNRESTRICTEDA locally hosted model can use web tools and run workflows, but access to browsers, files, credentials, and commands still needs deliberate controls.YOU DON'T NEED TO CHANGE EVERY SETTINGSome defaults are fine. Understand what each setting governs, then adjust it based on the task and the risk.A WORKING DEMO ISN'T A FINISHED EVALUATIONHermes creates a schedule and generates an initial briefing, but repeatability, source quality, and output consistency still need testing.START WITH YOUR WORKFLOW, NOT THE TOOLLook at what you retrieve, compare, update, email, and report each day. The valuable agent is the one that removes real work from that process.CHAPTERS00:00 From AI Infrastructure to a Working Agent02:29 Models, Reasoning, and Fallbacks03:30 Chat Personalities and Attachments05:39 Appearance and Desktop Preferences07:14 Workspaces, Approvals, and File Safety08:44 Memory, Compression, and Voice10:51 Advanced Settings and Agent Limits12:00 Providers, Local Models, and Gateways14:00 Tool Integrations, Plugins, and Skills16:55 Can Hermes Build Its Own Tools?18:24 Testing Web Access19:20 What Should You Actually Do With an Agent?21:12 Building a Scheduled AI News Brief28:18 Adding Sentiment Analysis30:58 Agent Personalities and Human Interaction33:45 Reviewing the Briefing and Sources37:20 Requesting a PDF Output39:01 RFPs and Business Automation Opportunities41:59 Finding Workflows Worth Automating47:05 Wrap-Up and What's Next

READ THE FULL EPISODE PAGE: https://devmesh.tech/podcast/anatomy-of-an-ai-agentAn AI agent isn't just a model with tools. What actually happens between your prompt and a completed task, and what keeps the system from falling apart when something goes wrong?In Episode 30 of System Prompt, Peter and Val take apart the Hermes Agent harness to examine the software engineering behind an AI agent. Building on their episodes about AI hardware and inference runtimes, they follow the agent loop from context assembly and model calls through tool execution, persistence, security, recovery, and observability. Hermes is the example, not the endorsement: the point is to show where probabilistic model decisions end and deterministic software takes over.They explore why switching model providers involves more than changing an API endpoint, why a tool call is a request rather than an action, and how seemingly ordinary components like SQLite, file-based memory, and approval rules support a sophisticated system. Along the way, they discuss evaluation, concurrency, prompt caching, local execution risks, and the trade-offs that come with every architecture.The agent loop may be simple. Making it durable, safe, and useful against real systems is the engineering challenge.• The model versus the harness: who decides what• The core agent loop and turn management• Context construction, caching, and token costs• Provider normalization across APIs and local models• Tool registries, validation, approvals, and execution• Persist-before-execute and crash recovery• Iteration limits, errors, and fallback behavior• Memory, skills, and what deserves to be remembered• Security boundaries and dangerous-command detection• SQLite, concurrency, observability, and evaluations• Hermes' strengths, limitations, and engineering trade-offsKEY TAKEAWAYSTHE MODEL REASONS. THE HARNESS GOVERNS.The model proposes tools, arguments, and responses. The harness determines what can run, where it runs, what gets persisted, and which actions require approval.A TOOL CALL IS NOT EXECUTIONThe model can request an action, but deterministic code validates the request and enforces permissions before a tool actually does anything.PROVIDER COMPATIBILITY IS MORE THAN AN ENDPOINTReasoning formats, message structures, streaming, and error handling differ. Normalizing those differences is substantial engineering work.RECOVERY IS BUILT FROM ORDINARY SOFTWARESession leases, database writes, iteration controls, and error handling make a simple model/tool loop survivable when processes crash or requests fail.SECURITY IS AN ENGINEERED BOUNDARYInstructions alone aren't permission controls. Approval states, restricted execution environments, and configuration determine what an agent can actually do.EVERY ARCHITECTURE HAS TRADE-OFFSHermes uses practical building blocks, but memory growth, provider complexity, local permissions, and SQLite concurrency still require deliberate choices.CHAPTERS00:00 Intro: Anatomy of an AI Agent04:15 Why Is the Hermes Repository So Large?06:33 Agent Loop, Turn State, and Context Assembly10:47 Specialized Harnesses and Context Costs12:35 Provider Abstraction and Tool Registries17:46 Who Actually Executes a Tool Call?19:41 Approvals, Failures, and Evaluations23:48 Recovery, Memory, and Skills27:35 Deterministic Security and Execution Boundaries31:21 Concurrency, SQLite, and Observability36:25 Testing, Prompt Injection, and Memory Decisions40:43 What the Model Decides vs. What the Harness Decides45:24 Hermes Limitations and Engineering Trade-Offs50:32 Final Takeaways: What Makes an Agent a System52:02 Wrap-Up and What's Next

READ THE FULL EPISODE PAGE: https://devmesh.tech/podcast/saas-vs-ai-part-2-revenge-of-the-sweSaaS isn't dead. But what happens when the people who know a business best can build the software they need themselves?In Episode 29 of System Prompt, Peter and Val return to SaaS vs. AI for an unscripted sequel: Revenge of the SWE. They debate whether agents become interfaces to existing SaaS products or replace some of those products entirely. From an accountant building tools with Claude Code and SQLite to small businesses recreating only the features they use, the question isn't simply whether AI can write code. It's who understands the process, validates the outcome, and maintains the result.The conversation explores software engineering, domain expertise, deterministic workflows, harnesses, MCP, AI operations, local inference, and the challenges of organizational data and persistent memory. They also discuss frontier subscriptions versus local models, AI devices, and why AI-generated answers still need scrutiny.This is a riff, not a prediction that every SaaS company disappears. The cost of code is changing. The cost of judgment is another story.• SaaS as the engine behind agents, or something agents replace• Domain experts building their own software• Claude Code, SQLite, and small internal applications• CRUD applications and custom alternatives• Engineering beyond writing application code• Deterministic workflows, validation, and TDD• Harnesses, MCP tools, and specialized agents• AI Ops: runtimes, hosting, access, and reliability• Local inference economics and AI hardware• Data quality, context, memory, and relevance• Questioning AI-generated plans and answersKEY TAKEAWAYSTHE AGENT MAY REPLACE THE INTERFACE OR THE APPLICATIONVal sees agents operating SaaS through integrations. Peter asks when a business can skip a product and build only what it needs. Both may happen, depending on the workflow.DOMAIN EXPERTISE CHANGES THE EQUATIONAn accountant or operations lead knows the requirements and exceptions. AI can lower the implementation barrier, but verifying the result still matters.THE COST OF CODE HAS COLLAPSED. THE PRICE OF JUDGMENT HAS NOT.Generating software is easier. Deciding what should exist, whether it works, who can access it, and how to maintain it still requires engineering judgment.AI OPS IS STILL SOFTWARE ENGINEERINGSomeone must own runtimes, harnesses, integrations, memory, evaluations, and infrastructure, whether inference is local or hosted.MEMORY IS NOT JUST STORAGESaving information is one thing. Knowing what is relevant, outdated, or worth retrieving for a particular task is harder.CHAPTERS00:00 Intro and an Off-the-Cuff Episode01:05 Anthropic, AI Sentience, and the Pope03:20 SaaS vs. AI: The Debate Returns08:17 When the Accountant Becomes the Builder11:41 Does AI Change the Need for UI/UX?14:00 Deterministic Paths and AI Costs17:36 The Value of Domain Expertise19:02 Local Models and the Future of Copilot22:18 Harnesses and Specialized Agents26:16 AI Devices and Distributed Inference30:37 AI Ops, Data Quality, and Validation34:03 AI Gadgets Versus Your Phone37:37 Small Businesses and Bespoke Software41:36 CRUD, Rails, and SaaS Alternatives45:40 The Cost of Code vs. the Price of Judgment48:41 Have We Solved AI Memory?53:24 SaaS, Copilot, and the Cost of AI58:28 Local Models and What's Next1:01:01 Closing Thoughts

READ THE FULL EPISODE PAGE :https://devmesh.tech/podcast/youve-bought-ai-hardware-now-whatBuying AI hardware is only the beginning.In Episode 28 of System Prompt, Peter and Val follow up on Before You Buy AI Hardware, Watch This by moving from planning into the actual hosting layer. Using GLM-5.3-Flash across two NVIDIA DGX Sparks with vLLM, they break down what the serving command is doing, which settings actually matter, and why hosting a model is very different from simply loading one. The conversation covers tensor parallelism, distributed execution, RoCE and NCCL networking, Docker, model mounts, KV cache, context length, concurrency, Mixture-of-Experts execution, chat templates, reasoning and tool-call parsers, DFlash speculative decoding, prefill, decode, and live serving metrics.Then they start the real cluster, bring the worker online before the head node, watch the model load across both Sparks, and run GLM-5.3-Flash through a coding workload. The live metrics show speculative decoding acceptance changing with the task, with generation throughput moving from the teens into the 30s and 40s as the workload becomes more predictable.• What happens after you buy AI hardware• Hosting GLM-5.3-Flash across two DGX Sparks• vLLM and distributed inference• Tensor parallelism across multiple devices• RoCE, NCCL, and cluster communication• Docker, model mounts, and runtime configuration• Context windows and KV cache• FP8 KV cache vs. FP16• Concurrency and max active sequences• Mixture-of-Experts execution• Chat templates, reasoning parsers, and tool-call parsers• DFlash speculative decoding• Prefill, decode, and generation throughput• Why real workloads have to be testedQUICK CORRECTIONAt one point I say the ConnectX-7 link is 200 gigabytes per second. I meant 200 gigabits per second, or 200 Gb/s. Yes, I know the difference. It was a live recording, I misspoke, and I am correcting it here.KEY TAKEAWAYSRUNNING A MODEL IS NOT THE SAME AS HOSTING ONELoading the weights is step one. Hosting adds networking, scheduling, memory management, concurrency, cache behavior, runtime compatibility, and observability.THE MODEL IS ONLY ONE PART OF THE STACKRuntime, quantization, KV cache, speculative decoding, context limits, scheduling, and hardware topology all affect system behavior.YOU DO NOT NEED TO ENGINEER EVERY LAYER YOURSELFStart with a known-good path, understand the major knobs, test it on your hardware, and go deeper only where the workload proves you need to.SPECULATIVE DECODING ONLY HELPS WHEN THE DRAFT IS RIGHTWhen DFlash acceptance is low, speculative work can become overhead. When the workload becomes more predictable, acceptance and generation throughput can rise significantly.THE WORKLOAD DETERMINES THE CONFIGURATIONThere is no universal best setup. Context, concurrency, precision, cache size, and speculative decoding all need to be tested against the actual task.CHAPTERS00:00 Intro and the Follow-Up03:23 Two DGX Sparks, vLLM, and GLM-5.3-Flash07:00 Breaking Down the Hosting Command11:00 Model Mounts, DFlash, and Runtime Patches14:12 NCCL, RoCE, and Distributed Networking17:30 API Behavior, Tool Calls, Reasoning, and Chat Templates21:40 The vLLM Knobs You Actually Tune25:45 FP8 KV Cache and Memory Tradeoffs27:30 DFlash and Speculative Decoding31:38 Starting the Two-Node Cluster41:51 Running GLM-5.3-Flash45:56 Reading the Live Metrics50:17 Why You Have to Test54:22 Coding Workloads and Higher Draft Acceptance59:37 What This Means for Businesses1:04:01 Closing Thoughts

READ THE FULL EPISODE PAGEhttps://devmesh.tech/podcast/before-you-buy-ai-hardware-watch-thisMost businesses do not need more AI hardware. They need a better understanding of the workload they are actually trying to run.In Episode 27 of System Prompt, Peter and Val break down inference runtime, the layer that sits between the model and the hardware serving it, and why runtime decisions matter before a business starts buying GPUs, DGX systems, or other local AI infrastructure.The conversation covers Ollama, llama.cpp, vLLM, SGLang, quantization, context windows, KV cache, prefill, decoding, batching, speculative decoding, and parallelism. But the bigger question is business capacity. How many users will actually be active at once? Which departments need the most inference? When does demand spike? What should run locally, and what should stay in the cloud?Using examples from a 150-person company to enterprise GPU deployments, Peter and Val argue that infrastructure should be sized from real usage data, not assumptions. A business may not need 150 concurrent sessions just because it has 150 employees, and it may not need more hardware when better scheduling can solve the problem.• What inference runtime actually is• Ollama, llama.cpp, vLLM, and SGLang• Why concurrency should be measured instead of guessed• How quantization affects memory, performance, and quality• Context windows, KV cache, prefill, and time to first token• Why continuous batching matters• Speculative decoding and runtime optimization• Local devices vs. shared inference• Why expensive hardware can sit mostly idle• How businesses should pilot before scalingKEY TAKEAWAYSAN UNTESTED CONFIGURATION IS WINDOW DRESSINGIf a model, runtime, quantization, and hardware configuration has not been tested against the real workflow, it is still an assumption. Production decisions should be based on measured behavior.HEADCOUNT IS NOT CONCURRENCYA 150-person company does not automatically need infrastructure for 150 simultaneous sessions. Real usage depends on departments, workflows, working hours, and demand patterns.RUNTIME CHANGES HOW HARDWARE GETS USEDBatching, caching, scheduling, memory management, and parallelism all affect how efficiently requests turn into useful GPU work.START SMALL AND COLLECT REAL DATAPilot with one department or workload, measure throughput and bottlenecks, then scale from evidence instead of buying for theoretical maximums.IDLE CAPACITY IS STILL WASTED MONEYA powerful GPU cluster does not create ROI by existing. If expensive accelerators sit idle most of the day, the infrastructure may be oversized or poorly scheduled.THE BUSINESS PROBLEM COMES FIRSTBefore choosing a model, runtime, or piece of hardware, define what the system needs to accomplish, how often it will be used, and what level of performance the workflow actually requires.CHAPTERS00:00 Intro and 20,000 Views01:51 Why Businesses Are Exploring Local AI03:25 What Inference Runtime Actually Is06:51 Real Concurrency vs. Headcount10:29 llama.cpp, GGUF, and Quantization14:13 An Untested Configuration Is Window Dressing15:22 Context and KV Cache17:06 vLLM and Continuous Batching18:28 Prefill, Decode, and Time to First Token21:12 Prefix Caching and Runtime Optimization25:17 Speculative Decoding29:05 Engineering for Capacity30:47 Local Devices vs. Shared Infrastructure35:09 When Local Inference Makes Sense39:53 Idle Hardware and Wasted Spend48:10 Scheduling Inference by Department49:40 The Full AI Infrastructure Stack