DiscoverThe Innovation Attorney Podcast
The Innovation Attorney Podcast
Claim Ownership

The Innovation Attorney Podcast

Author: The Innovation Attorney

Subscribed: 0Played: 2
Share

Description

The Innovation Attorney is a publication about the intersection of venture capital, technology, public policy and law.

theinnovationattorney.substack.com
274 Episodes
Reverse
Salesforce, Zendesk, and HubSpot shifted AI pricing toward usage based and outcome based models during 2025 and 2026 because seat based subscriptions could not track token costs or delivered value. A venture funded AI product should move to usage based or hybrid pricing when its inference cost per customer varies by more than roughly three to one across the user base, the threshold several pricing operators use to flag margin risk under flat seat pricing. Salesforce priced Agentforce at $2.00 per conversation at its October 2024 launch, then layered in $0.10 per action Flex Credits in May 2025, and still sells seats starting at $125 per user per month under its Agentic Enterprise License Agreement, because no single model fits every buyer. The right answer depends on gross margin at the ninetieth percentile user, deal size, and how predictably the company can forecast consumption, not on which model draws the most attention.The underlying driver is a token cost collapse that moved faster than most pricing committees tracked it. Input token prices for frontier grade language models fell from $30 per million tokens for GPT-4 in March 2023 to roughly $0.10 per million tokens for comparable capability by early 2026, a decline close to three hundred times in under three years. Anthropic cut its own 2025 gross margin forecast to about 40 percent as inference costs outran its revenue growth, even as 2025 revenue reached approximately $10 billion. A company billing a flat $29 monthly seat against a ninetieth percentile user consuming $21.00 in tokens has already lost money on that account before any other cost is counted.The comparative evidence favors a hybrid structure over either pure model standing alone. Vertice’s second quarter 2026 contract data placed usage based pricing as the single most common primary model at 36.5 percent of software contracts, ahead of per user pricing at 32.4 percent, while L.E.K. Consulting’s April 2025 valuation analysis found hybrid pricing companies carrying a 6.9x mean revenue multiple against 6.5x for pure subscription companies. For an investor, the practical implication is that the pricing model a portfolio company selects in 2026 shows up directly in gross margin, net revenue retention, and the multiple the company commands at its next financing, before any product feature does.Definition, Scope, and Technical ContextUsage based pricing charges a customer for the metered quantity of a resource consumed, most commonly the input and output tokens processed by a large language model, while seat based pricing charges a fixed fee per named user regardless of how much of the product that user consumes. This analysis treats outcome based pricing as a third category: a charge triggered by a completed result, such as HubSpot’s $0.50 fee per resolved Breeze Customer Agent conversation after April 14, 2026, or Salesforce’s $2.00 per Agentforce conversation at its October 2024 launch. The analysis covers venture funded, cloud delivered AI products sold to businesses, meaning software companies that purchase frontier model access through an application programming interface rather than companies that train and sell their own foundation models.The analysis excludes consumer facing subscription products such as a standalone chatbot application, because consumer willingness to pay follows different psychology than enterprise procurement, and it excludes open source, self hosted deployments, where the customer bears the inference cost directly and the vendor pricing question does not arise in the same form.The relevant engineering classification is metered cloud services pricing applied to a workload whose unit cost is itself falling rapidly and whose consumption per user varies far more than consumption of a conventional software seat. A seat on a conventional SaaS product, a project management tool or a word processor, consumes a roughly fixed quantity of server compute regardless of how actively the named user works, because the software is deterministic and the marginal cost of an additional click is near zero. An AI agent seat consumes a variable quantity of token generating compute tied directly to how many queries the user submits and how complex each query is, so the cost basis underneath the seat is no longer flat. That difference, not any feature of the product itself, is what forces the pricing question addressed throughout this report.Underlying Scientific and Engineering PrinciplesA large language model generates output one token at a time through autoregressive decoding, and the compute cost of that process is driven by the number of parameters activated per token, the length of the input context the model must attend to, and the hardware generation running the calculation, which is why token price is the correct unit of cost measurement rather than a flat per query or per seat charge. Every additional output token requires a full forward pass through the active parameters of the model, so output tokens cost substantially more per unit than input tokens in nearly every published price list. OpenAI priced GPT-5.2 input tokens at $1.75 per million and output tokens at $14.00 per million in the first quarter of 2026, an eight to one ratio that a flat per seat price cannot reflect without overcharging light users or losing money on heavy ones.Three forces drive the token cost decline that makes usage based pricing newly viable where it was not viable two years earlier. The first is architectural efficiency: mixture of experts designs, such as the one behind DeepSeek’s V3 model released in January 2025, activate only a fraction of total parameters per token, cutting compute per token without cutting model quality by a proportional amount. The second is hardware: NVIDIA’s own estimates put inference cost per token down roughly tenfold from its A100 generation to its B200 generation. The third is competitive pricing pressure among frontier model providers themselves, visible in OpenAI’s December 2024 cut of 60 percent on GPT-4o realtime input pricing and 87.5 percent on output pricing, a reduction driven by competition from Google’s Gemini line and Anthropic’s Claude line rather than by a change in the underlying hardware cost that same week.Some of these constraints are contingent and some are closer to fixed. The decline in cost per unit of model quality is contingent: it depends on continued algorithmic research, continued hardware generations, and continued competitive pressure among several serious frontier labs, any of which could slow. Closer to fixed, at least over a multi year horizon, is the shape of the cost curve itself: output tokens will keep costing more than input tokens because generation is inherently more compute intensive than comprehension, and heavy users will keep costing substantially more than light users because query complexity and token count move together. A pricing model built on the assumption that this variance will disappear is building on a trend that history gives no reason to expect.System Architecture and Process FlowA usage based or hybrid pricing system requires four subsystems that a seat based product does not need in the same form: a metering layer that counts the input and output tokens, application programming interface calls, or completed actions a customer generates; a rating engine that converts that raw count into a customer facing charge; a credit or wallet abstraction that decouples the vendor’s actual model cost from the price the customer sees; and an invoicing and dunning system able to handle postpaid overage charges rather than a single flat monthly draft. Cursor, built by Anysphere, illustrates the credit abstraction directly: each paid plan carries a monthly credit pool sized to roughly match its subscription price, the company’s own models draw down that pool at a favorable internal rate, and third party models such as Claude or GPT draw it down at each model’s actual … This is a public episode. If you'd like to discuss this with other subscribers or get access to bonus episodes, visit theinnovationattorney.substack.com/subscribe
A radar imaging satellite needs 5 to 10 meters per second of reserve, onboard precise navigation, and relative sensing to answer a close approach like COSMOS 2612.¹ That figure comes from first order orbital mechanics applied to the October 2026 encounter, in which the Russian satellite COSMOS 2612 closed from about 400 kilometers to roughly 500 meters of ICEYE X36, a 90 kilogram synthetic aperture radar satellite registered in the United States.¹²The encounter ran from October 15 to October 19, 2026. Public orbital elements show COSMOS 2612 spent about 67 meters per second of change in speed after September 30 to arrive, and about 105 to 117 meters per second in May to match the orbital plane of its target.² A radar satellite operator cannot match that budget. A reserve of 5 to 10 meters per second buys a separation of 85 to 170 kilometers per orbit after a coplanar lowering burn, which exceeds the roughly 10 kilometer standoff that states have generally kept.⁴Propellant is half the answer. Public element sets for small objects in this orbit carry in track uncertainty near 1.9 kilometers, which is nearly four times the distance LeoLabs reported at closest approach.² A satellite that depends on public tracking data cannot see an event of this size. It needs its own satellite navigation receiver, a way to sense a neighbor directly, and a software path that turns a detection into a burn within one orbit.The legal exposure follows the engineering. Operators licensed under 47 CFR 25.114(d)(14) already disclose collision avoidance capability, and the Liability Convention of 1972 assigns fault based liability for damage in space.⁷¹⁰ Neither text addresses a deliberate approach that causes no collision.Detailed FindingsWhat Happened Between COSMOS 2612 and ICEYE X36?COSMOS 2612 made a controlled, multi week approach to ICEYE X36 between September 30 and October 19, 2026, and held position within about 500 meters of the target on October 17 and 18, according to the commercial tracking firm LeoLabs.¹ The Russian satellite was one of six, COSMOS 2609 through 2614, launched on a single Soyuz 2.1b from Plesetsk in April 2026.³Public element sets show the approach began with phasing, not speed. COSMOS 2612 raised its orbit about 47 kilometers on September 30, descended 47.8 kilometers on October 11 and 12 to match the target altitude, and closed from about 480 kilometers to about 3 kilometers on October 15 and 16 with burns of under 2 meters per second each.² It then held station behind the target on a loop about 1.6 kilometers tall, and on October 18 raised its orbit 2.3 kilometers to leave. By October 19 it was 292 kilometers behind and drifting away at about 350 kilometers per day.²The closest distance carries uncertainty. LeoLabs reported about 500 meters with high confidence. An independent reading of the public catalog supports a range of roughly 150 to 650 meters in the early hours of October 18, and a 25 meter figure that circulated cannot be derived from public data at all.²³ ICEYE reported moving its own satellite and told the press its constellation architecture is designed for continuity.¹² The Pentagon declined to comment.⁴How Much Propellant Does a Radar Satellite Need to Respond to a Close Approach?A radar imaging satellite needs a dedicated avoidance reserve of 5 to 10 meters per second, sized for a coplanar phasing burn instead of a plane change. These figures are the author’s first order calculation, using a circular orbit near 500 kilometers, an orbital speed of about 7.6 kilometers per second, and a semimajor axis of about 6,871 kilometers. The altitude is an assumption drawn from published descriptions of the ICEYE constellation.⁶The calculation reproduces the Russian numbers, which is the reason to trust it. A two burn altitude change of Δa costs a total change in speed near v times Δa divided by twice the semimajor axis. For the 47.8 kilometer descent that gives about 26 meters per second, and public estimates put the real burn near 27 to 28.² A plane change of 0.88 degrees costs about v times the angle in radians, or about 117 meters per second, which sits at the top of the 105 to 117 meter per second range estimated for the May burns.²Applied to the defender, the same equations give the exchange rate that matters. ICEYE lowered its satellite about 0.5 kilometers on June 25 and 26, 2026, a maneuver under 1 meter per second, and the calculation puts that burn near 0.3 meters per second.² A 5 meter per second reserve raises the altitude change to about 9 kilometers. An altitude offset produces an along track drift of roughly three times pi times the offset per orbit, so a 9 kilometer offset opens about 85 kilometers per orbit, and a 10 meter per second reserve opens about 170 kilometers.Plane changes lie outside any defender’s budget. Moving the orbital plane by even 0.1 degree costs about 13 meters per second, and the attacker holds the geometry advantage because it has already matched the plane to within 0.002 degrees.² Evasion therefore has to be in plane, early, and repeatable. COSMOS 2612 closed at about 400 kilometers per day with a 2.3 kilometer altitude offset, which covers the last 3 kilometers in about 11 minutes. A defender that waits for confirmation inside that range has minutes to act, far less than the 95 minute orbit.²In my years as a nuclear submarine officer, the closest point of approach was a number the watch team computed continuously from bearings and ranges that each carried error. A plot whose error exceeded the predicted miss distance gave us nothing to act on, and we treated it as a contact of unknown intent. A satellite operator reading public elements with a 1.9 kilometer in track covariance against a 0.5 kilometer miss distance holds the same plot.²How Accurately Must a Satellite Know Its Own Position and Its Neighbor’s?A satellite needs onboard position knowledge in the meter range and neighbor knowledge within a few hundred meters to make an avoidance decision at these separations. Public element sets cannot supply either. KeepTrack’s analysis put the in track uncertainty for COSMOS 2612 near 1.9 kilometers, and it found gaps in the public record of about 14 hours for COSMOS 2612 and about 8 hours for ICEYE X36 on October 17 and 18, during the closest approach.²A dual frequency satellite navigation receiver gives the host satellite its own position independent of any catalog. The neighbor’s position still comes from outside, which makes the quality of the conjunction data service the limit on what the operator can know. The Department of Commerce is building the Traffic Coordination System for Space under Space Policy Directive 3 of June 18, 2018, and its September 2025 pilot plan called for up to five commercial conjunction assessment screening providers and one independent data quality monitor.⁵¹¹ The pilot describes screening for collisions, not detection of a deliberate approach by an object that stays outside collision thresholds.What Onboard Sensing Detects an Uncooperative Neighbor?Onboard sensing for an uncooperative neighbor requires a sensor that reports relative bearing without help from the neighbor, and the practical candidates are a wide field optical camera, a star tracker run in a detection mode, and a passive radio frequency receiver. This is the author’s engineering assessment. No public document describes what sensing, if any, ICEYE X36 carries for this purpose.A star tracker already images the sky many times per second. A tracker modified to report any moving point source inside a defined range would give bearing and bearing rate for a neighbor inside a few kilometers, which is the range COSMOS 2612 occupied for about two days. Range then comes from the change in bearing as the host maneuvers, a technique the submarine force knows as target motion analysis. A passive receiver tuned to expected crosslink and telemetry bands adds a second line of evidence and tells the operator whether the … This is a public episode. If you'd like to discuss this with other subscribers or get access to bonus episodes, visit theinnovationattorney.substack.com/subscribe
Rare Earth Metals

Rare Earth Metals

2026-11-1340:55

Antimony refining startups selling to defense buyers need ASTM B237 Grade A metal at 99.80 percent antimony or better, with lot level assays and mine origin records for every ingot.American Tungsten and Antimony reported on October 13, 2026 that one ingot from its Del Sol refinery in Amargosa Valley, Nevada assayed 99.8 percent antimony, while four earlier ingots assayed 99.5 to 99.6 percent. One passing sample clears the Grade A floor and shows a process change worked. It shows no repeatability, and the Defense Logistics Agency and the Army buy lots, records and continuity of supply. Financeable offtake terms include a stated minimum quantity, a price floor, an agreed assay protocol with a named umpire laboratory, and acceptance periods short enough to protect working capital.Three domestic projects frame the supplier side: a Defense Logistics Agency contract worth up to $245 million for ingots from United States Antimony Corporation, a $2.9 billion Export Import Bank loan to Perpetua Resources for its Idaho mine, and Nova Minerals’ Alaska plant, where all key equipment arrived on October 6, 2026.The Grade A threshold, the federal awards and the July 20, 2026 tracing order for defense contractors are established. Repeatability at Del Sol, completion of its parent’s acquisition of the site, and the price the government pays when spot prices fall are not.Detailed FindingsWhat Did the Del Sol Assay Actually Show?The Del Sol result is one 99.8 percent antimony assay on one ingot, produced after the company changed its electrowinning conditions, and it sits exactly at the ASTM B237 Grade A minimum of 99.80 percent. In its October 13, 2026 announcement the company said NSL Analytical Services of Cleveland, Ohio tested drill shavings from five ingots made from ore from the White Spar Antimony Mine in Yavapai County, Arizona. Sample 294 of the S260930 series returned 99.8 percent. Samples 290 through 293 returned 99.5 to 99.6 percent, and only the fifth ingot came from the optimized electrowinning conditions.The laboratory measured impurities by inductively coupled plasma mass spectrometry, sulfur by pyrohydrolysis followed by ion chromatography, and mercury by cold vapor atomic absorption. It calculated antimony by difference. A by difference result equals 100 percent minus every impurity measured, so an element nobody measured counts as antimony. A buyer’s specification therefore has to name the element list as well as the antimony floor. Reporting to one decimal place also leaves no margin: a true value of 99.76 percent rounds to 99.8, so the rounding rule belongs in the contract.Coverage of the announcement states that the 99.8 percent sample met the Grade A chemical limits for lead, arsenic, copper, nickel and iron. The coverage reviewed published no impurity values, no recovery rates, no volumes, no funding and no offtake commitment, and it records that the company’s acquisition of the entities holding Del Sol and White Spar has not completed. The company’s first ingot announcement came on September 22, 2026 and said the ingot was made without new processing infrastructure or new permitting for the current campaign. The company now says the result lets it approach the Defense Logistics Agency, Project Vault and defense primes. It also says it expects consistent output above the commercial grade, which it has not demonstrated.On October 5, 2026 a company executive joined a delegation at White House and Office of the United States Trade Representative roundtables, where the contacts included the Trade Representative, a National Security Council official and the director of the Office of Strategic Capital. The discussion centered on funding and offtake with the federal government and defense primes. No agreement was reported.What Grades and Impurity Limits Do Antimony Buyers Use?Antimony ingot trades against three reference points: a 99.65 percent commercial grade, a 99.85 percent premium grade, and ASTM B237 Grade A at 99.80 percent minimum. The standard covers refined antimony as ingot, pig or cake in Grades A and B, sets limits for arsenic, sulfur, lead and other impurities, and allows any smelting or refining route, from ore or recycled feed, that meets the chemistry. Reports of the standard give Grade B a 99.50 percent minimum.The 99.65 percent commercial grade, sold as Regulus II or Grade II, carries maximums of 0.15 percent arsenic, 0.005 percent selenium and 0.06 percent sulfur in one published trading specification. A supplier data sheet adds 0.20 percent lead, 0.002 percent copper and 0.02 percent iron for the same antimony floor. A 99.85 percent ingot assessed by a price reporting agency carries 0.05 percent maximum arsenic, 0.15 percent maximum lead and 100 parts per million maximum bismuth, and the supplier data sheet lists 0.085 percent lead, 0.001 percent copper and 0.04 percent sulfur for its 99.85 percent product.Naming creates a trap. In that supplier data sheet Grade A means 99.65 percent antimony and Grade B means 99.85 percent, the reverse of the ASTM labels, where Grade A is the 99.80 percent product. A contract must cite the standard and its edition, never the letter alone. Physical conventions are consistent across sources: ingots of 25 to 30 kilograms, bundled or palletized near 1,000 kilograms, lots of 25 metric tons in the trading specification, and a certificate of analysis with each delivery.The Defense Logistics Agency’s own ingot specification does not appear in the sources reviewed. The contractor describes it as stringent and gives no standard. A startup should obtain the specification attached to the stockpile solicitation and design to the stricter of that document and Grade A.Why Do Defense Buyers Want Antimony Metal and Antimony Trisulfide?Defense programs need antimony because ammunition consumes it and because foreign producers control most of the supply. Federal minerals data for 2025 estimates that 49 percent of United States use went to metal products including flame retardants, 40 percent to antimonial lead and ammunition, and 11 percent to ceramics, glass and rubber. Spent lead acid batteries supply most secondary antimony, which met 12 percent of 2025 apparent consumption.The Army states that antimony sulfide is a key ingredient in nearly 30 percent of military ammunition items, where it helps primers operate across all temperature ranges, and that the United States sourced it from China until 2021. Net import reliance for antimony was an estimated 91 percent in 2025. China supplied 55 percent of imported antimony metal and oxide from 2021 through 2024. China announced export restrictions in August 2024 and banned shipments to the United States in December 2024, and controls remained after it suspended that ban in November 2025. The average metal price rose from $9.80 per pound in August 2024 to $18.10 in December 2024, reached $27.50 in June 2025 and stood at $20.30 in November 2025.Product choice sets the customer. Ingot is what the stockpile contract buys. Trisulfide is what the Army and ammunition makers need. Oxide, the flame retardant feedstock, is the largest import line at an estimated 39,000 metric tons in 2025, against 4,500 metric tons of unwrought metal and powder. A refiner that serves defense buyers still competes for commercial volume, because defense stockpile demand is small. The fiscal year 2025 annual materials plan listed potential antimony acquisitions of 700 tons, and the fiscal year 2026 plan had not been released when the 2026 federal data were published.Which US Producers Are Already Selling or Building?United States Antimony Corporation is the supplier delivering metal ingots to the Defense Logistics Agency under the contract reviewed here, and its second quarter 2026 results show the price risk a startup would inherit. The company sells antimony oxide, metal ingots and trisulfide from North American smelters. It reported 428,425 pounds of antimony sold in the second quarter of 202… This is a public episode. If you'd like to discuss this with other subscribers or get access to bonus episodes, visit theinnovationattorney.substack.com/subscribe
An AI coding agent such as Claude Code now lets a single product manager draft a specification, synthesize customer interviews, build a working prototype, and in a growing number of companies open a pull request that ships to production, work that as recently as 2024 moved through an engineer by default. Reporting published June 27, 2026 described Anthropic’s own growth team being told to hire more product managers rather than fewer once its engineering organization began shipping at roughly three times its prior headcount, and the same reporting put the traditional one to eight product manager to engineer ratio at an effective one to twenty inside agent heavy teams. The shift is uneven. A 2024 Pragmatic Institute survey found that sixty five percent of product professionals had already integrated AI into their work, concentrated in routine and descriptive tasks, while a 2026 developer survey found that forty six percent of engineers still distrust the output AI tools produce, up from thirty one percent the year before.The central finding of this report is that the handoff point between product management and engineering has moved, not disappeared. Product managers at Amazon, Google, Meta, LinkedIn, and a small cluster of independent software companies are now shipping prototypes, internal tools, and in some documented cases production fixes directly, while engineers retain sign off authority on anything that touches authentication, data integrity, or system architecture. A March 18, 2026 content analysis of 638 published Lenny’s Newsletter and podcast items, built with a Claude Opus 4.6 multi agent research team, found that artificial intelligence became the dominant lens for discussing product management starting in the second quarter of 2024, rising from under four percent of coverage in late 2022 to sixty seven percent in the first quarter of 2026.For investors and operators allocating capital to product organizations, the practical implication is that headcount models built on a fixed product manager to engineer ratio are already out of date at companies that have adopted agentic coding tools, and the associate product manager track, the traditional entry point for junior hires, is contracting faster than any other segment of the profession.Definition, Scope, and Technical ContextAn AI agent, for purposes of this report, is a large language model deployed with tool access: the ability to read and write files, execute code, run tests, query databases, browse the web, and complete multi step tasks with limited human prompting between steps. This definition separates an agent from a conversational chatbot. Tal Raviv, writing for Lenny’s Newsletter on April 29, 2025, drew this same line, distinguishing agents by their capacity to act proactively, draw on company context and live data, take real actions, and check their own output. Claude Code, Anthropic’s command line coding agent, and comparable tools such as Cursor and OpenAI’s Codex, are the dominant instances of this category inside product organizations as of October 2026.The scope of this report is the set of tasks that moved, or are moving, from a product manager’s engineering handoff to an agent: drafting specifications, synthesizing user research, building interactive prototypes, writing code for internal tools and small production fixes, and the review structure that now governs that code before it reaches customers. The report excludes the law and regulation that may someday govern agent authored software, consistent with the scope the user set for this engagement, and it excludes agent use inside pure engineering teams that does not touch the product management function.The system under analysis sits inside the broader category of agentic software engineering, a 2025 and 2026 development that layers planning, tool use, and self checking on top of a foundation model’s text generation capability. The product management variant of this system differs from its engineering counterpart principally in who holds the keyboard: the operator is frequently a person without a formal computer science background, supervising an agent that performs the implementation work a software engineer once performed by hand.Underlying Scientific and Engineering PrinciplesAn agentic coding tool operates as a loop: the model receives an instruction, forms a plan, calls a tool such as a shell command or a file edit, observes the result, and repeats until it judges the task complete or asks for human input. Paweł Huryn, writing in The Product Compass on May 31, 2026, described building three public repositories this way, including a Visual Studio Code extension called PM Brain supported by more than eight hundred automated tests, without personally reading most of the underlying code. The mechanism that makes this possible is a combination of a large context window, the ability to execute and observe code and run it again, and an iterative test and correct cycle that substitutes for a human’s step by step verification.Two constraints govern what this loop can do, and they sit in different categories. The first is contingent: context window size, inference cost, and the breadth of tool integrations have all expanded rapidly since 2023 and will likely keep expanding, the same way compute cost per unit of inference has fallen steadily since large language models reached commercial deployment. The second sits closer to the specification problem long studied in software engineering: an agent verifies its output against the tests and instructions it was given, a narrower target than the business intent that produced those instructions. Huryn’s own account notes that models tend to overrate the severity of their own findings, so a human or a second model still has to judge which flagged issues matter, a judgment step that more context or faster inference has not yet resolved on its own.My own engineering background offers a useful point of comparison. A nuclear submarine’s engineering watch qualification process, like a semiconductor fabrication line’s design rule check, builds in an independent verification step performed by someone other than the person who did the work, because self assessment under operational pressure carries real failure rates in both domains, regardless of how competent the operator is. The code review requirement that persists at every company examined in this report, even where a product manager writes the code, follows the same pattern: an independent reviewer catches what the author of a change cannot see in the author’s own work, whether the author is a submarine officer, a fabrication engineer, or a large language model.System Architecture and Process FlowThe agent augmented product workflow documented across the companies in this report runs on three layers. The first is a context layer that the product manager maintains: strategy documents, architecture notes, and what Huryn calls agent contracts, which are written rules that tell an agent what it is and is not permitted to change. The second is the agent layer itself, which plans, writes code, and runs tests, frequently delegated to sub agents for narrower tasks such as a scheduled process that triages incoming pull requests and feature requests without a human initiating each run. The third is a verification layer, which combines automated test suites, mechanical permission gates that block certain classes of change outright, such as an auto merge rule that refuses anything touching authentication or secrets, and in several accounts a second model from a different vendor reviewing the first model’s output before a human looks at it.Anna Mulry, an AI product manager at Orbital Witness, described this architecture in practice in a February 18, 2026 account. She diagnosed an unreliable citation rendering bug in the company’s drafting output, changed the underlying logic herself, opened a pull request, and shipped the fix after review, while… This is a public episode. If you'd like to discuss this with other subscribers or get access to bonus episodes, visit theinnovationattorney.substack.com/subscribe
An AI model retirement migration before OpenAI’s October 23, 2026 shutdown of GPT 4 and o1 requires frozen test cases, repeated trials, and regression gates set per item.OpenAI announced the shutdown on April 22, 2026. The list covers GPT 4, GPT 4 Turbo, o1, o1 pro, o3 mini, o4 mini, GPT 4.1 nano, GPT 3.5 Turbo, a May 2024 snapshot of GPT 4o, and every fine tuned snapshot built on them, with GPT 5.6 Sol, Terra, and Luna named as successors. Google stopped serving Gemini 2.5 Pro, Flash, and Flash Lite on its Gemini Enterprise Agent Platform on October 20, 2026, and named Gemini 3.8 Flash as the replacement.A 2026 study of 900 benchmark items, each run 50 times across three consecutive GPT releases, found reliable improvements and reliable regressions inside every migration. In one comparison the aggregate score rose 7.3 points and 6.0 percent of items regressed anyway. A single run per item detected none of the 457 confirmed changes.Price adds a second variable. Moving a GPT 4 workload to Sol cuts list input cost 83.3 percent. A GPT 3.5 Turbo workload that lands on Terra pays eight times the old output rate, and Luna, at $0.20 input and $1.20 output per million tokens, sits below the old GPT 3.5 Turbo rate.Teams should freeze their own test cases from production traces, run 30 to 50 trials on high consequence items, gate cutover on the list of regressed items, and schedule the next wave: December 11, 2026 and January 14, 2027.Definition, Scope, and Technical ContextWhat Is an AI Model Retirement Migration?An AI model retirement migration is the engineering process of moving a production workload from a hosted model identifier that a provider has scheduled to stop serving to a successor identifier, while holding behavior, cost, and latency inside tolerances the team defines in advance. It belongs to software change management, and the dependency being changed is a probabilistic function whose internals the deployer cannot inspect. Deprecation is the announcement, retirement is the date the identifier stops answering requests, and drift is a behavior change that arrives with no announcement at all.The October 23, 2026 cutoff comes from OpenAI’s April 22, 2026 notice. The identifiers include GPT 4 (the 0613 snapshot), GPT 4 Turbo (the April 9, 2024 snapshot), a GPT 4 1106 preview, a GPT 4o snapshot dated May 13, 2024, GPT 4.1 nano, o1, o1 pro, o3 mini, o4 mini, GPT 3.5 Turbo, the completions variants of several of these, GPT Image 1, and the fine tuned versions built on GPT 3.5 Turbo, GPT 4, GPT 4.1 nano, and o4 mini. OpenAI named GPT 5.6 Sol as the successor for GPT 4, GPT 4 Turbo, o1, o3 mini, and the older GPT 4o snapshot, GPT 5.6 Terra for o4 mini and GPT 3.5 Turbo, GPT 5.6 Luna for GPT 4.1 nano, and a GPT Image 2.5 model for GPT Image 1.Today is October 22, 2026, so a team reading this has one day left. Calls to a retired identifier stop being served after the cutoff, which converts every unfinished migration into a production incident. The work described below fits a one day sprint for a small inventory and a multi week program for a large one, and the same pipeline serves the later waves.Which Providers Retire Models Between October 19 and October 23, 2026?Four providers retire models inside that window. Google ended calls to Gemini 2.5 Pro, Gemini 2.5 Flash, and Gemini 2.5 Flash Lite on its Gemini Enterprise Agent Platform, formerly Vertex AI, on October 20, 2026, while the Gemini API keeps serving those models with no shutdown date. Google shuts down Gemini Omni Flash Preview and three Veo 3.1 preview models on October 22, 2026, with Gemini Omni 1.1 Flash as the replacement, and retires a deep research preview agent on October 23, 2026. Together AI ends serverless access to Llama 3.3 70B Instruct Turbo and DeepSeek V4 Flash 0731 on October 22, 2026. NVIDIA stopped supporting Nemotron 3 Super on its free NIM endpoints after October 19, 2026.How Much Notice Do Providers Give Before Retiring a Model?OpenAI publishes a minimum of six months of notice for generally available models, three months for specialized variants such as chat, Codex, and deep research models, and as little as two weeks for preview models. The April 22, 2026 announcement preceded the October 23, 2026 shutdown by six months and one day. Fine tuned models follow the schedule of their base model, and the List models and Retrieve a model endpoints return a shutdown_date field that a script can read.Google labels the dates on its Gemini API deprecation page as the earliest possible shutdown dates and sets no minimum notice period. Anthropic commits to preserve the weights of every publicly released Claude model for at least the lifetime of the company, and the same commitment page states that preserved weights do not guarantee continued availability for inference. The analysis here covers hosted model APIs and excludes consumer chat products and self hosted open weight models.Underlying Scientific and Engineering PrinciplesWhy Does Changing the Model Change Application Behavior?Changing the model changes application behavior because a hosted model acts as a function of its weights, the prompt, the decoding settings, and the serving stack, and a retirement replaces the weights and usually the serving stack with them. Prompts written against one set of weights absorb that model’s habits: its tolerance for ambiguous instructions, its default verbosity, its willingness to hold a format. A December 2025 paper calls the resulting loss Model Drifting and reports that a prompt written for one model usually performs worse on another than a prompt tuned for the target.Sampling adds noise on top. The same prompt yields different outputs on repeated calls, so one run per test item mixes real change with chance. With 50 trials and a true pass rate near 50 percent, the standard error is about 7.1 points and a 95 percent interval spans roughly 14 points either side (a calculation from the binomial formula). A single trial per item has no interval at all.Which Constraints Are Fundamental and Which Are Contingent?Three constraints are fundamental. Sampled generation is stochastic. The retired model cannot be queried after shutdown, so the baseline has to be captured before the date. Provider internals stay opaque, so the deployer sees behavior and never the cause. Anthropic’s preservation of weights illustrates the second constraint: keeping weights on disk does not keep a model answering requests.Four constraints are contingent and open to engineering: prompt portability across models, the coverage of the evaluation set, the cost of repeated trials, and the price of the successor. Money and discipline move each of these four.How Large Is Behavioral Drift in Practice?A July 2023 study found that GPT 4 accuracy on a prime number identification task fell from 84 percent in March 2023 to 51 percent in June 2023 under the same model name, and a 2026 analysis of the hosted model supply chain cites a fall in directly executable generated code from 52 percent to 10 percent over the same interval. The 2026 study of three consecutive GPT releases found a quieter form of the same effect. On an instruction following benchmark the gap between loose and strict scoring widened to 9.0 points on Sol, with the strict score down 3.9 points and the loose score nearly flat, which points to a formatting regression and leaves reasoning ability intact.System Architecture and Process FlowWhat Components Does a Migration Pipeline Need?A migration pipeline needs seven components. A gateway layer holds model identifiers in configuration so that application code never names a model. A prompt registry versions every prompt. A frozen evaluation set is drawn from production traces. A trial runner repeats each item many times. Scorers include a strict parser, a tolerant parser, and optionally a model based judge. Telemetry records tokens, latency, and cost for every call. A router sends shadow and canary traffic and supports ro… This is a public episode. If you'd like to discuss this with other subscribers or get access to bonus episodes, visit theinnovationattorney.substack.com/subscribe
loading
Comments