From Coder to Manager: Navigating the Shift to Agentic Engineering with Notion Co-Founder Simon Last
Digest
This discussion explores Notion's evolving AI vision, shifting from direct human task execution to managing AI agents. It details the genesis of Notion AI, triggered by GPT-4, and outlines short-term (writing assistant) and long-term (general assistant) strategies. Key features like AI Writer and Q&A, powered by semantic indexing and embeddings, are discussed, alongside the challenges of indexing diverse data. Notion continuously iterates its AI harness and adapts its platform with specialized APIs for agent interaction. The impact of AI agents on engineering and team dynamics is profound, increasing individual output and fostering a more chaotic yet productive prototyping environment. Rigorous review processes ensure the safety of agent-generated code. Notion aims for self-bootstrapping agents and positions itself as a neutral platform for various AI models, including cost-effective open-source options. The concept of "tools for thought" is redefined as managing agents, with engineers becoming "agent managers." Non-technical teams are empowered to build agents through workshops, demonstrating broad adoption. The personal workflow of an agent manager is highlighted, emphasizing design, verification, and monitoring.
Outlines
Notion's AI Vision and Evolution
Simon Last discusses Notion's AI-powered collaboration vision, the evolution of its engineering and product organization, and the shift in "tools for thought" from direct human work to managing AI agents. The integration of AI agents has significantly increased individual engineer impact and ambition, leading to more prototypes and ambitious projects.
Developing and Integrating Notion AI Features
The pivotal moment for Notion AI integration was encountering GPT-4 in 2022. Notion adopted both short-term (writing assistant) and long-term (general assistant) AI visions, launching AI Writer in February 2023 and a Q&A feature in October 2023. This involved building real-time semantic indexes and expanding indexing to platforms like Slack and Google Drive, tackling complex data source challenges with AI-pilled savviness and iterative testing. Embeddings are used to handle workspace diversity, and Notion rewrites its AI harness approximately every six months to keep pace with evolving AI models.
AI Agents, Workflows, and User Empowerment
The integration of AI has led to increased chaos and prototyping, exemplified by the design team's "design playground." Rigorous pull request reviews are maintained for agent-generated code, ensuring safety and confidence. Notion aims for agents to bootstrap their own capabilities and positions itself as a "Switzerland for models," offering access to various AI models, including cost-effective open-source options. Notion is adapting its structure with specialized APIs for agents, using a markdown dialect and SQLite for efficient interaction. Designing agent-friendly APIs is an empirical process, and agents are treated as users in ongoing research. Personal workflows involve multiple agents for tasks like email triage and bug routing, with custom agents learning preferences over time. Notion empowers non-technical teams to build agents through workshops, enabling them to create valuable automated workflows.
The Future of Work: From Coder to Agent Manager
Notion's core mission has shifted to enabling humans to manage agents performing work. This transformation redefines the role of individuals, with Simon Last now acting as an "agent manager," designing tasks, verifying agent work, and monitoring progress, rather than writing code manually. The discussion concludes with encouragement to follow NoPriors and subscribe for updates.
Keywords
AI Agents
Autonomous software entities that can perform tasks, learn, and interact with their environment. In Notion, they can manage workflows, process information, and collaborate with humans, representing a significant shift in productivity.
Notion AI
Notion's suite of AI-powered features designed to enhance productivity and collaboration. This includes AI writers, Q&A capabilities, and increasingly sophisticated agents that can perform complex tasks within the Notion ecosystem.
Tools for Thought
Software designed to augment human cognition and thinking processes. Notion's evolution reflects a shift from direct task execution tools to platforms for managing and collaborating with AI agents, redefining how we think and work.
Semantic Indexing
The process of creating a searchable index based on the meaning and context of information, rather than just keywords. This allows AI to understand and retrieve relevant information from large datasets, crucial for Notion's Q&A features.
Embeddings
Numerical representations of data (like text) that capture semantic relationships. Embeddings enable AI to understand context and similarity, facilitating efficient retrieval and organization of information within Notion, regardless of user-defined structures.
AI Harness
The underlying system or framework that enables AI models to function and interact within a specific application. Notion continuously iterates on its AI harness to integrate the latest AI advancements and optimize performance.
Custom Agents
User-configurable AI agents within Notion that can be granted specific permissions and tasked with autonomous operations. These agents can manage workflows, process information, and learn over time, offering personalized automation.
Agent Manager
A role focused on designing, overseeing, and verifying the work of AI agents, rather than directly performing tasks. This signifies a shift in human involvement, moving from execution to strategic direction and quality control of AI-driven processes.
Agent API Design
The empirical process of creating interfaces for AI agents to interact with applications, focusing on efficiency, observation of limitations, and understanding model priors for natural interaction.
Open Source Models
Increasingly capable and cost-effective AI models that Notion plans to integrate, offering users flexibility and a cheaper alternative to frontier models for specific use cases.
Q&A
How has Notion's vision for its platform evolved with the advent of AI?
Notion's core goal has shifted from creating the best tool for humans to directly perform work, to creating the best tool for humans to manage agents that perform work for them. This involves new primitives for representing agents and their interactions.
What are the key AI features Notion has launched, and how have they evolved?
Notion launched AI Writer for text manipulation, followed by a Q&A feature using semantic indexing. They've expanded indexing to external sources and developed personal and custom agents capable of autonomous tasks and learning.
How does Notion ensure the quality and safety of code generated by AI agents?
All pull requests, even those generated by agents, undergo rigorous review. While agent-generated code can be more complex, it's often better tested, and Notion demands comprehensive end-to-end testing before deployment.
How does Notion adapt its platform to be more agent-friendly?
Notion has developed specialized, convenient APIs for agents, moving beyond human-centric formats. They use a markdown dialect for pages and SQLite for databases, optimizing data structures for agent interaction and efficiency.
What is the role of open-source models in Notion's AI strategy?
Notion sees open-source models as a valuable, cost-effective alternative to frontier models. They plan to integrate multiple open-source options, providing users with flexibility and choice in the AI models they utilize.
How does Notion empower non-technical users to leverage AI agents?
Through workshops and hackathons, Notion educates non-technical teams on building agents. The intuitive interface allows users, like those in the People team, to create custom agents and automate workflows effectively.
How has the role of engineers changed at Notion with the rise of AI agents?
Engineers are transitioning from direct coders to "agent managers." Their focus shifts to designing end-to-end tasks, verifying agent outputs, and monitoring their progress, significantly increasing their potential impact and efficiency.
How does Notion approach the design of APIs for agent interaction?
Designing agent-friendly APIs is an empirical process involving experimentation, observation of limitations, and understanding model priors to ensure efficient and natural agent interaction.
How does Notion conduct user research with AI agents?
Notion treats agents as users in their research, conducting ongoing evaluations and "user research" by interacting with them to allow for continuous improvement and adaptation of agent functionalities.
What is the future vision for Notion agents?
Notion aims for agents to bootstrap their own capabilities, potentially building integrations or connecting to new data sources, pushing towards more autonomous and self-improving agents.
Show Notes
Notion isn’t designing AI agents that just use tools. Their agents can autonomously build their own integrations, as well as write the code needed to finish a task. Sarah Guo sits down with Notion Co-Founder Simon Last to explore Notion’s rapid evolution from a simple writing assistant to a sophisticated platform for custom AI agents. Simon discusses the technical hurdles of indexing disparate data from sources like Slack and Google Drive, as well as the internal shift toward using coding agents to build Notion itself. Plus, Simon elaborates on what he sees as a fundamental transition in productivity: moving from a tool where humans do the work, to one where humans manage a swarm of agents.
Sign up for new podcasts every week. Email feedback to show@no-priors.com
Follow us on Twitter: @NoPriorsPod | @Saranormous | @EladGil | @simonlast | @NotionHQ
Chapters:
00:00 – Cold Open
00:05 – Simon Last Introduction
00:26 – Genesis of Notion AI
04:10 – Challenge of Semantic Indexing and Retrieval
07:16 – The Six-Month Rewrite Cycle
08:12 – Notion’s Coding Agent Era
09:44 – Impact on Team Dynamics
12:49 – Launching Custom Agents
15:39 – Notion as the ‘Switzerland’ for Models
17:33 – Designing APIs for Agent Customers
20:09 – Simon’s Personal Agentic Workflows
24:48 – Notion: Tool for Work is Now A Tool for Agents
27:28 – How Building Has Changed for Simon
29:00 – Conclusion
Table of contents
United States
