Artificially Speaking

The provided text introduces Cognitive Architectures for Language Agents (CoALA), a theoretical framework designed to standardize the development of artificial intelligence systems that use large language models as their core. By drawing parallels between modern AI and historical symbolic cognitive science, the authors propose a modular structure involving working and long-term memory, a diverse action space, and a structured decision-making process. This framework categorizes agent actions into grounding for external interaction and internal processes like reasoning, retrieval, and learning. CoALA provides a unified language to compare existing agents and identifies future research paths toward language-based general intelligence. Through various case studies, the sources demonstrate how this systematic approach can enhance an agent's ability to reason, plan, and adapt to complex environments.

SkillOpt is a novel optimization framework designed to improve the performance of AI agents by treating their skills as a trainable, external text document. Unlike traditional methods that rely on manual prompting or model fine-tuning, this system uses a separate optimizer model to analyze successes and failures from task trajectories and propose structured edits. These edits are governed by deep-learning-inspired controls, including a textual learning rate to limit change size and a validation gate that only accepts improvements. This process ensures the agent’s strategies remain stable and reproducible without requiring expensive updates to the underlying model weights. Across diverse benchmarks, the method significantly boosts accuracy and produces compact, human-readable skill artifacts that are easily understood. Furthermore, these optimized skills demonstrate strong generalization, retaining their value even when transferred across different model scales and execution environments.

The document introduces Vending-Bench, a novel simulated environment designed to evaluate the long-term coherence of autonomous agents powered by Large Language Models (LLMs). The benchmark tasks agents with managing a virtual vending machine business over extended periods, requiring them to handle inventory, pricing, ordering, and daily fees. While some advanced LLMs like Claude 3.5 Sonnet show promising performance, the results reveal high variance and a tendency for models to derail into tangential "meltdown" loops when encountering unexpected situations, such as misinterpreting delivery schedules. The research indicates that these failures are not directly caused by context window limitations, suggesting a deeper challenge in sustained, coherent decision-making for LLMs over long time horizons. The authors hope Vending-Bench will aid in assessing and preparing for stronger AI systems, including those with dual-use capabilities.

This research paper explores the capabilities of large language models (LLMs) in competitive programming. It compares the performance of OpenAI's o1 and o3 LLMs, highlighting the significant improvement in performance achieved by o3 through increased reinforcement learning. The study also examines a specialized LLM, o1-ioi, fine-tuned for the International Olympiad in Informatics (IOI), demonstrating that scaling general-purpose models surpasses the performance gains from specialized, hand-engineered approaches. Furthermore, the paper evaluates the LLMs' performance on real-world software engineering tasks, showcasing the broad applicability of enhanced reasoning capabilities in coding. Overall, the findings suggest that scaling reinforcement learning in general-purpose LLMs offers a robust path towards achieving state-of-the-art AI in complex reasoning domains.

This research paper explores using executable Python code as actions for Large Language Model (LLM) agents. The authors introduce CodeAct, a framework enabling LLMs to generate and execute Python code, dynamically adapting actions based on observations. Experiments across 17 LLMs demonstrate CodeAct's superior performance in complex tasks, achieving up to a 20% higher success rate than alternatives. A new instruction-tuning dataset, CodeActInstruct, is created to improve open-source LLMs' CodeAct capabilities, resulting in CodeActAgent, an open-source agent capable of sophisticated tasks. The paper concludes by discussing the potential benefits and risks of such autonomous agents.