In this episode of the Data Science Bulletin podcast, hosts delve into the historical significance of AI research by exploring a list of recommended papers, including Geoffrey Hinton's "Keeping Neural Networks Simple by Minimizing the Description Length of the Weights," which addresses overfitting in neural networks. They discuss the concept of harness engineering, which involves creating supportive infrastructures around AI models, particularly in programming and productivity contexts, highlighting the differences between coding agents and broader productivity tools. The episode concludes with an exploration of new trends in AI engineering, such as graph engineering, and encourages listeners to stay informed about the evolving landscape of AI technology.blog post
In this episode, hosts delve into the concept of "world models" in AI, discussing their potential applications and the debates surrounding their development, particularly focusing on the insights of experts like Jan LeCun and Fei-Fei Li. They explore the challenges of hierarchical planning and representation in AI, as well as the niche nature of world models compared to mainstream large language models. The episode also touches on the June's controversy involving Anthropic and the U.S. government's restrictions on AI tools, highlighting the complexities and geopolitical implications of AI regulation.Blog post
In the 20th episode of the Czech-Belgian Data Science Bulletin podcast, hosts Martin and Jakub explore two main topics. They delve into a paper titled "Evaluating Agents Markdown: Are Repository-Level Context Files Helpful for Coding Agents?" which discusses the impact of adding Markdown files to code repositories for coding assistants, weighing human-written versus AI-generated files. Additionally, they discuss Andrej’s insights on personal knowledge bases and how to effectively organize and query personal wikis using language models.Blog post
In the nineteenth episode of the Data Science Bulletin podcast, hosts discuss two articles focusing on the influence of large language models and AI on the future of various fields."Something Big is Happening" This article analyzes the potential impact of AI on our lives, highlighting both the current enthusiasm and the possible overestimation of the capabilities of today's language models."Global Intelligence Boom 2028" This piece offers an optimistic look at the economic effects of AI, speculating on deflation and the creation of new job opportunities—though the authors critique some of its assumptions as being unrealistic.Short writeup on DSB website
In this episode we discuss the research paper 'Large Language Model Reasoning Failures,' which was published in Transactions on Machine Learning Research. We focus on a categorization framework for identifying situations where large language models fail at reasoning, pointing out that the goal of the paper is not to evaluate whether these models actually think, but rather to understand their limitations compared to human reasoning. The episode also highlights that these errors can be similar to cognitive biases in humans and discusses possible approaches to overcoming them.Blog post with detailed description.