What's the latest with artificial intelligence and models' accuracy?

Three objectives:

1. Compare and contrast traditional autoregressive LLMs (Gemini/chatGPT/Claude/LLaMA) vs nonautoregressive AI (NotebookLM) vs chain of thought reasoning models (Strawberry o1) – including the benefits, detriments, and tradeoffs of each


2. Error rates for NotebookLM vs traditional LLMs vs CoT reasoning models – focusing on accuracy benefits from a smaller corpus of curated source materials that users feed NotebookLM when promoting (vs an LLM trained on and inferencing from the entire web)


3. Which model (or combination of models) will be the future standard?

See also:🧵https://x.com/AnthPB/status/1848186962856865904

This podcast is AI-generated with NotebookLM, using the following sources, research, and analysis:

• An Investigation of Language Model Interpretability via Sentence Editing (OSU, Stevens, 2021.04)


• Are Auto-Regressive Large Language Models Here to Stay? (Medium, Bettencourt, 2023.12.28)


• Attention Is All You Need (Google Brain, Vaswani/Shazeer/Parmar/Uszkoreit/Jones/Gomez/Kaiser/Polosukhin, 2017.06.12)


• BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension (Facebook AI, Lewis/Liu/Goyal/Ghazvininejad/Mohamed/Levy/Stoyanov/Zettlemoyer, 2019.10.29)


• Chain of Preference Optimization: Improving Chain-of-Thought Reasoning in LLMs (Zhang/Du/Pang/Liu/Gao/Lin, 2024.06.13)


• Contra LeCun on "Autoregressive LLMs are doomed" (LessWrong, rotatingpaguro, 2023.04.10)


• Do large language models need sensory grounding for meaning and understanding? (LeCun, 2023.03.24)


• Experimenting with Power Divergences for Language Modeling (Labeau/Cohen, 2019)


• Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer (Raffel/Shazeer/Roberts/Lee/Narang/Matena/Zhou/Li/Liu, 2023.09.19)


• Improving Non-Autoregressive Translation Models Without Distillation (Huang/Perez/Volkovs, 2022.01.28)


• Non-Autoregressive Neural Machine Translation (Gu/Bradbury/Xiong/Li/Socher, 2017.11.27)


• On the Learning of Non-Autoregressive Transformers (Huang/Tao/Zhou/Li/Huang, 2022.06.13)


• Towards Better Chain-of-Thought Prompting Strategies: A Survey (Yu/He/Wu/Dai/Chen, 2023.10.08).pdf


• XLNet: Generalized Autoregressive Pretraining for Language Understanding (Yang/Dai/Yang/Carbonell/Salakhutdinov/Le, 2020.01.22)

Not investment advice; do your own due diligence!

# tech technology machine learning ML