In this episode, Luca Antiga, CTO of Lightning AI and ODSC East 2024 speaker, joins us to discuss Small Language Models (SLMs).

Lightning AI, well regarded for providing tools and platforms that streamline workflows for machine learning engineers and AI researchers, was founded by the creators of the very popular open-source framework PyTorch Lightning.

Luca himself was an early contributor to the PyTorch core library and co-authored the book “Deep Learning with PyTorch”.

In this interview, we’ll explore the differences between SLMs and Large Language Models (LLMs), examine the importance of data quality in training SLMs, and talk about key SLM frameworks such as TinyLama.

We’ll also touch on the process of combining SLMs with Retrieval Augmented Generation (RAG), address hallucinations within SLMs in particular, and how to deploy SLMs into production. 

SHOWNOTE TOPICS:

1- Guest Introduction

2- Explanation of Small Language Models (SLMs) is and how they differ from Large Language Models (LLMs)?

3- How SLMs are trained and the importance of data quality

4- Continued pre-training vs fine-tuning

5- Incontext learning for LLMs

6- Benchmarking LLMS with MMLU

7- Small Language Model (SLM) frameworks such as MiniCPM, TinyLlama, and LitGPT

8- Combining SLMs with Retrieval Augmented Generation (RAG) and fine-tuning 

9- Why SLMs are more prone to hallucinations than LLMs and how to solve

10- LLMs and SLMs in production - minimizing risks and maximizing value 

11- The problem of over optimizing on AI solutions 

12- Practical applications of SLMs, 

13- Small Language Model (SLM) training challenges

14- Overview of Lightning studio

15- AI Agents and challenges

16- Compute access from single to multi-GPUs

17- Using lightning studio to easily switch between compute setups

18- Quick fire questions:

 - Will more foundational LLMs emerge?

 - Open source vs closed-scouce LLM

 - Artificial General Intelligence? 

 - In this fast paced environment what should practitioners focus on?

 - What are companies getting wrong with AI now?

 - What are companies getting right with AI?

SHOW NOTES:

Learn more about Luca Antiga, CTO at Lightning AI: 

https://www.linkedin.com/in/lantiga/?originalSubdomain=it

Learn more about Lightning AI and PyTorch Lightning:

https://lightning.ai/

https://lightning.ai/docs/pytorch/stable/

https://github.com/Lightning-AI

https://pytorch.org/

Learn more about Small Language Models

https://opendatascience.com/the-growth-of-small-language-models/

https://opendatascience.com/6-small-language-models-to-get-the-job-done-with-ease/

MMLU (Multi-task Language Understanding) Benchmark:

https://arxiv.org/pdf/2009.03300

MiniCPM: Unveiling the Potential of Small Language Models with Scalable Training Strategies

https://ar5iv.labs.arxiv.org/html/2404.06395

Small Language Model Frameworks:

https://github.com/jzhang38/TinyLlama

https://github.com/Lightning-AI/litgpt

Why Larger Language Models Do In-context Learning Differently? 

https://arxiv.org/abs/2405.19592

Learn more about RAG:

https://github.blog/2024-04-04-what-is-retrieval-augmented-generation-and-what-does-it-do-for-generative-ai/

Learn more about dottxt:

https://dottxt.co/

Learn more about structured text generation:

https://github.com/outlines-dev/outlines

Learn more about HyperTalk:

https://en.wikipedia.org/wiki/HyperTalk

Learn more about crewAI:

https://github.com/joaomdmoura/crewAI

Learn more about Lightning AI Studios:

https://lightning.ai/studios

Learn more about Concurrent Versions System:

https://en.wikipedia.org/wiki/Concurrent_Versions_System