DiscoverThe Stack Overflow PodcastHow Stack Overflow fends off scraping bots
How Stack Overflow fends off scraping bots

How Stack Overflow fends off scraping bots

Update: 2024-07-30
Share

Description

As Josh explains, DDoS attacks aim to take down a website, while bot scrapers try to gather as much data as possible without getting caught.

Josh Zhang is a staff site reliability engineer (SRE) at Stack Overflow. Connect with him on LinkedIn.

ICYMI: In 2022, Josh wrote an article for our blog about how Stack defends itself against DDoS attacks.

Stack Overflow user Serge Ballesta won a Lifeboat badge for answering What does |= mean in c++.

Comments 
Top Podcasts
The Best New Comedy Podcast Right Now – June 2024The Best News Podcast Right Now – June 2024The Best New Business Podcast Right Now – June 2024The Best New Sports Podcast Right Now – June 2024The Best New True Crime Podcast Right Now – June 2024The Best New Joe Rogan Experience Podcast Right Now – June 20The Best New Dan Bongino Show Podcast Right Now – June 20The Best New Mark Levin Podcast – June 2024
In Channel
Balancing business and open source in 2024

Balancing business and open source in 2024

2024-12-2733:36

How developer jobs (and the job market) changed in 2024

How developer jobs (and the job market) changed in 2024

2024-12-2433:21

“I wanted to play with computers”: a chat with a new Stack Overflow engineer

“I wanted to play with computers”: a chat with a new Stack Overflow engineer

2024-12-2026:19

Legal advice from an AI is illegal

Legal advice from an AI is illegal

2024-12-1705:19

AI agents that help doctors get paid

AI agents that help doctors get paid

2024-12-1321:02

What security teams need to understand about developers

What security teams need to understand about developers

2024-12-1022:27

From bugs to performance to perfection: pushing code quality in mobile apps

From bugs to performance to perfection: pushing code quality in mobile apps

2024-12-0606:18

Even high-quality code can lead to tech debt

Even high-quality code can lead to tech debt

2024-12-0328:38

Your docs are your infrastructure

Your docs are your infrastructure

2024-11-2622:56

The app that fights for your data privacy rights

The app that fights for your data privacy rights

2024-11-2225:50

We'll Be In Touch: A New Podcast From Stack Overflow!

We'll Be In Touch: A New Podcast From Stack Overflow!

2024-11-2133:27

The open-source ecosystem built to reduce tech debt

The open-source ecosystem built to reduce tech debt

2024-11-1927:14

Meet the guy responsible for building the Call of Duty game engine

Meet the guy responsible for building the Call of Duty game engine

2024-11-1521:46

A student of Geoff Hinton, Yann LeCun, and Jeff Dean explains where AI is headed

A student of Geoff Hinton, Yann LeCun, and Jeff Dean explains where AI is headed

2024-11-1234:52

One of the world’s biggest web scrapers has some thoughts on data ownership

One of the world’s biggest web scrapers has some thoughts on data ownership

2024-11-0834:02

How Google is helping developers get better answers from AI

How Google is helping developers get better answers from AI

2024-11-0525:40

How a creator of React is rethinking IDEs

How a creator of React is rethinking IDEs

2024-11-0124:36

Life in the Fastlane: SDK tools built with developers in mind

Life in the Fastlane: SDK tools built with developers in mind

2024-10-3031:06

How can you get your kids into coding? We asked an 8-year-old app builder.

How can you get your kids into coding? We asked an 8-year-old app builder.

2024-10-2923:00

Tragedy of the (data) commons

Tragedy of the (data) commons

2024-10-2530:36

loading
Google Play
Download from Google Play
Castbox
Download from App Store
usUnited States
00:00
00:00
x

0.5x

0.8x

1.0x

1.25x

1.5x

2.0x

3.0x

Sleep Timer

Off

End of Episode

5 Minutes

10 Minutes

15 Minutes

30 Minutes

45 Minutes

60 Minutes

120 Minutes

How Stack Overflow fends off scraping bots

How Stack Overflow fends off scraping bots

Ryan Donovan, Josh Zhang, Eira May