DiscoverThe Backend Engineering Show with Hussein NasserHow Apache Kafka got faster by switching ext4 to XFS
How Apache Kafka got faster by switching ext4 to XFS

How Apache Kafka got faster by switching ext4 to XFS

Update: 2024-04-29
Share

Description

https://oscourse.win




Allegro improved their Kafka produce tail latency by over 80% when they switched from ext4 to xfs. What I enjoyed most about this article is the detailed analysis and tweaking the team made to ext4 before considering switching to xfs. This is a classic case of how a good tech blog looks like in my opinion. 




0:00 Intro


0:30 Summary


2:35 How Kafka Works?


5:00 Producers Writes are Slow


7:10 Tracing Kafka Protocol 


12:00 Tracing Kernel System Calls


16:00 Journaled File Systems


21:00 Improving ext4


26:00 Switching to XFS






Blog


https://blog.allegro.tech/2024/03/kafka-performance-analysis.html





Comments 
loading
00:00
00:00
x

0.5x

0.8x

1.0x

1.25x

1.5x

2.0x

3.0x

Sleep Timer

Off

End of Episode

5 Minutes

10 Minutes

15 Minutes

30 Minutes

45 Minutes

60 Minutes

120 Minutes

How Apache Kafka got faster by switching ext4 to XFS

How Apache Kafka got faster by switching ext4 to XFS

Hussein Nasser