DiscoverSoftware Engineering DailyMetaflow: Netflix Machine Learning Platform with Savin Goyal
Metaflow: Netflix Machine Learning Platform with Savin Goyal

Metaflow: Netflix Machine Learning Platform with Savin Goyal

Update: 2020-07-132
Share

Description


Netflix runs all of its infrastructure on Amazon Web Services. This includes business logic, data infrastructure, and machine learning. By tightly coupling itself to AWS, Netflix has been able to move faster and have strong defaults about engineering decisions. And today, AWS has such an expanse of services that it can be used as a platform to build custom tools.


Metaflow is an open source machine learning platform built on top of AWS that allows engineers at Netflix to build directed acyclic graphs for training models. These DAGs get deployed to AWS as Step Functions, a serverless orchestration platform.


Savin Goyal is a machine learning engineer with Netflix, and he joins the show to talk about the machine learning challenges within Netflix, and his experience working on Metaflow. We also talk about DAG systems such as AWS Step Functions and Airflow.


Sponsorship inquiries: sponsor@softwareengineeringdaily.com


The post Metaflow: Netflix Machine Learning Platform with Savin Goyal appeared first on Software Engineering Daily.

Comments 
00:00
00:00
x

0.5x

0.8x

1.0x

1.25x

1.5x

2.0x

3.0x

Sleep Timer

Off

End of Episode

5 Minutes

10 Minutes

15 Minutes

30 Minutes

45 Minutes

60 Minutes

120 Minutes

Metaflow: Netflix Machine Learning Platform with Savin Goyal

Metaflow: Netflix Machine Learning Platform with Savin Goyal

Software Engineering Daily