Data Tinkerer

Data Tinkerer

Share this post

Data Tinkerer
Data Tinkerer
Netflix's Secret to Cloud Efficiency
Data Engineering

Netflix's Secret to Cloud Efficiency

How Netflix uses data to reduce its cloud costs

Data Tinkerer's avatar
Data Tinkerer
Dec 19, 2024
∙ Paid
1

Share this post

Data Tinkerer
Data Tinkerer
Netflix's Secret to Cloud Efficiency
1
Share
black laptop computer on red table
Photo by Juraj Gabriel on Unsplash

TL;DR


Situation

Netflix relies on AWS for its cloud infrastructure, supporting a vast array of applications and services. This extensive ecosystem generates large volumes of data, necessitating efficient resource management to optimize costs and performance.

Task

The Platform Data Science and Engineering (DSE) team was asked to build a tool that could show engineers how much cloud resources they were using, how efficient they were being, and how much it was costing. The goal was to help teams make smarter and more cost-effective decisions.

Action

The team developed a two-component solution:

  1. Foundational Platform Data (FPD): This centralized data layer offers a consistent data model and standardized processing methods by establishing data contracts with various platform data providers. This ensures data quality and reliability across the board.

  2. Cloud Efficiency Analytics (CEA): Built atop FPD, CEA delivers time-series efficiency metrics tailored to diverse business needs. It provides detailed cost and ownership attribution, enabling teams to understand their resource consumption and associated expenses.

Result

Netflix’s teams now have access to clear and trustworthy data about their cloud usage and costs. This helps them make better decisions about how to use resources, which improves efficiency and saves money.

Use Cases

Cloud Resource Usage, Cost Attribution, Efficiency Analysis

Tech Stack/Framework

Apache Spark, Data Contracts


Explained Further


The Problem: Managing Complexity in the Cloud

This post is for paid subscribers

Already a paid subscriber? Sign in
© 2025 Data Tinkerer
Privacy ∙ Terms ∙ Collection notice
Start writingGet the app
Substack is the home for great culture

Share