<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:itunes="http://www.itunes.com/dtds/podcast-1.0.dtd" xmlns:googleplay="http://www.google.com/schemas/play-podcasts/1.0"><channel><title><![CDATA[Data Tinkerer: Data Engineering]]></title><description><![CDATA[Dive into the latest trends and updates in data engineering!]]></description><link>https://www.datatinkerer.io/s/data-engineering</link><image><url>https://substackcdn.com/image/fetch/$s_!JEdj!,w_256,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F6bea5ccd-f356-4154-a1db-16268300510e_500x500.png</url><title>Data Tinkerer: Data Engineering</title><link>https://www.datatinkerer.io/s/data-engineering</link></image><generator>Substack</generator><lastBuildDate>Fri, 20 Nov 2026 04:57:19 GMT</lastBuildDate><atom:link href="https://www.datatinkerer.io/feed" rel="self" type="application/rss+xml"/><copyright><![CDATA[Data Tinkerer]]></copyright><language><![CDATA[en]]></language><webMaster><![CDATA[datatinkerer@substack.com]]></webMaster><itunes:owner><itunes:email><![CDATA[datatinkerer@substack.com]]></itunes:email><itunes:name><![CDATA[Data Tinkerer]]></itunes:name></itunes:owner><itunes:author><![CDATA[Data Tinkerer]]></itunes:author><googleplay:owner><![CDATA[datatinkerer@substack.com]]></googleplay:owner><googleplay:email><![CDATA[datatinkerer@substack.com]]></googleplay:email><googleplay:author><![CDATA[Data Tinkerer]]></googleplay:author><itunes:block><![CDATA[Yes]]></itunes:block><item><title><![CDATA[How Notion Cut Data Team Requests by 80% With an Internal AI Analyst]]></title><description><![CDATA[Data Scout answers routine questions in minutes using per-user Snowflake access, maintained metric definitions and query checks.]]></description><link>https://www.datatinkerer.io/p/how-notion-cut-data-team-requests-by-80-percent</link><guid isPermaLink="false">https://www.datatinkerer.io/p/how-notion-cut-data-team-requests-by-80-percent</guid><dc:creator><![CDATA[Data Tinkerer]]></dc:creator><pubDate>Thu, 12 Nov 2026 03:15:30 GMT</pubDate><enclosure url="https://images.unsplash.com/photo-1651497634590-ce388a6a686a?crop=entropy&amp;cs=tinysrgb&amp;fit=max&amp;fm=jpg&amp;ixid=M3wzMDAzMzh8MHwxfHNlYXJjaHw4fHxub3Rpb258ZW58MHx8fHwxNzk0MTI2Nzk2fDA&amp;ixlib=rb-4.1.0&amp;q=80&amp;w=1080" length="0" type="image/jpeg"/><content:encoded><![CDATA[<p>Fellow Data Tinkerers!</p><p>An account manager preparing for a renewal wants to know whether more people at a customer company are using AI. Suppose 8 people used it in one workspace and 6 used it in another, with two people in both. Adding the workspace counts gives 14. The account has 12 distinct users. Either number can come from SQL without running into an error.</p><p>To answer these types of questions, Notion has built an internal agent called Data Scout. Employees ask in plain language, the agent queries governed data in Snowflake and uses company knowledge in Notion to interpret the answer. Within two months, weekly requests to its data team fell by about 80%. Roughly half of its employees now use Data Scout.</p><p>Now before we dive into it, just wanted to say referring <strong>1 person</strong> to Data Tinkerer gives you access to a library of data resources:</p><div class="captioned-image-container"><figure><a class="image-link image2 is-viewable-img" target="_blank" href="https://substackcdn.com/image/fetch/$s_!nlH_!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F1012f149-167a-467e-8a85-8b481beb98bc_800x402.gif" data-component-name="Image2ToDOM"><div class="image2-inset"><picture><source type="image/webp" srcset="https://substackcdn.com/image/fetch/$s_!nlH_!,w_424,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F1012f149-167a-467e-8a85-8b481beb98bc_800x402.gif 424w, https://substackcdn.com/image/fetch/$s_!nlH_!,w_848,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F1012f149-167a-467e-8a85-8b481beb98bc_800x402.gif 848w, https://substackcdn.com/image/fetch/$s_!nlH_!,w_1272,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F1012f149-167a-467e-8a85-8b481beb98bc_800x402.gif 1272w, https://substackcdn.com/image/fetch/$s_!nlH_!,w_1456,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F1012f149-167a-467e-8a85-8b481beb98bc_800x402.gif 1456w" sizes="100vw"><img src="https://substackcdn.com/image/fetch/$s_!nlH_!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F1012f149-167a-467e-8a85-8b481beb98bc_800x402.gif" width="800" height="402" data-attrs="{&quot;src&quot;:&quot;https://substack-post-media.s3.amazonaws.com/public/images/1012f149-167a-467e-8a85-8b481beb98bc_800x402.gif&quot;,&quot;srcNoWatermark&quot;:null,&quot;fullscreen&quot;:null,&quot;imageSize&quot;:null,&quot;height&quot;:402,&quot;width&quot;:800,&quot;resizeWidth&quot;:null,&quot;bytes&quot;:3369150,&quot;alt&quot;:null,&quot;title&quot;:null,&quot;type&quot;:&quot;image/gif&quot;,&quot;href&quot;:null,&quot;belowTheFold&quot;:false,&quot;topImage&quot;:true,&quot;internalRedirect&quot;:&quot;https://www.datatinkerer.io/i/223984118?img=https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F1012f149-167a-467e-8a85-8b481beb98bc_800x402.gif&quot;,&quot;isProcessing&quot;:false,&quot;align&quot;:null,&quot;offset&quot;:false}" class="sizing-normal" alt="" srcset="https://substackcdn.com/image/fetch/$s_!nlH_!,w_424,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F1012f149-167a-467e-8a85-8b481beb98bc_800x402.gif 424w, https://substackcdn.com/image/fetch/$s_!nlH_!,w_848,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F1012f149-167a-467e-8a85-8b481beb98bc_800x402.gif 848w, https://substackcdn.com/image/fetch/$s_!nlH_!,w_1272,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F1012f149-167a-467e-8a85-8b481beb98bc_800x402.gif 1272w, https://substackcdn.com/image/fetch/$s_!nlH_!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F1012f149-167a-467e-8a85-8b481beb98bc_800x402.gif 1456w" sizes="100vw" fetchpriority="high"></picture><div class="image-link-expand"><div class="pencraft pc-display-flex pc-gap-8 pc-reset"><button class="pencraft pc-reset pencraft icon-container restack-image buttonBase-tqxFUX" tabindex="0" type="button"><svg aria-hidden="true" width="20" height="20" viewBox="0 0 20 20" fill="none" stroke-width="1.5" stroke="var(--color-fg-primary)" stroke-linecap="round" stroke-linejoin="round" xmlns="http://www.w3.org/2000/svg" class="icon-noB79L"><g><path d="M2.53001 7.81595C3.49179 4.73911 6.43281 2.5 9.91173 2.5C13.1684 2.5 15.9537 4.46214 17.0852 7.23684L17.6179 8.67647M17.6179 8.67647L18.5002 4.26471M17.6179 8.67647L13.6473 6.91176M17.4995 12.1841C16.5378 15.2609 13.5967 17.5 10.1178 17.5C6.86118 17.5 4.07589 15.5379 2.94432 12.7632L2.41165 11.3235M2.41165 11.3235L1.5293 15.7353M2.41165 11.3235L6.38224 13.0882"></path></g></svg></button><button class="pencraft pc-reset pencraft icon-container view-image buttonBase-tqxFUX" tabindex="0" type="button"><svg xmlns="http://www.w3.org/2000/svg" width="20" height="20" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" class="lucide lucide-maximize2 lucide-maximize-2 icon-noB79L"><polyline points="15 3 21 3 21 9"></polyline><polyline points="9 21 3 21 3 15"></polyline><line x1="21" x2="14" y1="3" y2="10"></line><line x1="3" x2="10" y1="21" y2="14"></line></svg></button></div></div></div></a></figure></div><p>There are 100+ resources to learn all things data (science, engineering, analysis). It includes videos, courses, projects and can be filtered by tech stack (Python, SQL, Spark and etc), skill level (Beginner, Intermediate and so on) provider name or free/paid. So if you know other people who like staying up to date on all things data, please share Data Tinkerer with them!</p><p class="button-wrapper" data-attrs="{&quot;url&quot;:&quot;https://www.datatinkerer.io/leaderboard?&amp;utm_source=post&quot;,&quot;text&quot;:&quot;Refer a friend&quot;,&quot;action&quot;:null,&quot;class&quot;:null}" data-component-name="ButtonCreateButton"><a class="button primary" href="https://www.datatinkerer.io/leaderboard?&amp;utm_source=post"><span>Refer a friend</span></a></p><p>Now, with that out of the way, let&#8217;s get into how Notion created its own AI analyst.</p><div class="captioned-image-container"><figure><a class="image-link image2 is-viewable-img" target="_blank" href="https://images.unsplash.com/photo-1651497634590-ce388a6a686a?crop=entropy&amp;cs=tinysrgb&amp;fit=max&amp;fm=jpg&amp;ixid=M3wzMDAzMzh8MHwxfHNlYXJjaHw4fHxub3Rpb258ZW58MHx8fHwxNzk0MTI2Nzk2fDA&amp;ixlib=rb-4.1.0&amp;q=80&amp;w=1080" data-component-name="Image2ToDOM"><div class="image2-inset"><picture><source type="image/webp" srcset="https://images.unsplash.com/photo-1651497634590-ce388a6a686a?crop=entropy&amp;cs=tinysrgb&amp;fit=max&amp;fm=jpg&amp;ixid=M3wzMDAzMzh8MHwxfHNlYXJjaHw4fHxub3Rpb258ZW58MHx8fHwxNzk0MTI2Nzk2fDA&amp;ixlib=rb-4.1.0&amp;q=80&amp;w=1080 424w, https://images.unsplash.com/photo-1651497634590-ce388a6a686a?crop=entropy&amp;cs=tinysrgb&amp;fit=max&amp;fm=jpg&amp;ixid=M3wzMDAzMzh8MHwxfHNlYXJjaHw4fHxub3Rpb258ZW58MHx8fHwxNzk0MTI2Nzk2fDA&amp;ixlib=rb-4.1.0&amp;q=80&amp;w=1080 848w, https://images.unsplash.com/photo-1651497634590-ce388a6a686a?crop=entropy&amp;cs=tinysrgb&amp;fit=max&amp;fm=jpg&amp;ixid=M3wzMDAzMzh8MHwxfHNlYXJjaHw4fHxub3Rpb258ZW58MHx8fHwxNzk0MTI2Nzk2fDA&amp;ixlib=rb-4.1.0&amp;q=80&amp;w=1080 1272w, https://images.unsplash.com/photo-1651497634590-ce388a6a686a?crop=entropy&amp;cs=tinysrgb&amp;fit=max&amp;fm=jpg&amp;ixid=M3wzMDAzMzh8MHwxfHNlYXJjaHw4fHxub3Rpb258ZW58MHx8fHwxNzk0MTI2Nzk2fDA&amp;ixlib=rb-4.1.0&amp;q=80&amp;w=1080 1456w" sizes="100vw"><img src="https://images.unsplash.com/photo-1651497634590-ce388a6a686a?crop=entropy&amp;cs=tinysrgb&amp;fit=max&amp;fm=jpg&amp;ixid=M3wzMDAzMzh8MHwxfHNlYXJjaHw4fHxub3Rpb258ZW58MHx8fHwxNzk0MTI2Nzk2fDA&amp;ixlib=rb-4.1.0&amp;q=80&amp;w=1080" width="2160" height="3840" data-attrs="{&quot;src&quot;:&quot;https://images.unsplash.com/photo-1651497634590-ce388a6a686a?crop=entropy&amp;cs=tinysrgb&amp;fit=max&amp;fm=jpg&amp;ixid=M3wzMDAzMzh8MHwxfHNlYXJjaHw4fHxub3Rpb258ZW58MHx8fHwxNzk0MTI2Nzk2fDA&amp;ixlib=rb-4.1.0&amp;q=80&amp;w=1080&quot;,&quot;srcNoWatermark&quot;:null,&quot;fullscreen&quot;:null,&quot;imageSize&quot;:null,&quot;height&quot;:3840,&quot;width&quot;:2160,&quot;resizeWidth&quot;:null,&quot;bytes&quot;:null,&quot;alt&quot;:&quot;diagram&quot;,&quot;title&quot;:null,&quot;type&quot;:&quot;image/jpg&quot;,&quot;href&quot;:null,&quot;belowTheFold&quot;:false,&quot;topImage&quot;:false,&quot;internalRedirect&quot;:null,&quot;isProcessing&quot;:false,&quot;align&quot;:null,&quot;offset&quot;:false}" class="sizing-normal" alt="diagram" title="diagram" srcset="https://images.unsplash.com/photo-1651497634590-ce388a6a686a?crop=entropy&amp;cs=tinysrgb&amp;fit=max&amp;fm=jpg&amp;ixid=M3wzMDAzMzh8MHwxfHNlYXJjaHw4fHxub3Rpb258ZW58MHx8fHwxNzk0MTI2Nzk2fDA&amp;ixlib=rb-4.1.0&amp;q=80&amp;w=1080 424w, https://images.unsplash.com/photo-1651497634590-ce388a6a686a?crop=entropy&amp;cs=tinysrgb&amp;fit=max&amp;fm=jpg&amp;ixid=M3wzMDAzMzh8MHwxfHNlYXJjaHw4fHxub3Rpb258ZW58MHx8fHwxNzk0MTI2Nzk2fDA&amp;ixlib=rb-4.1.0&amp;q=80&amp;w=1080 848w, https://images.unsplash.com/photo-1651497634590-ce388a6a686a?crop=entropy&amp;cs=tinysrgb&amp;fit=max&amp;fm=jpg&amp;ixid=M3wzMDAzMzh8MHwxfHNlYXJjaHw4fHxub3Rpb258ZW58MHx8fHwxNzk0MTI2Nzk2fDA&amp;ixlib=rb-4.1.0&amp;q=80&amp;w=1080 1272w, https://images.unsplash.com/photo-1651497634590-ce388a6a686a?crop=entropy&amp;cs=tinysrgb&amp;fit=max&amp;fm=jpg&amp;ixid=M3wzMDAzMzh8MHwxfHNlYXJjaHw4fHxub3Rpb258ZW58MHx8fHwxNzk0MTI2Nzk2fDA&amp;ixlib=rb-4.1.0&amp;q=80&amp;w=1080 1456w" sizes="100vw"></picture><div class="image-link-expand"><div class="pencraft pc-display-flex pc-gap-8 pc-reset"><button class="pencraft pc-reset pencraft icon-container restack-image buttonBase-tqxFUX" tabindex="0" type="button"><svg aria-hidden="true" width="20" height="20" viewBox="0 0 20 20" fill="none" stroke-width="1.5" stroke="var(--color-fg-primary)" stroke-linecap="round" stroke-linejoin="round" xmlns="http://www.w3.org/2000/svg" class="icon-noB79L"><g><path d="M2.53001 7.81595C3.49179 4.73911 6.43281 2.5 9.91173 2.5C13.1684 2.5 15.9537 4.46214 17.0852 7.23684L17.6179 8.67647M17.6179 8.67647L18.5002 4.26471M17.6179 8.67647L13.6473 6.91176M17.4995 12.1841C16.5378 15.2609 13.5967 17.5 10.1178 17.5C6.86118 17.5 4.07589 15.5379 2.94432 12.7632L2.41165 11.3235M2.41165 11.3235L1.5293 15.7353M2.41165 11.3235L6.38224 13.0882"></path></g></svg></button><button class="pencraft pc-reset pencraft icon-container view-image buttonBase-tqxFUX" tabindex="0" type="button"><svg xmlns="http://www.w3.org/2000/svg" width="20" height="20" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" class="lucide lucide-maximize2 lucide-maximize-2 icon-noB79L"><polyline points="15 3 21 3 21 9"></polyline><polyline points="9 21 3 21 3 15"></polyline><line x1="21" x2="14" y1="3" y2="10"></line><line x1="3" x2="10" y1="21" y2="14"></line></svg></button></div></div></div></a><figcaption class="image-caption">Photo by <a href="https://unsplash.com/@jonnyuiux">Jonny Caspari</a> on <a href="https://unsplash.com">Unsplash</a></figcaption></figure></div><h3>TL;DR</h3><div><hr></div><h4><strong>Situation</strong></h4><p>Routine questions could spend days in the data team&#8217;s Slack queue.</p><h4>Task</h4><p>Let employees investigate approved data themselves while preserving their Snowflake permissions and making the agent&#8217;s choice of metric, source and interpretation reviewable.</p><h4>Action</h4><p>Notion connected Data Scout to Snowflake under each employee&#8217;s identity, indexed approved guidance and added query tracing and nightly review.</p><h4>Result</h4><p>Notion reports about 80% fewer weekly requests to the data team after two months, questions answered in minutes rather than waiting days and used by about half of employees.</p><h4><strong>Use Cases</strong></h4><p>Data quality validation, internal support bot, automated BI reporting</p><p><strong>Tech Stack/Framework</strong></p><p>Snowflake, SQL, Slack, GitHub</p><div><hr></div><h3>Explained further</h3><div><hr></div><h4>One question, several kinds of evidence</h4><p>Notion sampled 200 real questions and found uses across Sales, Engineering, Customer Experience, Product and Marketing. In a GTM example, an account manager preparing for a renewal asks whether AI adoption is growing in a customer&#8217;s account.</p><p>Snowflake can provide usage history. The account notes and past meetings in Notion can explain the customer&#8217;s rollout plan. If usage rises, the account manager may ask whether more people started using AI or whether existing users became more active. A subsequent question might split the trend by team or workspace. The agent has to carry the account scope and the chosen definition through those follow-ups.</p><div class="captioned-image-container"><figure><a class="image-link image2 is-viewable-img" target="_blank" href="https://substackcdn.com/image/fetch/$s_!5Dd4!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F9dace2a2-e82f-473c-b567-9d9ca8d40247_3840x2160.webp" data-component-name="Image2ToDOM"><div class="image2-inset"><picture><source type="image/webp" srcset="https://substackcdn.com/image/fetch/$s_!5Dd4!,w_424,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F9dace2a2-e82f-473c-b567-9d9ca8d40247_3840x2160.webp 424w, https://substackcdn.com/image/fetch/$s_!5Dd4!,w_848,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F9dace2a2-e82f-473c-b567-9d9ca8d40247_3840x2160.webp 848w, https://substackcdn.com/image/fetch/$s_!5Dd4!,w_1272,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F9dace2a2-e82f-473c-b567-9d9ca8d40247_3840x2160.webp 1272w, https://substackcdn.com/image/fetch/$s_!5Dd4!,w_1456,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F9dace2a2-e82f-473c-b567-9d9ca8d40247_3840x2160.webp 1456w" sizes="auto, 100vw"><img src="https://substackcdn.com/image/fetch/$s_!5Dd4!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F9dace2a2-e82f-473c-b567-9d9ca8d40247_3840x2160.webp" width="1456" height="819" data-attrs="{&quot;src&quot;:&quot;https://substack-post-media.s3.amazonaws.com/public/images/9dace2a2-e82f-473c-b567-9d9ca8d40247_3840x2160.webp&quot;,&quot;srcNoWatermark&quot;:null,&quot;fullscreen&quot;:null,&quot;imageSize&quot;:null,&quot;height&quot;:819,&quot;width&quot;:1456,&quot;resizeWidth&quot;:null,&quot;bytes&quot;:92832,&quot;alt&quot;:null,&quot;title&quot;:null,&quot;type&quot;:&quot;image/webp&quot;,&quot;href&quot;:null,&quot;belowTheFold&quot;:true,&quot;topImage&quot;:false,&quot;internalRedirect&quot;:&quot;https://www.datatinkerer.io/i/223984118?img=https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F9dace2a2-e82f-473c-b567-9d9ca8d40247_3840x2160.webp&quot;,&quot;isProcessing&quot;:false,&quot;align&quot;:null,&quot;offset&quot;:false}" class="sizing-normal" alt="" srcset="https://substackcdn.com/image/fetch/$s_!5Dd4!,w_424,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F9dace2a2-e82f-473c-b567-9d9ca8d40247_3840x2160.webp 424w, https://substackcdn.com/image/fetch/$s_!5Dd4!,w_848,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F9dace2a2-e82f-473c-b567-9d9ca8d40247_3840x2160.webp 848w, https://substackcdn.com/image/fetch/$s_!5Dd4!,w_1272,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F9dace2a2-e82f-473c-b567-9d9ca8d40247_3840x2160.webp 1272w, https://substackcdn.com/image/fetch/$s_!5Dd4!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F9dace2a2-e82f-473c-b567-9d9ca8d40247_3840x2160.webp 1456w" sizes="auto, 100vw" loading="lazy"></picture><div class="image-link-expand"><div class="pencraft pc-display-flex pc-gap-8 pc-reset"><button class="pencraft pc-reset pencraft icon-container restack-image buttonBase-tqxFUX" tabindex="0" type="button"><svg aria-hidden="true" width="20" height="20" viewBox="0 0 20 20" fill="none" stroke-width="1.5" stroke="var(--color-fg-primary)" stroke-linecap="round" stroke-linejoin="round" xmlns="http://www.w3.org/2000/svg" class="icon-noB79L"><g><path d="M2.53001 7.81595C3.49179 4.73911 6.43281 2.5 9.91173 2.5C13.1684 2.5 15.9537 4.46214 17.0852 7.23684L17.6179 8.67647M17.6179 8.67647L18.5002 4.26471M17.6179 8.67647L13.6473 6.91176M17.4995 12.1841C16.5378 15.2609 13.5967 17.5 10.1178 17.5C6.86118 17.5 4.07589 15.5379 2.94432 12.7632L2.41165 11.3235M2.41165 11.3235L1.5293 15.7353M2.41165 11.3235L6.38224 13.0882"></path></g></svg></button><button class="pencraft pc-reset pencraft icon-container view-image buttonBase-tqxFUX" tabindex="0" type="button"><svg xmlns="http://www.w3.org/2000/svg" width="20" height="20" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" class="lucide lucide-maximize2 lucide-maximize-2 icon-noB79L"><polyline points="15 3 21 3 21 9"></polyline><polyline points="9 21 3 21 3 15"></polyline><line x1="21" x2="14" y1="3" y2="10"></line><line x1="3" x2="10" y1="21" y2="14"></line></svg></button></div></div></div></a><figcaption class="image-caption"><em>Data Scout combines governed Snowflake data with company knowledge in Notion. (Source: Notion)</em></figcaption></figure></div><p>Data Scout Reporter uses saved report definitions to produce scheduled reports in Notion and Slack. Teams have also built specialised agents that connect to Data Scout over its shared MCP connection, reusing its data access and context while adding domain-specific guidance. Scheduled reporting requires its own approved access model.</p><h4>The permission boundary follows the person</h4><p>There are two permission systems here. Notion controls who can access the agent, its pages and its available tools. Snowflake controls what the authenticated person can query. Each employee signs in to the Snowflake MCP connection using their own Snowflake credentials, so the query executes under that identity and the relevant role and grants. Masking and row-level policies apply where configured.</p><p>If a query is denied under the person&#8217;s Snowflake identity, the agent should report the access limitation. A person may also be allowed to see a result that should not be pasted into a broad Slack channel or a widely shared report.</p><h4>Find the definition before the table</h4><p>An LLM with a warehouse connection can browse schemas and produce executable SQL without establishing whether workspace memberships are an acceptable proxy for people, which account mapping is authoritative, or whether a daily snapshot should be summed. Notion indexes Skills, Metric Definitions, Table Schemas, FAQs and known data changes in a Notion database for Data Scout to consult.</p><p>The row properties act as retrieval metadata: title, domain, type, description or keywords, and priority. The page body contains the procedure, calculation, caveats and links. A Skill can say how to investigate account adoption before a renewal; a Metric Definition specifies the numerator, denominator and exclusions; a Table Schema gives grain, relationships and join cautions. When a definition is maintained in another system, the page links to it instead of creating a second version.</p><p>The agent begins with one or two likely domains and relevant Skills. An exact high-priority entry can answer a point lookup. Otherwise it opens the selected pages, follows their references and expands its search only if guidance is missing. It does not load the entire library into every conversation or choose a table before deciding what the question means.</p><div class="captioned-image-container"><figure><a class="image-link image2 is-viewable-img" target="_blank" href="https://substackcdn.com/image/fetch/$s_!CPqT!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F23f3751f-40b5-4ba0-af37-f158fb4e1db1_3840x2160.webp" data-component-name="Image2ToDOM"><div class="image2-inset"><picture><source type="image/webp" srcset="https://substackcdn.com/image/fetch/$s_!CPqT!,w_424,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F23f3751f-40b5-4ba0-af37-f158fb4e1db1_3840x2160.webp 424w, https://substackcdn.com/image/fetch/$s_!CPqT!,w_848,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F23f3751f-40b5-4ba0-af37-f158fb4e1db1_3840x2160.webp 848w, https://substackcdn.com/image/fetch/$s_!CPqT!,w_1272,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F23f3751f-40b5-4ba0-af37-f158fb4e1db1_3840x2160.webp 1272w, https://substackcdn.com/image/fetch/$s_!CPqT!,w_1456,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F23f3751f-40b5-4ba0-af37-f158fb4e1db1_3840x2160.webp 1456w" sizes="auto, 100vw"><img src="https://substackcdn.com/image/fetch/$s_!CPqT!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F23f3751f-40b5-4ba0-af37-f158fb4e1db1_3840x2160.webp" width="1456" height="819" data-attrs="{&quot;src&quot;:&quot;https://substack-post-media.s3.amazonaws.com/public/images/23f3751f-40b5-4ba0-af37-f158fb4e1db1_3840x2160.webp&quot;,&quot;srcNoWatermark&quot;:null,&quot;fullscreen&quot;:null,&quot;imageSize&quot;:null,&quot;height&quot;:819,&quot;width&quot;:1456,&quot;resizeWidth&quot;:null,&quot;bytes&quot;:71998,&quot;alt&quot;:null,&quot;title&quot;:null,&quot;type&quot;:&quot;image/webp&quot;,&quot;href&quot;:null,&quot;belowTheFold&quot;:true,&quot;topImage&quot;:false,&quot;internalRedirect&quot;:&quot;https://www.datatinkerer.io/i/223984118?img=https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F23f3751f-40b5-4ba0-af37-f158fb4e1db1_3840x2160.webp&quot;,&quot;isProcessing&quot;:false,&quot;align&quot;:null,&quot;offset&quot;:false}" class="sizing-normal" alt="" srcset="https://substackcdn.com/image/fetch/$s_!CPqT!,w_424,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F23f3751f-40b5-4ba0-af37-f158fb4e1db1_3840x2160.webp 424w, https://substackcdn.com/image/fetch/$s_!CPqT!,w_848,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F23f3751f-40b5-4ba0-af37-f158fb4e1db1_3840x2160.webp 848w, https://substackcdn.com/image/fetch/$s_!CPqT!,w_1272,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F23f3751f-40b5-4ba0-af37-f158fb4e1db1_3840x2160.webp 1272w, https://substackcdn.com/image/fetch/$s_!CPqT!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F23f3751f-40b5-4ba0-af37-f158fb4e1db1_3840x2160.webp 1456w" sizes="auto, 100vw" loading="lazy"></picture><div class="image-link-expand"><div class="pencraft pc-display-flex pc-gap-8 pc-reset"><button class="pencraft pc-reset pencraft icon-container restack-image buttonBase-tqxFUX" tabindex="0" type="button"><svg aria-hidden="true" width="20" height="20" viewBox="0 0 20 20" fill="none" stroke-width="1.5" stroke="var(--color-fg-primary)" stroke-linecap="round" stroke-linejoin="round" xmlns="http://www.w3.org/2000/svg" class="icon-noB79L"><g><path d="M2.53001 7.81595C3.49179 4.73911 6.43281 2.5 9.91173 2.5C13.1684 2.5 15.9537 4.46214 17.0852 7.23684L17.6179 8.67647M17.6179 8.67647L18.5002 4.26471M17.6179 8.67647L13.6473 6.91176M17.4995 12.1841C16.5378 15.2609 13.5967 17.5 10.1178 17.5C6.86118 17.5 4.07589 15.5379 2.94432 12.7632L2.41165 11.3235M2.41165 11.3235L1.5293 15.7353M2.41165 11.3235L6.38224 13.0882"></path></g></svg></button><button class="pencraft pc-reset pencraft icon-container view-image buttonBase-tqxFUX" tabindex="0" type="button"><svg xmlns="http://www.w3.org/2000/svg" width="20" height="20" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" class="lucide lucide-maximize2 lucide-maximize-2 icon-noB79L"><polyline points="15 3 21 3 21 9"></polyline><polyline points="9 21 3 21 3 15"></polyline><line x1="21" x2="14" y1="3" y2="10"></line><line x1="3" x2="10" y1="21" y2="14"></line></svg></button></div></div></div></a><figcaption class="image-caption"><em>Data Scout matches a question to a context entry, then follows its references to the definitions and tools it needs. (Source: Notion)</em></figcaption></figure></div><p>Notion tried putting all of the guidance into database properties for quicker retrieval. The metadata became overloaded and harder for people to maintain, so properties now locate a page; they do not substitute for reading its body. Priority is a retrieval hint, not proof that an entry is approved or current. Ownership, review status and freshness still need to be managed.</p><p>Data context determines the calculation: source, metric, grain, required filters and freshness. Business context helps explain a result through account plans, rollout notes, incidents or decisions. Routing guidance directs the agent to the appropriate connection and sources, potentially across more than one platform. Notion&#8217;s instructions keep business context from redefining the approved metric calculation.</p><p>The page library does not enforce metric definitions at query time or replace a semantic layer. Notion acknowledges that limitation and says teams can keep definitions in existing tooling, including Snowflake Semantic Views or Cortex Analyst while linking them to business context in Notion.</p><h4>What happens between retrieval and an answer</h4><p>Notion&#8217;s agent instructions require it to resolve the metric, account or population, period and comparison before calculating. Then it routes to the approved connection and source and checks the query logic and returned data for join cardinality, filters, denominators, incomplete periods, tracking changes and backfills.</p><p>For the renewal question, these checks catch different mistakes:</p><ul><li><p>If one person belongs to several workspaces, joining account and workspace data may multiply that person&#8217;s activity. A syntactically valid <code>COUNT</code> is still wrong for a people-based adoption metric.</p></li><li><p>If the numerator and denominator use different populations or exclusion rules, the rate is meaningless even if both counts are individually correct.</p></li><li><p>If a daily snapshot has one row per entity per day, summing several days double-counts a current-state population. A single as-of date answers a current-state question; a date range belongs in an explicit trend.</p></li><li><p>If the latest period is incomplete or an event was renamed, the apparent change may reflect data collection rather than customer behaviour.</p></li></ul><p>The discovery instructions also cover failure handling. When a necessary column is undocumented, inspect the schema. When Snowflake returns an invalid identifier, verify the actual schema rather than guessing another plausible column. When the loaded guidance already documents the fields, avoid an unnecessary wide schema retrieval. If a definition is missing, the query is denied, or a check fails, the agent should say what it cannot establish.</p><div class="captioned-image-container"><figure><a class="image-link image2 is-viewable-img" target="_blank" href="https://substackcdn.com/image/fetch/$s_!mJxR!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F16ded9c3-530d-467f-a7ca-a86d5fa1b2c1_3840x2160.webp" data-component-name="Image2ToDOM"><div class="image2-inset"><picture><source type="image/webp" srcset="https://substackcdn.com/image/fetch/$s_!mJxR!,w_424,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F16ded9c3-530d-467f-a7ca-a86d5fa1b2c1_3840x2160.webp 424w, https://substackcdn.com/image/fetch/$s_!mJxR!,w_848,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F16ded9c3-530d-467f-a7ca-a86d5fa1b2c1_3840x2160.webp 848w, https://substackcdn.com/image/fetch/$s_!mJxR!,w_1272,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F16ded9c3-530d-467f-a7ca-a86d5fa1b2c1_3840x2160.webp 1272w, https://substackcdn.com/image/fetch/$s_!mJxR!,w_1456,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F16ded9c3-530d-467f-a7ca-a86d5fa1b2c1_3840x2160.webp 1456w" sizes="auto, 100vw"><img src="https://substackcdn.com/image/fetch/$s_!mJxR!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F16ded9c3-530d-467f-a7ca-a86d5fa1b2c1_3840x2160.webp" width="1456" height="819" data-attrs="{&quot;src&quot;:&quot;https://substack-post-media.s3.amazonaws.com/public/images/16ded9c3-530d-467f-a7ca-a86d5fa1b2c1_3840x2160.webp&quot;,&quot;srcNoWatermark&quot;:null,&quot;fullscreen&quot;:null,&quot;imageSize&quot;:null,&quot;height&quot;:819,&quot;width&quot;:1456,&quot;resizeWidth&quot;:null,&quot;bytes&quot;:105664,&quot;alt&quot;:null,&quot;title&quot;:null,&quot;type&quot;:&quot;image/webp&quot;,&quot;href&quot;:null,&quot;belowTheFold&quot;:true,&quot;topImage&quot;:false,&quot;internalRedirect&quot;:&quot;https://www.datatinkerer.io/i/223984118?img=https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F16ded9c3-530d-467f-a7ca-a86d5fa1b2c1_3840x2160.webp&quot;,&quot;isProcessing&quot;:false,&quot;align&quot;:null,&quot;offset&quot;:false}" class="sizing-normal" alt="" srcset="https://substackcdn.com/image/fetch/$s_!mJxR!,w_424,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F16ded9c3-530d-467f-a7ca-a86d5fa1b2c1_3840x2160.webp 424w, https://substackcdn.com/image/fetch/$s_!mJxR!,w_848,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F16ded9c3-530d-467f-a7ca-a86d5fa1b2c1_3840x2160.webp 848w, https://substackcdn.com/image/fetch/$s_!mJxR!,w_1272,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F16ded9c3-530d-467f-a7ca-a86d5fa1b2c1_3840x2160.webp 1272w, https://substackcdn.com/image/fetch/$s_!mJxR!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F16ded9c3-530d-467f-a7ca-a86d5fa1b2c1_3840x2160.webp 1456w" sizes="auto, 100vw" loading="lazy"></picture><div class="image-link-expand"><div class="pencraft pc-display-flex pc-gap-8 pc-reset"><button class="pencraft pc-reset pencraft icon-container restack-image buttonBase-tqxFUX" tabindex="0" type="button"><svg aria-hidden="true" width="20" height="20" viewBox="0 0 20 20" fill="none" stroke-width="1.5" stroke="var(--color-fg-primary)" stroke-linecap="round" stroke-linejoin="round" xmlns="http://www.w3.org/2000/svg" class="icon-noB79L"><g><path d="M2.53001 7.81595C3.49179 4.73911 6.43281 2.5 9.91173 2.5C13.1684 2.5 15.9537 4.46214 17.0852 7.23684L17.6179 8.67647M17.6179 8.67647L18.5002 4.26471M17.6179 8.67647L13.6473 6.91176M17.4995 12.1841C16.5378 15.2609 13.5967 17.5 10.1178 17.5C6.86118 17.5 4.07589 15.5379 2.94432 12.7632L2.41165 11.3235M2.41165 11.3235L1.5293 15.7353M2.41165 11.3235L6.38224 13.0882"></path></g></svg></button><button class="pencraft pc-reset pencraft icon-container view-image buttonBase-tqxFUX" tabindex="0" type="button"><svg xmlns="http://www.w3.org/2000/svg" width="20" height="20" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" class="lucide lucide-maximize2 lucide-maximize-2 icon-noB79L"><polyline points="15 3 21 3 21 9"></polyline><polyline points="9 21 3 21 3 15"></polyline><line x1="21" x2="14" y1="3" y2="10"></line><line x1="3" x2="10" y1="21" y2="14"></line></svg></button></div></div></div></a><figcaption class="image-caption"><em>Query metadata travels with the execution, and failed checks send the agent back to investigate. (Source: Notion)</em></figcaption></figure></div><p>Only after checking the measurement does Data Scout use account notes to discuss whether the change fits the customer&#8217;s planned rollout. It should distinguish &#8220;usage increased under this definition&#8221; from &#8220;the rollout caused the increase&#8221;. The latter is an interpretation and the account team still owns the renewal decision.</p><h4>An answer needs a trace</h4><p>Suppose two people ask about the same account and receive different adoption figures. Their Snowflake roles might legitimately expose different data. Or the agent may have selected different account mappings, time windows or definitions. A transcript containing only the polished answer does not make that easy to diagnose.</p><p>Notion built an internal execution tool called <code>run_tagged_query</code>. It accepts SQL and metadata together, appends the metadata as a structured JSON comment in the SQL and calls the upstream Snowflake query tool. The metadata can identify a canonical metric, definition reference, context pages, filters, time window and question ID. The wrapper also emits execution telemetry. A question ID can tie several investigative queries to one question; an execution ID identifies a particular run when returned.</p><p>A JSON comment in the SQL text does not set Snowflake&#8217;s native <code>QUERY_TAG</code> session parameter. The comment can be a searchable marker if the execution tool preserves it. To populate <code>QUERY_TAG</code>, the execution layer must set the parameter on the session that runs the SQL. Separate tool calls need not share a session, and pooled sessions must clear or replace a previous tag. Otherwise query-history attribution can be wrong. Notion&#8217;s guide also notes that Snowflake&#8217;s <code>AGENT_TYPE</code>, when the connection is recognised as agentic, gives coarse classification rather than question-level lineage.</p><p>This wrapper is Notion&#8217;s internal tool, not something automatically installed with a Snowflake MCP connection. Merely mentioning its name in an agent prompt will not tag direct Snowflake calls.</p><p>If a trace shows the correct metric definition but the wrong table, the routing guidance needs attention. Duplicated people point to the procedure or join guidance. An unsupported explanation of a rollout points to interpretation. Notion says its wrapper is lenient by default: incomplete metadata does not block an otherwise valid query. A successful call can therefore still leave gaps in the trace.</p><h4>Who updates the agent when the warehouse changes?</h4><p>Schemas, event names, backfills and business plans move. Notion runs a nightly review agent across Data Scout conversations and queries, recently edited context pages, relevant Slack and Notion material, and GitHub changes. It records missing, stale or conflicting guidance with supporting evidence in a review database. A separate triage agent deduplicates findings, applies small, well-supported and authorised edits, and sends ambiguous or higher-risk ones to a person. A weekly Slack summary surfaces patterns and unresolved findings.</p><div class="captioned-image-container"><figure><a class="image-link image2 is-viewable-img" target="_blank" href="https://substackcdn.com/image/fetch/$s_!4Zuc!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F703952df-9fc6-4760-8560-ccb341a58183_3840x2160.webp" data-component-name="Image2ToDOM"><div class="image2-inset"><picture><source type="image/webp" srcset="https://substackcdn.com/image/fetch/$s_!4Zuc!,w_424,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F703952df-9fc6-4760-8560-ccb341a58183_3840x2160.webp 424w, https://substackcdn.com/image/fetch/$s_!4Zuc!,w_848,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F703952df-9fc6-4760-8560-ccb341a58183_3840x2160.webp 848w, https://substackcdn.com/image/fetch/$s_!4Zuc!,w_1272,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F703952df-9fc6-4760-8560-ccb341a58183_3840x2160.webp 1272w, https://substackcdn.com/image/fetch/$s_!4Zuc!,w_1456,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F703952df-9fc6-4760-8560-ccb341a58183_3840x2160.webp 1456w" sizes="auto, 100vw"><img src="https://substackcdn.com/image/fetch/$s_!4Zuc!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F703952df-9fc6-4760-8560-ccb341a58183_3840x2160.webp" width="1456" height="819" data-attrs="{&quot;src&quot;:&quot;https://substack-post-media.s3.amazonaws.com/public/images/703952df-9fc6-4760-8560-ccb341a58183_3840x2160.webp&quot;,&quot;srcNoWatermark&quot;:null,&quot;fullscreen&quot;:null,&quot;imageSize&quot;:null,&quot;height&quot;:819,&quot;width&quot;:1456,&quot;resizeWidth&quot;:null,&quot;bytes&quot;:90356,&quot;alt&quot;:null,&quot;title&quot;:null,&quot;type&quot;:&quot;image/webp&quot;,&quot;href&quot;:null,&quot;belowTheFold&quot;:true,&quot;topImage&quot;:false,&quot;internalRedirect&quot;:&quot;https://www.datatinkerer.io/i/223984118?img=https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F703952df-9fc6-4760-8560-ccb341a58183_3840x2160.webp&quot;,&quot;isProcessing&quot;:false,&quot;align&quot;:null,&quot;offset&quot;:false}" class="sizing-normal" alt="" srcset="https://substackcdn.com/image/fetch/$s_!4Zuc!,w_424,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F703952df-9fc6-4760-8560-ccb341a58183_3840x2160.webp 424w, https://substackcdn.com/image/fetch/$s_!4Zuc!,w_848,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F703952df-9fc6-4760-8560-ccb341a58183_3840x2160.webp 848w, https://substackcdn.com/image/fetch/$s_!4Zuc!,w_1272,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F703952df-9fc6-4760-8560-ccb341a58183_3840x2160.webp 1272w, https://substackcdn.com/image/fetch/$s_!4Zuc!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F703952df-9fc6-4760-8560-ccb341a58183_3840x2160.webp 1456w" sizes="auto, 100vw" loading="lazy"></picture><div class="image-link-expand"><div class="pencraft pc-display-flex pc-gap-8 pc-reset"><button class="pencraft pc-reset pencraft icon-container restack-image buttonBase-tqxFUX" tabindex="0" type="button"><svg aria-hidden="true" width="20" height="20" viewBox="0 0 20 20" fill="none" stroke-width="1.5" stroke="var(--color-fg-primary)" stroke-linecap="round" stroke-linejoin="round" xmlns="http://www.w3.org/2000/svg" class="icon-noB79L"><g><path d="M2.53001 7.81595C3.49179 4.73911 6.43281 2.5 9.91173 2.5C13.1684 2.5 15.9537 4.46214 17.0852 7.23684L17.6179 8.67647M17.6179 8.67647L18.5002 4.26471M17.6179 8.67647L13.6473 6.91176M17.4995 12.1841C16.5378 15.2609 13.5967 17.5 10.1178 17.5C6.86118 17.5 4.07589 15.5379 2.94432 12.7632L2.41165 11.3235M2.41165 11.3235L1.5293 15.7353M2.41165 11.3235L6.38224 13.0882"></path></g></svg></button><button class="pencraft pc-reset pencraft icon-container view-image buttonBase-tqxFUX" tabindex="0" type="button"><svg xmlns="http://www.w3.org/2000/svg" width="20" height="20" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" class="lucide lucide-maximize2 lucide-maximize-2 icon-noB79L"><polyline points="15 3 21 3 21 9"></polyline><polyline points="9 21 3 21 3 15"></polyline><line x1="21" x2="14" y1="3" y2="10"></line><line x1="3" x2="10" y1="21" y2="14"></line></svg></button></div></div></div></a><figcaption class="image-caption"><em>One agent gathers evidence, a second triages it, and the owner reviews changes that need approval. (Source: Notion)</em></figcaption></figure></div><p>A correction in Slack gives the review process evidence to investigate; changing a metric still requires approval. A merged PR does not prove the change is deployed or historical data has been backfilled. Before editing, the triage process checks current guidance and preserves newer human changes; afterwards it verifies the saved update and rechecks affected questions. Metric meaning, source selection, grain, filters, access and sensitive-data handling remain owner-review decisions.</p><p>The maintenance agents review approved sources and update Notion pages. Scheduled runs do not inherit an employee&#8217;s Snowflake access. If query logs feed the review, they need a separately approved source.</p><div><hr></div><h2>The full scoop</h2><p>To learn more about this, check <a href="https://www.notion.com/blog/how-we-built-a-personal-data-scientist-for-every-notion-employee">Notion's Engineering Blog</a> post on this topic</p><div><hr></div><div class="subscription-widget-wrap-editor" data-attrs="{&quot;url&quot;:&quot;https://www.datatinkerer.io/subscribe?&quot;,&quot;text&quot;:&quot;Subscribe&quot;,&quot;language&quot;:&quot;en&quot;}" data-component-name="SubscribeWidgetToDOM"><div class="subscription-widget show-subscribe"><div class="preamble"><p class="cta-caption">If you liked this post and don&#8217;t want to miss the next one, subscribe to Data Tinkerer!</p></div><form class="subscription-widget-subscribe"><input type="email" class="email-input" name="email" placeholder="Type your email&#8230;" tabindex="-1"><input type="submit" class="button primary" value="Subscribe"><div class="fake-input-wrapper"><div class="fake-input"></div><div class="fake-button"></div></div></form></div></div><p>If you are already subscribed and enjoyed the article, please give it a like and/or share it others, really appreciate it &#128591;</p><p class="button-wrapper" data-attrs="{&quot;url&quot;:&quot;https://www.datatinkerer.io/p/how-notion-cut-data-team-requests-by-80-percent?utm_source=substack&utm_medium=email&utm_content=share&action=share&quot;,&quot;text&quot;:&quot;Share&quot;,&quot;action&quot;:null,&quot;class&quot;:null}" data-component-name="ButtonCreateButton"><a class="button primary" href="https://www.datatinkerer.io/p/how-notion-cut-data-team-requests-by-80-percent?utm_source=substack&utm_medium=email&utm_content=share&action=share"><span>Share</span></a></p><div><hr></div><h3>Keep learning</h3><div class="digest-post-embed" data-attrs="{&quot;nodeId&quot;:&quot;5590acfb-0df2-4051-bec8-f59d73d39afe&quot;,&quot;caption&quot;:&quot;Grab&#8217;s analytics team supported 1,000+ monthly users across 15,000+ tables, but repetitive 'quick questions' were eating 40% of engineering time.\n\nTo fix it, the team built a multi-agent AI system that could answer data questions, trace lineage, inspect code, check pipeline health and draft common enhancement requests.\n\nThis piece breaks down how Grab moved data support from manual firefighting to agent-assisted workflows with specialist agents, SQL guardrails, human review and a feedback loop.&quot;,&quot;cta&quot;:null,&quot;showBylines&quot;:true,&quot;showDescription&quot;:true,&quot;showImage&quot;:true,&quot;size&quot;:&quot;lg&quot;,&quot;isEditorNode&quot;:true,&quot;title&quot;:&quot;How Grab Reclaimed Hundreds of Data Engineering Hours With Multi-Agent AI&quot;,&quot;publishedBylines&quot;:[{&quot;id&quot;:291590442,&quot;name&quot;:&quot;Data Tinkerer&quot;,&quot;bio&quot;:&quot;Head of analytics sharing deep dives and learnings about AI and all things data (science, engineering, analysis)&quot;,&quot;photo_url&quot;:&quot;https://substack-post-media.s3.amazonaws.com/public/images/83d3bbe0-5fb8-4f8d-9b74-036abbd6fec9_500x500.png&quot;,&quot;is_guest&quot;:false,&quot;bestseller_tier&quot;:null}],&quot;post_date&quot;:&quot;2026-05-28T04:10:36.815Z&quot;,&quot;cover_image&quot;:&quot;https://images.unsplash.com/photo-1691513585424-962740372f18?crop=entropy&amp;cs=tinysrgb&amp;fit=max&amp;fm=jpg&amp;ixid=M3wzMDAzMzh8MHwxfHNlYXJjaHwzfHxncmFifGVufDB8fHx8MTc3OTcwNDM2Nnww&amp;ixlib=rb-4.1.0&amp;q=80&amp;w=1080&quot;,&quot;cover_image_alt&quot;:null,&quot;cover_image_crop&quot;:null,&quot;canonical_url&quot;:&quot;https://www.datatinkerer.io/p/how-grab-reclaimed-hundreds-of-data-engineering-hours-with-multi-agent-ai&quot;,&quot;section_name&quot;:&quot;Data Engineering&quot;,&quot;video_upload_id&quot;:null,&quot;id&quot;:199038802,&quot;type&quot;:&quot;newsletter&quot;,&quot;reaction_count&quot;:13,&quot;comment_count&quot;:1,&quot;publication_id&quot;:3422740,&quot;publication_name&quot;:&quot;Data Tinkerer&quot;,&quot;publication_logo_url&quot;:&quot;https://substackcdn.com/image/fetch/$s_!JEdj!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F6bea5ccd-f356-4154-a1db-16268300510e_500x500.png&quot;,&quot;belowTheFold&quot;:true,&quot;youtube_url&quot;:null,&quot;show_links&quot;:null,&quot;feed_url&quot;:null}"></div><div class="digest-post-embed" data-attrs="{&quot;nodeId&quot;:&quot;f930115f-b011-4fde-87ca-e488fb7225ef&quot;,&quot;caption&quot;:&quot;Airtable moved petabytes of archive data out of MySQL, made storage 100x cheaper and kept interactive query latency intact.<br /><br />To get there, the team built a two-tier storage system that pushed old audit-log data into S3-backed Parquet files and queried them through an embedded Rust engine.<br /><br />This piece breaks down how Airtable handled the migration, why it chose DataFusion and the optimizations that made cold storage feel fast.&quot;,&quot;cta&quot;:null,&quot;showBylines&quot;:true,&quot;showDescription&quot;:true,&quot;showImage&quot;:true,&quot;size&quot;:&quot;lg&quot;,&quot;isEditorNode&quot;:true,&quot;title&quot;:&quot;How Airtable Saved Millions by Cutting Archive Storage Costs by 100x&quot;,&quot;publishedBylines&quot;:[{&quot;id&quot;:291590442,&quot;name&quot;:&quot;Data Tinkerer&quot;,&quot;bio&quot;:&quot;Head of analytics sharing deep dives and learnings about AI and all things data (science, engineering, analysis)&quot;,&quot;photo_url&quot;:&quot;https://substack-post-media.s3.amazonaws.com/public/images/83d3bbe0-5fb8-4f8d-9b74-036abbd6fec9_500x500.png&quot;,&quot;is_guest&quot;:false,&quot;bestseller_tier&quot;:null}],&quot;post_date&quot;:&quot;2026-04-23T04:53:35.458Z&quot;,&quot;cover_image&quot;:&quot;https://substack-post-media.s3.amazonaws.com/public/images/0887cd3f-fd83-4fbd-93df-e009c31ed22b_1536x1024.png&quot;,&quot;cover_image_alt&quot;:null,&quot;cover_image_crop&quot;:null,&quot;canonical_url&quot;:&quot;https://www.datatinkerer.io/p/how-airtable-saved-millions-by-cutting&quot;,&quot;section_name&quot;:&quot;Data Engineering&quot;,&quot;video_upload_id&quot;:null,&quot;id&quot;:194669935,&quot;type&quot;:&quot;newsletter&quot;,&quot;reaction_count&quot;:13,&quot;comment_count&quot;:0,&quot;publication_id&quot;:3422740,&quot;publication_name&quot;:&quot;Data Tinkerer&quot;,&quot;publication_logo_url&quot;:&quot;https://substackcdn.com/image/fetch/$s_!JEdj!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F6bea5ccd-f356-4154-a1db-16268300510e_500x500.png&quot;,&quot;belowTheFold&quot;:true,&quot;youtube_url&quot;:null,&quot;show_links&quot;:null,&quot;feed_url&quot;:null}"></div>]]></content:encoded></item><item><title><![CDATA[How Grab Reclaimed Hundreds of Data Engineering Hours With Multi-Agent AI]]></title><description><![CDATA[How a specialist agent system helped Grab answer data questions, check pipeline health and handle common enhancement requests without drowning engineers in support work.]]></description><link>https://www.datatinkerer.io/p/how-grab-reclaimed-hundreds-of-data-engineering-hours-with-multi-agent-ai</link><guid isPermaLink="false">https://www.datatinkerer.io/p/how-grab-reclaimed-hundreds-of-data-engineering-hours-with-multi-agent-ai</guid><dc:creator><![CDATA[Data Tinkerer]]></dc:creator><pubDate>Thu, 28 May 2026 04:10:36 GMT</pubDate><enclosure url="https://images.unsplash.com/photo-1691513585424-962740372f18?crop=entropy&amp;cs=tinysrgb&amp;fit=max&amp;fm=jpg&amp;ixid=M3wzMDAzMzh8MHwxfHNlYXJjaHwzfHxncmFifGVufDB8fHx8MTc3OTcwNDM2Nnww&amp;ixlib=rb-4.1.0&amp;q=80&amp;w=1080" length="0" type="image/jpeg"/><content:encoded><![CDATA[<p>Fellow Data Tinkerers</p><p>Today we will look at how Grab used AI agents to cut repetitive data support from hours to minutes.</p><p>But before that, I wanted to share with you what you could unlock if you share Data Tinkerer with just <strong>1 more person</strong>.</p><div class="captioned-image-container"><figure><a class="image-link image2 is-viewable-img" target="_blank" href="https://substackcdn.com/image/fetch/$s_!6cDW!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F1cc04ad1-9acb-47b2-b59d-b190b4a3accf_800x402.gif" data-component-name="Image2ToDOM"><div class="image2-inset"><picture><source type="image/webp" srcset="https://substackcdn.com/image/fetch/$s_!6cDW!,w_424,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F1cc04ad1-9acb-47b2-b59d-b190b4a3accf_800x402.gif 424w, https://substackcdn.com/image/fetch/$s_!6cDW!,w_848,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F1cc04ad1-9acb-47b2-b59d-b190b4a3accf_800x402.gif 848w, https://substackcdn.com/image/fetch/$s_!6cDW!,w_1272,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F1cc04ad1-9acb-47b2-b59d-b190b4a3accf_800x402.gif 1272w, https://substackcdn.com/image/fetch/$s_!6cDW!,w_1456,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F1cc04ad1-9acb-47b2-b59d-b190b4a3accf_800x402.gif 1456w" sizes="100vw"><img src="https://substackcdn.com/image/fetch/$s_!6cDW!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F1cc04ad1-9acb-47b2-b59d-b190b4a3accf_800x402.gif" width="800" height="402" data-attrs="{&quot;src&quot;:&quot;https://substack-post-media.s3.amazonaws.com/public/images/1cc04ad1-9acb-47b2-b59d-b190b4a3accf_800x402.gif&quot;,&quot;srcNoWatermark&quot;:null,&quot;fullscreen&quot;:null,&quot;imageSize&quot;:null,&quot;height&quot;:402,&quot;width&quot;:800,&quot;resizeWidth&quot;:null,&quot;bytes&quot;:3369150,&quot;alt&quot;:null,&quot;title&quot;:null,&quot;type&quot;:&quot;image/gif&quot;,&quot;href&quot;:null,&quot;belowTheFold&quot;:false,&quot;topImage&quot;:true,&quot;internalRedirect&quot;:&quot;https://www.datatinkerer.io/i/199038802?img=https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F1cc04ad1-9acb-47b2-b59d-b190b4a3accf_800x402.gif&quot;,&quot;isProcessing&quot;:false,&quot;align&quot;:null,&quot;offset&quot;:false}" class="sizing-normal" alt="" srcset="https://substackcdn.com/image/fetch/$s_!6cDW!,w_424,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F1cc04ad1-9acb-47b2-b59d-b190b4a3accf_800x402.gif 424w, https://substackcdn.com/image/fetch/$s_!6cDW!,w_848,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F1cc04ad1-9acb-47b2-b59d-b190b4a3accf_800x402.gif 848w, https://substackcdn.com/image/fetch/$s_!6cDW!,w_1272,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F1cc04ad1-9acb-47b2-b59d-b190b4a3accf_800x402.gif 1272w, https://substackcdn.com/image/fetch/$s_!6cDW!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F1cc04ad1-9acb-47b2-b59d-b190b4a3accf_800x402.gif 1456w" sizes="100vw" fetchpriority="high"></picture><div class="image-link-expand"><div class="pencraft pc-display-flex pc-gap-8 pc-reset"><button class="pencraft pc-reset pencraft icon-container restack-image buttonBase-tqxFUX" tabindex="0" type="button"><svg aria-hidden="true" width="20" height="20" viewBox="0 0 20 20" fill="none" stroke-width="1.5" stroke="var(--color-fg-primary)" stroke-linecap="round" stroke-linejoin="round" xmlns="http://www.w3.org/2000/svg" class="icon-noB79L"><g><path d="M2.53001 7.81595C3.49179 4.73911 6.43281 2.5 9.91173 2.5C13.1684 2.5 15.9537 4.46214 17.0852 7.23684L17.6179 8.67647M17.6179 8.67647L18.5002 4.26471M17.6179 8.67647L13.6473 6.91176M17.4995 12.1841C16.5378 15.2609 13.5967 17.5 10.1178 17.5C6.86118 17.5 4.07589 15.5379 2.94432 12.7632L2.41165 11.3235M2.41165 11.3235L1.5293 15.7353M2.41165 11.3235L6.38224 13.0882"></path></g></svg></button><button class="pencraft pc-reset pencraft icon-container view-image buttonBase-tqxFUX" tabindex="0" type="button"><svg xmlns="http://www.w3.org/2000/svg" width="20" height="20" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" class="lucide lucide-maximize2 lucide-maximize-2 icon-noB79L"><polyline points="15 3 21 3 21 9"></polyline><polyline points="9 21 3 21 3 15"></polyline><line x1="21" x2="14" y1="3" y2="10"></line><line x1="3" x2="10" y1="21" y2="14"></line></svg></button></div></div></div></a></figure></div><p>There are 100+ resources to learn all things data (science, engineering, analysis). It includes videos, courses, projects and can be filtered by tech stack (Python, SQL, Spark and etc), skill level (Beginner, Intermediate and so on) provider name or free/paid. So if you know other people who like staying up to date on all things data, please share Data Tinkerer with them!</p><p class="button-wrapper" data-attrs="{&quot;url&quot;:&quot;https://www.datatinkerer.io/leaderboard?&amp;utm_source=post&quot;,&quot;text&quot;:&quot;Refer a friend&quot;,&quot;action&quot;:null,&quot;class&quot;:null}" data-component-name="ButtonCreateButton"><a class="button primary" href="https://www.datatinkerer.io/leaderboard?&amp;utm_source=post"><span>Refer a friend</span></a></p><p>Now, with that out of the way, let&#8217;s get into how Grab turned repetitive data support into an AI agent workflow.</p>
      <p>
          <a href="https://www.datatinkerer.io/p/how-grab-reclaimed-hundreds-of-data-engineering-hours-with-multi-agent-ai">
              Read more
          </a>
      </p>
   ]]></content:encoded></item><item><title><![CDATA[How Airtable Saved Millions by Cutting Archive Storage Costs by 100x]]></title><description><![CDATA[Airtable moved petabytes of cold log data out of MySQL and built a cheaper archive layer on S3 and Parquet without sacrificing fast queries.]]></description><link>https://www.datatinkerer.io/p/how-airtable-saved-millions-by-cutting</link><guid isPermaLink="false">https://www.datatinkerer.io/p/how-airtable-saved-millions-by-cutting</guid><dc:creator><![CDATA[Data Tinkerer]]></dc:creator><pubDate>Thu, 23 Apr 2026 04:53:35 GMT</pubDate><enclosure url="https://substack-post-media.s3.amazonaws.com/public/images/0887cd3f-fd83-4fbd-93df-e009c31ed22b_1536x1024.png" length="0" type="image/jpeg"/><content:encoded><![CDATA[<p>Fellow Data Tinkerers</p><p>Today we will look at how Airtable cut archive storage costs by 100x and saved millions.</p><p>But before that, I wanted to share with you what you could unlock if you share Data Tinkerer with just <strong>1 more person</strong>.</p><div class="captioned-image-container"><figure><a class="image-link image2 is-viewable-img" target="_blank" href="https://substackcdn.com/image/fetch/$s_!-UN6!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fa84d7578-0b5f-40e6-9c87-afcbb488bda1_800x402.gif" data-component-name="Image2ToDOM"><div class="image2-inset"><picture><source type="image/webp" srcset="https://substackcdn.com/image/fetch/$s_!-UN6!,w_424,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fa84d7578-0b5f-40e6-9c87-afcbb488bda1_800x402.gif 424w, https://substackcdn.com/image/fetch/$s_!-UN6!,w_848,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fa84d7578-0b5f-40e6-9c87-afcbb488bda1_800x402.gif 848w, https://substackcdn.com/image/fetch/$s_!-UN6!,w_1272,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fa84d7578-0b5f-40e6-9c87-afcbb488bda1_800x402.gif 1272w, https://substackcdn.com/image/fetch/$s_!-UN6!,w_1456,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fa84d7578-0b5f-40e6-9c87-afcbb488bda1_800x402.gif 1456w" sizes="100vw"><img src="https://substackcdn.com/image/fetch/$s_!-UN6!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fa84d7578-0b5f-40e6-9c87-afcbb488bda1_800x402.gif" width="800" height="402" data-attrs="{&quot;src&quot;:&quot;https://substack-post-media.s3.amazonaws.com/public/images/a84d7578-0b5f-40e6-9c87-afcbb488bda1_800x402.gif&quot;,&quot;srcNoWatermark&quot;:null,&quot;fullscreen&quot;:null,&quot;imageSize&quot;:null,&quot;height&quot;:402,&quot;width&quot;:800,&quot;resizeWidth&quot;:null,&quot;bytes&quot;:3369150,&quot;alt&quot;:null,&quot;title&quot;:null,&quot;type&quot;:&quot;image/gif&quot;,&quot;href&quot;:null,&quot;belowTheFold&quot;:false,&quot;topImage&quot;:true,&quot;internalRedirect&quot;:&quot;https://www.datatinkerer.io/i/194669935?img=https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fa84d7578-0b5f-40e6-9c87-afcbb488bda1_800x402.gif&quot;,&quot;isProcessing&quot;:false,&quot;align&quot;:null,&quot;offset&quot;:false}" class="sizing-normal" alt="" srcset="https://substackcdn.com/image/fetch/$s_!-UN6!,w_424,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fa84d7578-0b5f-40e6-9c87-afcbb488bda1_800x402.gif 424w, https://substackcdn.com/image/fetch/$s_!-UN6!,w_848,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fa84d7578-0b5f-40e6-9c87-afcbb488bda1_800x402.gif 848w, https://substackcdn.com/image/fetch/$s_!-UN6!,w_1272,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fa84d7578-0b5f-40e6-9c87-afcbb488bda1_800x402.gif 1272w, https://substackcdn.com/image/fetch/$s_!-UN6!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fa84d7578-0b5f-40e6-9c87-afcbb488bda1_800x402.gif 1456w" sizes="100vw" fetchpriority="high"></picture><div class="image-link-expand"><div class="pencraft pc-display-flex pc-gap-8 pc-reset"><button class="pencraft pc-reset pencraft icon-container restack-image buttonBase-GK1x3M" tabindex="0" type="button"><svg aria-hidden="true" width="20" height="20" viewBox="0 0 20 20" fill="none" stroke-width="1.5" stroke="var(--color-fg-primary)" stroke-linecap="round" stroke-linejoin="round" xmlns="http://www.w3.org/2000/svg" class="icon-noB79L"><g><path d="M2.53001 7.81595C3.49179 4.73911 6.43281 2.5 9.91173 2.5C13.1684 2.5 15.9537 4.46214 17.0852 7.23684L17.6179 8.67647M17.6179 8.67647L18.5002 4.26471M17.6179 8.67647L13.6473 6.91176M17.4995 12.1841C16.5378 15.2609 13.5967 17.5 10.1178 17.5C6.86118 17.5 4.07589 15.5379 2.94432 12.7632L2.41165 11.3235M2.41165 11.3235L1.5293 15.7353M2.41165 11.3235L6.38224 13.0882"></path></g></svg></button><button class="pencraft pc-reset pencraft icon-container view-image buttonBase-GK1x3M" tabindex="0" type="button"><svg xmlns="http://www.w3.org/2000/svg" width="20" height="20" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" class="lucide lucide-maximize2 lucide-maximize-2 icon-noB79L"><polyline points="15 3 21 3 21 9"></polyline><polyline points="9 21 3 21 3 15"></polyline><line x1="21" x2="14" y1="3" y2="10"></line><line x1="3" x2="10" y1="21" y2="14"></line></svg></button></div></div></div></a></figure></div><p>There are 100+ resources to learn all things data (science, engineering, analysis). It includes videos, courses, projects and can be filtered by tech stack (Python, SQL, Spark and etc), skill level (Beginner, Intermediate and so on) provider name or free/paid. So if you know other people who like staying up to date on all things data, please share Data Tinkerer with them!</p><p class="button-wrapper" data-attrs="{&quot;url&quot;:&quot;https://www.datatinkerer.io/leaderboard?&amp;utm_source=post&quot;,&quot;text&quot;:&quot;Refer a friend&quot;,&quot;action&quot;:null,&quot;class&quot;:null}" data-component-name="ButtonCreateButton"><a class="button primary" href="https://www.datatinkerer.io/leaderboard?&amp;utm_source=post"><span>Refer a friend</span></a></p><p>Now, with that out of the way, let&#8217;s get into how Airtable pulled it off.</p><h3>TL;DR</h3>
      <p>
          <a href="https://www.datatinkerer.io/p/how-airtable-saved-millions-by-cutting">
              Read more
          </a>
      </p>
   ]]></content:encoded></item><item><title><![CDATA[How Notion Scaled AI Q&A to Millions of Workspaces]]></title><description><![CDATA[Kafka, Spark and Ray powering low-latency, high-throughput search pipelines]]></description><link>https://www.datatinkerer.io/p/how-notion-scaled-ai-q-and-a-to-millions-of-workspaces</link><guid isPermaLink="false">https://www.datatinkerer.io/p/how-notion-scaled-ai-q-and-a-to-millions-of-workspaces</guid><dc:creator><![CDATA[Data Tinkerer]]></dc:creator><pubDate>Thu, 26 Mar 2026 04:00:33 GMT</pubDate><enclosure url="https://images.unsplash.com/photo-1683114010575-3ead4403180e?crop=entropy&amp;cs=tinysrgb&amp;fit=max&amp;fm=jpg&amp;ixid=M3wzMDAzMzh8MHwxfHNlYXJjaHwxfHxub3Rpb258ZW58MHx8fHwxNzc0MjU5NjgzfDA&amp;ixlib=rb-4.1.0&amp;q=80&amp;w=1080" length="0" type="image/jpeg"/><content:encoded><![CDATA[<p>Fellow Data Tinkerers</p><p>Today we will look at how Notion scaled its AI Q&amp;A to millions of users.</p><p>But before that, I wanted to share with you what you could unlock if you share Data Tinkerer with just <strong>1 more person</strong>.</p><div class="captioned-image-container"><figure><a class="image-link image2 is-viewable-img" target="_blank" href="https://substackcdn.com/image/fetch/$s_!HsBV!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F084aaae7-58c7-464e-b0f8-7c31523985ef_800x402.gif" data-component-name="Image2ToDOM"><div class="image2-inset"><picture><source type="image/webp" srcset="https://substackcdn.com/image/fetch/$s_!HsBV!,w_424,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F084aaae7-58c7-464e-b0f8-7c31523985ef_800x402.gif 424w, https://substackcdn.com/image/fetch/$s_!HsBV!,w_848,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F084aaae7-58c7-464e-b0f8-7c31523985ef_800x402.gif 848w, https://substackcdn.com/image/fetch/$s_!HsBV!,w_1272,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F084aaae7-58c7-464e-b0f8-7c31523985ef_800x402.gif 1272w, https://substackcdn.com/image/fetch/$s_!HsBV!,w_1456,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F084aaae7-58c7-464e-b0f8-7c31523985ef_800x402.gif 1456w" sizes="100vw"><img src="https://substackcdn.com/image/fetch/$s_!HsBV!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F084aaae7-58c7-464e-b0f8-7c31523985ef_800x402.gif" width="800" height="402" data-attrs="{&quot;src&quot;:&quot;https://substack-post-media.s3.amazonaws.com/public/images/084aaae7-58c7-464e-b0f8-7c31523985ef_800x402.gif&quot;,&quot;srcNoWatermark&quot;:null,&quot;fullscreen&quot;:null,&quot;imageSize&quot;:null,&quot;height&quot;:402,&quot;width&quot;:800,&quot;resizeWidth&quot;:null,&quot;bytes&quot;:3369150,&quot;alt&quot;:null,&quot;title&quot;:null,&quot;type&quot;:&quot;image/gif&quot;,&quot;href&quot;:null,&quot;belowTheFold&quot;:false,&quot;topImage&quot;:true,&quot;internalRedirect&quot;:&quot;https://www.datatinkerer.io/i/191742179?img=https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F084aaae7-58c7-464e-b0f8-7c31523985ef_800x402.gif&quot;,&quot;isProcessing&quot;:false,&quot;align&quot;:null,&quot;offset&quot;:false}" class="sizing-normal" alt="" srcset="https://substackcdn.com/image/fetch/$s_!HsBV!,w_424,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F084aaae7-58c7-464e-b0f8-7c31523985ef_800x402.gif 424w, https://substackcdn.com/image/fetch/$s_!HsBV!,w_848,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F084aaae7-58c7-464e-b0f8-7c31523985ef_800x402.gif 848w, https://substackcdn.com/image/fetch/$s_!HsBV!,w_1272,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F084aaae7-58c7-464e-b0f8-7c31523985ef_800x402.gif 1272w, https://substackcdn.com/image/fetch/$s_!HsBV!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F084aaae7-58c7-464e-b0f8-7c31523985ef_800x402.gif 1456w" sizes="100vw" fetchpriority="high"></picture><div class="image-link-expand"><div class="pencraft pc-display-flex pc-gap-8 pc-reset"><button class="pencraft pc-reset pencraft icon-container restack-image buttonBase-tqxFUX" tabindex="0" type="button"><svg aria-hidden="true" width="20" height="20" viewBox="0 0 20 20" fill="none" stroke-width="1.5" stroke="var(--color-fg-primary)" stroke-linecap="round" stroke-linejoin="round" xmlns="http://www.w3.org/2000/svg" class="icon-noB79L"><g><path d="M2.53001 7.81595C3.49179 4.73911 6.43281 2.5 9.91173 2.5C13.1684 2.5 15.9537 4.46214 17.0852 7.23684L17.6179 8.67647M17.6179 8.67647L18.5002 4.26471M17.6179 8.67647L13.6473 6.91176M17.4995 12.1841C16.5378 15.2609 13.5967 17.5 10.1178 17.5C6.86118 17.5 4.07589 15.5379 2.94432 12.7632L2.41165 11.3235M2.41165 11.3235L1.5293 15.7353M2.41165 11.3235L6.38224 13.0882"></path></g></svg></button><button class="pencraft pc-reset pencraft icon-container view-image buttonBase-tqxFUX" tabindex="0" type="button"><svg xmlns="http://www.w3.org/2000/svg" width="20" height="20" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" class="lucide lucide-maximize2 lucide-maximize-2 icon-noB79L"><polyline points="15 3 21 3 21 9"></polyline><polyline points="9 21 3 21 3 15"></polyline><line x1="21" x2="14" y1="3" y2="10"></line><line x1="3" x2="10" y1="21" y2="14"></line></svg></button></div></div></div></a></figure></div><p>There are 100+ resources to learn all things data (science, engineering, analysis). It includes videos, courses, projects and can be filtered by tech stack (Python, SQL, Spark and etc), skill level (Beginner, Intermediate and so on) provider name or free/paid. So if you know other people who like staying up to date on all things data, please share Data Tinkerer with them!</p><p class="button-wrapper" data-attrs="{&quot;url&quot;:&quot;https://www.datatinkerer.io/leaderboard?&amp;utm_source=post&quot;,&quot;text&quot;:&quot;Refer a friend&quot;,&quot;action&quot;:null,&quot;class&quot;:null}" data-component-name="ButtonCreateButton"><a class="button primary" href="https://www.datatinkerer.io/leaderboard?&amp;utm_source=post"><span>Refer a friend</span></a></p>
      <p>
          <a href="https://www.datatinkerer.io/p/how-notion-scaled-ai-q-and-a-to-millions-of-workspaces">
              Read more
          </a>
      </p>
   ]]></content:encoded></item><item><title><![CDATA[How LinkedIn Built a Pipeline That Scales to 230M Records/sec Without Breaking SLAs]]></title><description><![CDATA[From partition strategy to adaptive throttling, the playbook behind Venice&#8217;s ingestion evolution.]]></description><link>https://www.datatinkerer.io/p/how-linkedin-built-a-pipeline-that-scales-to-230-million-records</link><guid isPermaLink="false">https://www.datatinkerer.io/p/how-linkedin-built-a-pipeline-that-scales-to-230-million-records</guid><dc:creator><![CDATA[Data Tinkerer]]></dc:creator><pubDate>Thu, 19 Feb 2026 04:00:52 GMT</pubDate><enclosure url="https://images.unsplash.com/photo-1712217559097-cc2aaf698767?crop=entropy&amp;cs=tinysrgb&amp;fit=max&amp;fm=jpg&amp;ixid=M3wzMDAzMzh8MHwxfHNlYXJjaHwxM3x8bGlua2VkaW58ZW58MHx8fHwxNzcxMTMwNjQ1fDA&amp;ixlib=rb-4.1.0&amp;q=80&amp;w=1080" length="0" type="image/jpeg"/><content:encoded><![CDATA[<p>Fellow Data Tinkerers</p><p>Today we will look at how LinkedIn ingests data at scale.</p><p>But before that, I wanted to share with you what you could unlock if you share Data Tinkerer with just <strong>1 more person</strong>.</p><div class="captioned-image-container"><figure><a class="image-link image2 is-viewable-img" target="_blank" href="https://substackcdn.com/image/fetch/$s_!y5YD!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F42e71ce5-0a12-4514-b7a0-ddfd0460289a_800x402.gif" data-component-name="Image2ToDOM"><div class="image2-inset"><picture><source type="image/webp" srcset="https://substackcdn.com/image/fetch/$s_!y5YD!,w_424,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F42e71ce5-0a12-4514-b7a0-ddfd0460289a_800x402.gif 424w, https://substackcdn.com/image/fetch/$s_!y5YD!,w_848,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F42e71ce5-0a12-4514-b7a0-ddfd0460289a_800x402.gif 848w, https://substackcdn.com/image/fetch/$s_!y5YD!,w_1272,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F42e71ce5-0a12-4514-b7a0-ddfd0460289a_800x402.gif 1272w, https://substackcdn.com/image/fetch/$s_!y5YD!,w_1456,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F42e71ce5-0a12-4514-b7a0-ddfd0460289a_800x402.gif 1456w" sizes="100vw"><img src="https://substackcdn.com/image/fetch/$s_!y5YD!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F42e71ce5-0a12-4514-b7a0-ddfd0460289a_800x402.gif" width="800" height="402" data-attrs="{&quot;src&quot;:&quot;https://substack-post-media.s3.amazonaws.com/public/images/42e71ce5-0a12-4514-b7a0-ddfd0460289a_800x402.gif&quot;,&quot;srcNoWatermark&quot;:null,&quot;fullscreen&quot;:null,&quot;imageSize&quot;:null,&quot;height&quot;:402,&quot;width&quot;:800,&quot;resizeWidth&quot;:null,&quot;bytes&quot;:3369150,&quot;alt&quot;:null,&quot;title&quot;:null,&quot;type&quot;:&quot;image/gif&quot;,&quot;href&quot;:null,&quot;belowTheFold&quot;:false,&quot;topImage&quot;:true,&quot;internalRedirect&quot;:&quot;https://www.datatinkerer.io/i/187999868?img=https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F42e71ce5-0a12-4514-b7a0-ddfd0460289a_800x402.gif&quot;,&quot;isProcessing&quot;:false,&quot;align&quot;:null,&quot;offset&quot;:false}" class="sizing-normal" alt="" srcset="https://substackcdn.com/image/fetch/$s_!y5YD!,w_424,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F42e71ce5-0a12-4514-b7a0-ddfd0460289a_800x402.gif 424w, https://substackcdn.com/image/fetch/$s_!y5YD!,w_848,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F42e71ce5-0a12-4514-b7a0-ddfd0460289a_800x402.gif 848w, https://substackcdn.com/image/fetch/$s_!y5YD!,w_1272,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F42e71ce5-0a12-4514-b7a0-ddfd0460289a_800x402.gif 1272w, https://substackcdn.com/image/fetch/$s_!y5YD!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F42e71ce5-0a12-4514-b7a0-ddfd0460289a_800x402.gif 1456w" sizes="100vw" fetchpriority="high"></picture><div class="image-link-expand"><div class="pencraft pc-display-flex pc-gap-8 pc-reset"><button class="pencraft pc-reset pencraft icon-container restack-image buttonBase-GK1x3M" tabindex="0" type="button"><svg aria-hidden="true" width="20" height="20" viewBox="0 0 20 20" fill="none" stroke-width="1.5" stroke="var(--color-fg-primary)" stroke-linecap="round" stroke-linejoin="round" xmlns="http://www.w3.org/2000/svg" class="icon-noB79L"><g><path d="M2.53001 7.81595C3.49179 4.73911 6.43281 2.5 9.91173 2.5C13.1684 2.5 15.9537 4.46214 17.0852 7.23684L17.6179 8.67647M17.6179 8.67647L18.5002 4.26471M17.6179 8.67647L13.6473 6.91176M17.4995 12.1841C16.5378 15.2609 13.5967 17.5 10.1178 17.5C6.86118 17.5 4.07589 15.5379 2.94432 12.7632L2.41165 11.3235M2.41165 11.3235L1.5293 15.7353M2.41165 11.3235L6.38224 13.0882"></path></g></svg></button><button class="pencraft pc-reset pencraft icon-container view-image buttonBase-GK1x3M" tabindex="0" type="button"><svg xmlns="http://www.w3.org/2000/svg" width="20" height="20" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" class="lucide lucide-maximize2 lucide-maximize-2 icon-noB79L"><polyline points="15 3 21 3 21 9"></polyline><polyline points="9 21 3 21 3 15"></polyline><line x1="21" x2="14" y1="3" y2="10"></line><line x1="3" x2="10" y1="21" y2="14"></line></svg></button></div></div></div></a></figure></div><p>There are 100+ resources to learn all things data (science, engineering, analysis). It includes videos, courses, projects and can be filtered by tech stack (Python, SQL, Spark and etc), skill level (Beginner, Intermediate and so on) provider name or free/paid. So if you know other people who like staying up to date on all things data, please share Data Tinkerer with them!</p><p class="button-wrapper" data-attrs="{&quot;url&quot;:&quot;https://www.datatinkerer.io/leaderboard?&amp;utm_source=post&quot;,&quot;text&quot;:&quot;Refer a friend&quot;,&quot;action&quot;:null,&quot;class&quot;:null}" data-component-name="ButtonCreateButton"><a class="button primary" href="https://www.datatinkerer.io/leaderboard?&amp;utm_source=post"><span>Refer a friend</span></a></p><p>Now, with that out of the way, let&#8217;s get to Venice: LinkedIn&#8217;s data storage platform</p>
      <p>
          <a href="https://www.datatinkerer.io/p/how-linkedin-built-a-pipeline-that-scales-to-230-million-records">
              Read more
          </a>
      </p>
   ]]></content:encoded></item><item><title><![CDATA[How Grab Detects Data Issues across 100+ Kafka Topics Before They Spread]]></title><description><![CDATA[Real-time stream validation surfaces poison records early and notifies owners with context]]></description><link>https://www.datatinkerer.io/p/how-grab-detects-data-issues-across-100-kafka-topics</link><guid isPermaLink="false">https://www.datatinkerer.io/p/how-grab-detects-data-issues-across-100-kafka-topics</guid><dc:creator><![CDATA[Data Tinkerer]]></dc:creator><pubDate>Thu, 15 Jan 2026 04:15:57 GMT</pubDate><enclosure url="https://images.unsplash.com/photo-1624957083543-9a67140fabfd?crop=entropy&amp;cs=tinysrgb&amp;fit=max&amp;fm=jpg&amp;ixid=M3wzMDAzMzh8MHwxfHNlYXJjaHw0fHxncmFifGVufDB8fHx8MTc2NzkzMTIwOXww&amp;ixlib=rb-4.1.0&amp;q=80&amp;w=1080" length="0" type="image/jpeg"/><content:encoded><![CDATA[<p>Fellow Data Tinkerers</p><p>Today we will look at how Grab detects data issues in real-time. </p><p>But before that, I wanted to share with you what you could unlock if you share Data Tinkerer with just <strong>1 more person</strong>.</p><div class="captioned-image-container"><figure><a class="image-link image2 is-viewable-img" target="_blank" href="https://substackcdn.com/image/fetch/$s_!6Doc!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F5f76efe7-9d16-4ec3-8e25-0e3d2282705b_800x402.gif" data-component-name="Image2ToDOM"><div class="image2-inset"><picture><source type="image/webp" srcset="https://substackcdn.com/image/fetch/$s_!6Doc!,w_424,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F5f76efe7-9d16-4ec3-8e25-0e3d2282705b_800x402.gif 424w, https://substackcdn.com/image/fetch/$s_!6Doc!,w_848,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F5f76efe7-9d16-4ec3-8e25-0e3d2282705b_800x402.gif 848w, https://substackcdn.com/image/fetch/$s_!6Doc!,w_1272,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F5f76efe7-9d16-4ec3-8e25-0e3d2282705b_800x402.gif 1272w, https://substackcdn.com/image/fetch/$s_!6Doc!,w_1456,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F5f76efe7-9d16-4ec3-8e25-0e3d2282705b_800x402.gif 1456w" sizes="100vw"><img src="https://substackcdn.com/image/fetch/$s_!6Doc!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F5f76efe7-9d16-4ec3-8e25-0e3d2282705b_800x402.gif" width="800" height="402" data-attrs="{&quot;src&quot;:&quot;https://substack-post-media.s3.amazonaws.com/public/images/5f76efe7-9d16-4ec3-8e25-0e3d2282705b_800x402.gif&quot;,&quot;srcNoWatermark&quot;:null,&quot;fullscreen&quot;:null,&quot;imageSize&quot;:null,&quot;height&quot;:402,&quot;width&quot;:800,&quot;resizeWidth&quot;:null,&quot;bytes&quot;:3369150,&quot;alt&quot;:null,&quot;title&quot;:null,&quot;type&quot;:&quot;image/gif&quot;,&quot;href&quot;:null,&quot;belowTheFold&quot;:false,&quot;topImage&quot;:true,&quot;internalRedirect&quot;:&quot;https://www.datatinkerer.io/i/183755897?img=https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F5f76efe7-9d16-4ec3-8e25-0e3d2282705b_800x402.gif&quot;,&quot;isProcessing&quot;:false,&quot;align&quot;:null,&quot;offset&quot;:false}" class="sizing-normal" alt="" srcset="https://substackcdn.com/image/fetch/$s_!6Doc!,w_424,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F5f76efe7-9d16-4ec3-8e25-0e3d2282705b_800x402.gif 424w, https://substackcdn.com/image/fetch/$s_!6Doc!,w_848,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F5f76efe7-9d16-4ec3-8e25-0e3d2282705b_800x402.gif 848w, https://substackcdn.com/image/fetch/$s_!6Doc!,w_1272,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F5f76efe7-9d16-4ec3-8e25-0e3d2282705b_800x402.gif 1272w, https://substackcdn.com/image/fetch/$s_!6Doc!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F5f76efe7-9d16-4ec3-8e25-0e3d2282705b_800x402.gif 1456w" sizes="100vw" fetchpriority="high"></picture><div class="image-link-expand"><div class="pencraft pc-display-flex pc-gap-8 pc-reset"><button class="pencraft pc-reset pencraft icon-container restack-image buttonBase-GK1x3M" tabindex="0" type="button"><svg aria-hidden="true" width="20" height="20" viewBox="0 0 20 20" fill="none" stroke-width="1.5" stroke="var(--color-fg-primary)" stroke-linecap="round" stroke-linejoin="round" xmlns="http://www.w3.org/2000/svg" class="icon-noB79L"><g><path d="M2.53001 7.81595C3.49179 4.73911 6.43281 2.5 9.91173 2.5C13.1684 2.5 15.9537 4.46214 17.0852 7.23684L17.6179 8.67647M17.6179 8.67647L18.5002 4.26471M17.6179 8.67647L13.6473 6.91176M17.4995 12.1841C16.5378 15.2609 13.5967 17.5 10.1178 17.5C6.86118 17.5 4.07589 15.5379 2.94432 12.7632L2.41165 11.3235M2.41165 11.3235L1.5293 15.7353M2.41165 11.3235L6.38224 13.0882"></path></g></svg></button><button class="pencraft pc-reset pencraft icon-container view-image buttonBase-GK1x3M" tabindex="0" type="button"><svg xmlns="http://www.w3.org/2000/svg" width="20" height="20" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" class="lucide lucide-maximize2 lucide-maximize-2 icon-noB79L"><polyline points="15 3 21 3 21 9"></polyline><polyline points="9 21 3 21 3 15"></polyline><line x1="21" x2="14" y1="3" y2="10"></line><line x1="3" x2="10" y1="21" y2="14"></line></svg></button></div></div></div></a></figure></div><p>There are 100+ resources to learn all things data (science, engineering, analysis). It includes videos, courses, projects and can be filtered by tech stack (Python, SQL, Spark and etc), skill level (Beginner, Intermediate and so on) provider name or free/paid. So if you know other people who like staying up to date on all things data, please share Data Tinkerer with them!</p><p class="button-wrapper" data-attrs="{&quot;url&quot;:&quot;https://www.datatinkerer.io/leaderboard?&amp;utm_source=post&quot;,&quot;text&quot;:&quot;Refer a friend&quot;,&quot;action&quot;:null,&quot;class&quot;:null}" data-component-name="ButtonCreateButton"><a class="button primary" href="https://www.datatinkerer.io/leaderboard?&amp;utm_source=post"><span>Refer a friend</span></a></p><p>Now, with that out of the way, let&#8217;s get to Grab&#8217;s real-time work!</p>
      <p>
          <a href="https://www.datatinkerer.io/p/how-grab-detects-data-issues-across-100-kafka-topics">
              Read more
          </a>
      </p>
   ]]></content:encoded></item><item><title><![CDATA[How Uber Cut Data Lake Freshness From Hours to Minutes With Flink]]></title><description><![CDATA[Why Uber moved ingestion from Spark batch to Flink streaming and what it took to run thousands of jobs reliably at petabyte scale.]]></description><link>https://www.datatinkerer.io/p/how-uber-cut-data-lake-freshness-from-hours-to-minutes-with-flink</link><guid isPermaLink="false">https://www.datatinkerer.io/p/how-uber-cut-data-lake-freshness-from-hours-to-minutes-with-flink</guid><dc:creator><![CDATA[Data Tinkerer]]></dc:creator><pubDate>Fri, 02 Jan 2026 04:30:31 GMT</pubDate><enclosure url="https://images.unsplash.com/photo-1615929362369-4af8423ebd68?crop=entropy&amp;cs=tinysrgb&amp;fit=max&amp;fm=jpg&amp;ixid=M3wzMDAzMzh8MHwxfHNlYXJjaHwxNHx8dWJlcnxlbnwwfHx8fDE3NjcwNzc2NDZ8MA&amp;ixlib=rb-4.1.0&amp;q=80&amp;w=1080" length="0" type="image/jpeg"/><content:encoded><![CDATA[<p>Fellow Data Tinkerers!</p><p>Today we will look at how Uber moved from batch to streaming in their data lake.</p><p>But before that, I wanted to share with you what you could unlock if you share Data Tinkerer with just <strong>1 more person</strong>.</p><div class="captioned-image-container"><figure><a class="image-link image2 is-viewable-img" target="_blank" href="https://substackcdn.com/image/fetch/$s_!05-P!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F6a2affdf-e44f-452e-b02a-93809ad29f60_800x402.gif" data-component-name="Image2ToDOM"><div class="image2-inset"><picture><source type="image/webp" srcset="https://substackcdn.com/image/fetch/$s_!05-P!,w_424,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F6a2affdf-e44f-452e-b02a-93809ad29f60_800x402.gif 424w, https://substackcdn.com/image/fetch/$s_!05-P!,w_848,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F6a2affdf-e44f-452e-b02a-93809ad29f60_800x402.gif 848w, https://substackcdn.com/image/fetch/$s_!05-P!,w_1272,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F6a2affdf-e44f-452e-b02a-93809ad29f60_800x402.gif 1272w, https://substackcdn.com/image/fetch/$s_!05-P!,w_1456,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F6a2affdf-e44f-452e-b02a-93809ad29f60_800x402.gif 1456w" sizes="100vw"><img src="https://substackcdn.com/image/fetch/$s_!05-P!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F6a2affdf-e44f-452e-b02a-93809ad29f60_800x402.gif" width="800" height="402" data-attrs="{&quot;src&quot;:&quot;https://substack-post-media.s3.amazonaws.com/public/images/6a2affdf-e44f-452e-b02a-93809ad29f60_800x402.gif&quot;,&quot;srcNoWatermark&quot;:null,&quot;fullscreen&quot;:null,&quot;imageSize&quot;:null,&quot;height&quot;:402,&quot;width&quot;:800,&quot;resizeWidth&quot;:null,&quot;bytes&quot;:3369150,&quot;alt&quot;:null,&quot;title&quot;:null,&quot;type&quot;:&quot;image/gif&quot;,&quot;href&quot;:null,&quot;belowTheFold&quot;:false,&quot;topImage&quot;:true,&quot;internalRedirect&quot;:&quot;https://www.datatinkerer.io/i/182833470?img=https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F6a2affdf-e44f-452e-b02a-93809ad29f60_800x402.gif&quot;,&quot;isProcessing&quot;:false,&quot;align&quot;:null,&quot;offset&quot;:false}" class="sizing-normal" alt="" srcset="https://substackcdn.com/image/fetch/$s_!05-P!,w_424,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F6a2affdf-e44f-452e-b02a-93809ad29f60_800x402.gif 424w, https://substackcdn.com/image/fetch/$s_!05-P!,w_848,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F6a2affdf-e44f-452e-b02a-93809ad29f60_800x402.gif 848w, https://substackcdn.com/image/fetch/$s_!05-P!,w_1272,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F6a2affdf-e44f-452e-b02a-93809ad29f60_800x402.gif 1272w, https://substackcdn.com/image/fetch/$s_!05-P!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F6a2affdf-e44f-452e-b02a-93809ad29f60_800x402.gif 1456w" sizes="100vw" fetchpriority="high"></picture><div class="image-link-expand"><div class="pencraft pc-display-flex pc-gap-8 pc-reset"><button class="pencraft pc-reset pencraft icon-container restack-image buttonBase-GK1x3M" tabindex="0" type="button"><svg aria-hidden="true" width="20" height="20" viewBox="0 0 20 20" fill="none" stroke-width="1.5" stroke="var(--color-fg-primary)" stroke-linecap="round" stroke-linejoin="round" xmlns="http://www.w3.org/2000/svg" class="icon-noB79L"><g><path d="M2.53001 7.81595C3.49179 4.73911 6.43281 2.5 9.91173 2.5C13.1684 2.5 15.9537 4.46214 17.0852 7.23684L17.6179 8.67647M17.6179 8.67647L18.5002 4.26471M17.6179 8.67647L13.6473 6.91176M17.4995 12.1841C16.5378 15.2609 13.5967 17.5 10.1178 17.5C6.86118 17.5 4.07589 15.5379 2.94432 12.7632L2.41165 11.3235M2.41165 11.3235L1.5293 15.7353M2.41165 11.3235L6.38224 13.0882"></path></g></svg></button><button class="pencraft pc-reset pencraft icon-container view-image buttonBase-GK1x3M" tabindex="0" type="button"><svg xmlns="http://www.w3.org/2000/svg" width="20" height="20" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" class="lucide lucide-maximize2 lucide-maximize-2 icon-noB79L"><polyline points="15 3 21 3 21 9"></polyline><polyline points="9 21 3 21 3 15"></polyline><line x1="21" x2="14" y1="3" y2="10"></line><line x1="3" x2="10" y1="21" y2="14"></line></svg></button></div></div></div></a></figure></div><p>There are 100+ resources to learn all things data (science, engineering, analysis). It includes videos, courses, projects and can be filtered by tech stack (Python, SQL, Spark and etc), skill level (Beginner, Intermediate and so on) provider name or free/paid. So if you know other people who like staying up to date on all things data, please share Data Tinkerer with them!</p><p class="button-wrapper" data-attrs="{&quot;url&quot;:&quot;https://www.datatinkerer.io/leaderboard?&amp;utm_source=post&quot;,&quot;text&quot;:&quot;Refer a friend&quot;,&quot;action&quot;:null,&quot;class&quot;:null}" data-component-name="ButtonCreateButton"><a class="button primary" href="https://www.datatinkerer.io/leaderboard?&amp;utm_source=post"><span>Refer a friend</span></a></p><p>Now, with that out of the way, let&#8217;s get to Uber&#8217;s streaming solution</p>
      <p>
          <a href="https://www.datatinkerer.io/p/how-uber-cut-data-lake-freshness-from-hours-to-minutes-with-flink">
              Read more
          </a>
      </p>
   ]]></content:encoded></item><item><title><![CDATA[How Snap Rebuilt Its ML Platform to Handle 10,000+ Daily Spark Jobs]]></title><description><![CDATA[Inside Prism, the system that turned scattered Spark workflows into a unified, ML-ready platform.]]></description><link>https://www.datatinkerer.io/p/how-snap-rebuilt-its-ml-platform-to-handle-10000-daily-spark-jobs</link><guid isPermaLink="false">https://www.datatinkerer.io/p/how-snap-rebuilt-its-ml-platform-to-handle-10000-daily-spark-jobs</guid><dc:creator><![CDATA[Data Tinkerer]]></dc:creator><pubDate>Thu, 20 Nov 2025 04:59:47 GMT</pubDate><enclosure url="https://images.unsplash.com/photo-1620396749162-d61bdb715793?crop=entropy&amp;cs=tinysrgb&amp;fit=max&amp;fm=jpg&amp;ixid=M3wzMDAzMzh8MHwxfHNlYXJjaHw1fHxzbmFwY2hhdHxlbnwwfHx8fDE3NjM1Mjk0MDZ8MA&amp;ixlib=rb-4.1.0&amp;q=80&amp;w=1080" length="0" type="image/jpeg"/><content:encoded><![CDATA[<p>Fellow Data Tinkerers!</p><p>Today we will look at how Snap unified Spark, ML workflows and 10k+ daily jobs under one platform.</p><p>But before that, I wanted to share with you what you could unlock if you share Data Tinkerer with just <strong>1 more person</strong>.</p><div class="captioned-image-container"><figure><a class="image-link image2 is-viewable-img" target="_blank" href="https://substackcdn.com/image/fetch/$s_!TBRd!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F177f23c2-03d2-4ec5-bc67-33e5bb8b23fd_800x402.gif" data-component-name="Image2ToDOM"><div class="image2-inset"><picture><source type="image/webp" srcset="https://substackcdn.com/image/fetch/$s_!TBRd!,w_424,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F177f23c2-03d2-4ec5-bc67-33e5bb8b23fd_800x402.gif 424w, https://substackcdn.com/image/fetch/$s_!TBRd!,w_848,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F177f23c2-03d2-4ec5-bc67-33e5bb8b23fd_800x402.gif 848w, https://substackcdn.com/image/fetch/$s_!TBRd!,w_1272,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F177f23c2-03d2-4ec5-bc67-33e5bb8b23fd_800x402.gif 1272w, https://substackcdn.com/image/fetch/$s_!TBRd!,w_1456,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F177f23c2-03d2-4ec5-bc67-33e5bb8b23fd_800x402.gif 1456w" sizes="100vw"><img src="https://substackcdn.com/image/fetch/$s_!TBRd!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F177f23c2-03d2-4ec5-bc67-33e5bb8b23fd_800x402.gif" width="800" height="402" data-attrs="{&quot;src&quot;:&quot;https://substack-post-media.s3.amazonaws.com/public/images/177f23c2-03d2-4ec5-bc67-33e5bb8b23fd_800x402.gif&quot;,&quot;srcNoWatermark&quot;:null,&quot;fullscreen&quot;:null,&quot;imageSize&quot;:null,&quot;height&quot;:402,&quot;width&quot;:800,&quot;resizeWidth&quot;:null,&quot;bytes&quot;:3369150,&quot;alt&quot;:null,&quot;title&quot;:null,&quot;type&quot;:&quot;image/gif&quot;,&quot;href&quot;:null,&quot;belowTheFold&quot;:false,&quot;topImage&quot;:true,&quot;internalRedirect&quot;:&quot;https://www.datatinkerer.io/i/179211962?img=https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F177f23c2-03d2-4ec5-bc67-33e5bb8b23fd_800x402.gif&quot;,&quot;isProcessing&quot;:false,&quot;align&quot;:null,&quot;offset&quot;:false}" class="sizing-normal" alt="" srcset="https://substackcdn.com/image/fetch/$s_!TBRd!,w_424,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F177f23c2-03d2-4ec5-bc67-33e5bb8b23fd_800x402.gif 424w, https://substackcdn.com/image/fetch/$s_!TBRd!,w_848,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F177f23c2-03d2-4ec5-bc67-33e5bb8b23fd_800x402.gif 848w, https://substackcdn.com/image/fetch/$s_!TBRd!,w_1272,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F177f23c2-03d2-4ec5-bc67-33e5bb8b23fd_800x402.gif 1272w, https://substackcdn.com/image/fetch/$s_!TBRd!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F177f23c2-03d2-4ec5-bc67-33e5bb8b23fd_800x402.gif 1456w" sizes="100vw" fetchpriority="high"></picture><div class="image-link-expand"><div class="pencraft pc-display-flex pc-gap-8 pc-reset"><button tabindex="0" type="button" class="pencraft pc-reset pencraft icon-container restack-image buttonBase-GK1x3M"><svg aria-hidden="true" width="20" height="20" viewBox="0 0 20 20" fill="none" stroke-width="1.5" stroke="var(--color-fg-primary)" stroke-linecap="round" stroke-linejoin="round" xmlns="http://www.w3.org/2000/svg" class="icon-noB79L"><g><path d="M2.53001 7.81595C3.49179 4.73911 6.43281 2.5 9.91173 2.5C13.1684 2.5 15.9537 4.46214 17.0852 7.23684L17.6179 8.67647M17.6179 8.67647L18.5002 4.26471M17.6179 8.67647L13.6473 6.91176M17.4995 12.1841C16.5378 15.2609 13.5967 17.5 10.1178 17.5C6.86118 17.5 4.07589 15.5379 2.94432 12.7632L2.41165 11.3235M2.41165 11.3235L1.5293 15.7353M2.41165 11.3235L6.38224 13.0882"></path></g></svg></button><button tabindex="0" type="button" class="pencraft pc-reset pencraft icon-container view-image buttonBase-GK1x3M"><svg xmlns="http://www.w3.org/2000/svg" width="20" height="20" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" class="lucide lucide-maximize2 lucide-maximize-2 icon-noB79L"><polyline points="15 3 21 3 21 9"></polyline><polyline points="9 21 3 21 3 15"></polyline><line x1="21" x2="14" y1="3" y2="10"></line><line x1="3" x2="10" y1="21" y2="14"></line></svg></button></div></div></div></a></figure></div><p>There are 100+ resources to learn all things data (science, engineering, analysis). It includes videos, courses, projects and can be filtered by tech stack (Python, SQL, Spark and etc), skill level (Beginner, Intermediate and so on) provider name or free/paid. So if you know other people who like staying up to date on all things data, please share Data Tinkerer with them!</p><p class="button-wrapper" data-attrs="{&quot;url&quot;:&quot;https://www.datatinkerer.io/leaderboard?&amp;utm_source=post&quot;,&quot;text&quot;:&quot;Refer a friend&quot;,&quot;action&quot;:null,&quot;class&quot;:null}" data-component-name="ButtonCreateButton"><a class="button primary" href="https://www.datatinkerer.io/leaderboard?&amp;utm_source=post"><span>Refer a friend</span></a></p><p>Now, with that out of the way, let&#8217;s get to Snap&#8217;s ML platform transformation.</p>
      <p>
          <a href="https://www.datatinkerer.io/p/how-snap-rebuilt-its-ml-platform-to-handle-10000-daily-spark-jobs">
              Read more
          </a>
      </p>
   ]]></content:encoded></item><item><title><![CDATA[From Marketing to Data Engineering: How I Made the Switch]]></title><description><![CDATA[How one marketer followed the trail of tracking pixels into pipelines and built a career turning messy data into usable systems.]]></description><link>https://www.datatinkerer.io/p/from-marketing-to-data-engineering-how-i-made-the-switch</link><guid isPermaLink="false">https://www.datatinkerer.io/p/from-marketing-to-data-engineering-how-i-made-the-switch</guid><dc:creator><![CDATA[Data Tinkerer]]></dc:creator><pubDate>Thu, 16 Oct 2025 04:01:28 GMT</pubDate><enclosure url="https://substack-post-media.s3.amazonaws.com/public/images/9f252334-7437-40b5-9f82-08c981de2f6d_761x764.png" length="0" type="image/jpeg"/><content:encoded><![CDATA[<p>Fellow Data Tinkerers,</p><p>Lately, I&#8217;ve been thinking about starting a new series where people working in data share how they got here, what they&#8217;ve learned along the way and what their day-to-day looks like.</p><p>So, I&#8217;m kicking it off today with <span class="mention-wrap" data-attrs="{&quot;name&quot;:&quot;Alejandro Aboy&quot;,&quot;id&quot;:22949723,&quot;type&quot;:&quot;user&quot;,&quot;url&quot;:null,&quot;photo_url&quot;:&quot;https://substackcdn.com/image/fetch/$s_!u1Ao!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fdca2c63d-9f5e-4cd3-99ac-7d8e71dc114b_1024x1024.jpeg&quot;,&quot;uuid&quot;:&quot;ee6ab692-9d69-4714-8986-9b599e2b5557&quot;}" data-component-name="MentionToDOM"></span>, Senior Data Engineer at Workpath and writer of <em>The Pipe and The Line</em> newsletter.</p><div class="embedded-publication-wrap" data-attrs="{&quot;id&quot;:1196229,&quot;embedding_publication_id&quot;:null,&quot;name&quot;:&quot;The Pipe &amp; The Line&quot;,&quot;logo_url&quot;:&quot;https://substackcdn.com/image/fetch/$s_!vmrQ!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F4d5b2131-da28-4621-ad6f-9574cbc41a1e_500x500.png&quot;,&quot;base_url&quot;:&quot;https://thepipeandtheline.substack.com&quot;,&quot;hero_text&quot;:&quot;Hands-on guides, tools, and experiments to sharpen your Data &amp; AI Engineering skills from someone who learned it all in the wild.&quot;,&quot;author_name&quot;:&quot;Alejandro Aboy&quot;,&quot;show_subscribe&quot;:true,&quot;logo_bg_color&quot;:&quot;#131826&quot;,&quot;language&quot;:&quot;en&quot;}" data-component-name="EmbeddedPublicationToDOMWithSubscribe"><div class="embedded-publication show-subscribe"><a class="embedded-publication-link-part" native="true" href="https://thepipeandtheline.substack.com?utm_source=substack&amp;utm_campaign=publication_embed&amp;utm_medium=web"><img class="embedded-publication-logo" src="https://substackcdn.com/image/fetch/$s_!vmrQ!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F4d5b2131-da28-4621-ad6f-9574cbc41a1e_500x500.png" width="56" height="56" style="background-color: rgb(19, 24, 38);"><span class="embedded-publication-name">The Pipe &amp; The Line</span><div class="embedded-publication-hero-text">Hands-on guides, tools, and experiments to sharpen your Data &amp; AI Engineering skills from someone who learned it all in the wild.</div><div class="embedded-publication-author-name">By Alejandro Aboy</div></a><form class="embedded-publication-subscribe" method="GET" action="https://thepipeandtheline.substack.com/subscribe?"><input type="hidden" name="source" value="publication-embed"><input type="hidden" name="autoSubmit" value="true"><input type="email" class="email-input" name="email" placeholder="Type your email..."><input type="submit" class="button primary" value="Subscribe"></form></div></div><p>We talked about how he went from marketing to data engineering, what his workflow looks like, why he was called an <em>octopus</em> and why he thinks &#8220;big data&#8221; is a fool&#8217;s errand for most teams.</p><p>So without further ado, let&#8217;s get into it!</p>
      <p>
          <a href="https://www.datatinkerer.io/p/from-marketing-to-data-engineering-how-i-made-the-switch">
              Read more
          </a>
      </p>
   ]]></content:encoded></item><item><title><![CDATA[How Shopify Uses Change Data Capture to Serve Millions of Merchants]]></title><description><![CDATA[From batch queries to streaming 100k records per second during peak load]]></description><link>https://www.datatinkerer.io/p/how-shopify-uses-change-data-capture-to-serve-millions-of-merchants</link><guid isPermaLink="false">https://www.datatinkerer.io/p/how-shopify-uses-change-data-capture-to-serve-millions-of-merchants</guid><dc:creator><![CDATA[Data Tinkerer]]></dc:creator><pubDate>Thu, 18 Sep 2025 07:53:42 GMT</pubDate><enclosure url="https://images.unsplash.com/photo-1730818874996-dea4bddf5554?crop=entropy&amp;cs=tinysrgb&amp;fit=max&amp;fm=jpg&amp;ixid=M3wzMDAzMzh8MHwxfHNlYXJjaHw1fHxzaG9waWZ5fGVufDB8fHx8MTc1ODE4MDY0NHww&amp;ixlib=rb-4.1.0&amp;q=80&amp;w=1080" length="0" type="image/jpeg"/><content:encoded><![CDATA[<p>Fellow Data Tinkerers!</p><p>Today we will look at how Shopify built a real-time data pipeline at 400TB scale.</p><p>But before that, I wanted to share with you what you could unlock if you share Data Tinkerer with just <strong>1 more person</strong>.</p><div class="captioned-image-container"><figure><a class="image-link image2 is-viewable-img" target="_blank" href="https://substackcdn.com/image/fetch/$s_!K9fn!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fa1ed777b-9a73-44d8-9095-d6c0df4aa853_800x402.gif" data-component-name="Image2ToDOM"><div class="image2-inset"><picture><source type="image/webp" srcset="https://substackcdn.com/image/fetch/$s_!K9fn!,w_424,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fa1ed777b-9a73-44d8-9095-d6c0df4aa853_800x402.gif 424w, https://substackcdn.com/image/fetch/$s_!K9fn!,w_848,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fa1ed777b-9a73-44d8-9095-d6c0df4aa853_800x402.gif 848w, https://substackcdn.com/image/fetch/$s_!K9fn!,w_1272,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fa1ed777b-9a73-44d8-9095-d6c0df4aa853_800x402.gif 1272w, https://substackcdn.com/image/fetch/$s_!K9fn!,w_1456,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fa1ed777b-9a73-44d8-9095-d6c0df4aa853_800x402.gif 1456w" sizes="100vw"><img src="https://substackcdn.com/image/fetch/$s_!K9fn!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fa1ed777b-9a73-44d8-9095-d6c0df4aa853_800x402.gif" width="800" height="402" data-attrs="{&quot;src&quot;:&quot;https://substack-post-media.s3.amazonaws.com/public/images/a1ed777b-9a73-44d8-9095-d6c0df4aa853_800x402.gif&quot;,&quot;srcNoWatermark&quot;:null,&quot;fullscreen&quot;:null,&quot;imageSize&quot;:null,&quot;height&quot;:402,&quot;width&quot;:800,&quot;resizeWidth&quot;:null,&quot;bytes&quot;:3369150,&quot;alt&quot;:null,&quot;title&quot;:null,&quot;type&quot;:&quot;image/gif&quot;,&quot;href&quot;:null,&quot;belowTheFold&quot;:false,&quot;topImage&quot;:true,&quot;internalRedirect&quot;:&quot;https://www.datatinkerer.io/i/173822667?img=https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fa1ed777b-9a73-44d8-9095-d6c0df4aa853_800x402.gif&quot;,&quot;isProcessing&quot;:false,&quot;align&quot;:null,&quot;offset&quot;:false}" class="sizing-normal" alt="" srcset="https://substackcdn.com/image/fetch/$s_!K9fn!,w_424,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fa1ed777b-9a73-44d8-9095-d6c0df4aa853_800x402.gif 424w, https://substackcdn.com/image/fetch/$s_!K9fn!,w_848,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fa1ed777b-9a73-44d8-9095-d6c0df4aa853_800x402.gif 848w, https://substackcdn.com/image/fetch/$s_!K9fn!,w_1272,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fa1ed777b-9a73-44d8-9095-d6c0df4aa853_800x402.gif 1272w, https://substackcdn.com/image/fetch/$s_!K9fn!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fa1ed777b-9a73-44d8-9095-d6c0df4aa853_800x402.gif 1456w" sizes="100vw" fetchpriority="high"></picture><div class="image-link-expand"><div class="pencraft pc-display-flex pc-gap-8 pc-reset"><button tabindex="0" type="button" class="pencraft pc-reset pencraft icon-container restack-image buttonBase-GK1x3M"><svg aria-hidden="true" width="20" height="20" viewBox="0 0 20 20" fill="none" stroke-width="1.5" stroke="var(--color-fg-primary)" stroke-linecap="round" stroke-linejoin="round" xmlns="http://www.w3.org/2000/svg" class="icon-noB79L"><g><path d="M2.53001 7.81595C3.49179 4.73911 6.43281 2.5 9.91173 2.5C13.1684 2.5 15.9537 4.46214 17.0852 7.23684L17.6179 8.67647M17.6179 8.67647L18.5002 4.26471M17.6179 8.67647L13.6473 6.91176M17.4995 12.1841C16.5378 15.2609 13.5967 17.5 10.1178 17.5C6.86118 17.5 4.07589 15.5379 2.94432 12.7632L2.41165 11.3235M2.41165 11.3235L1.5293 15.7353M2.41165 11.3235L6.38224 13.0882"></path></g></svg></button><button tabindex="0" type="button" class="pencraft pc-reset pencraft icon-container view-image buttonBase-GK1x3M"><svg xmlns="http://www.w3.org/2000/svg" width="20" height="20" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" class="lucide lucide-maximize2 lucide-maximize-2 icon-noB79L"><polyline points="15 3 21 3 21 9"></polyline><polyline points="9 21 3 21 3 15"></polyline><line x1="21" x2="14" y1="3" y2="10"></line><line x1="3" x2="10" y1="21" y2="14"></line></svg></button></div></div></div></a></figure></div><p>There are 100+ resources to learn all things data (science, engineering, analysis). It includes videos, courses, projects and can be filtered by tech stack (Python, SQL, Spark and etc), skill level (Beginner, Intermediate and so on) provider name or free/paid. So if you know other people who like staying up to date on all things data, please share Data Tinkerer with them!</p><p class="button-wrapper" data-attrs="{&quot;url&quot;:&quot;https://www.datatinkerer.io/leaderboard?&amp;utm_source=post&quot;,&quot;text&quot;:&quot;Refer a friend&quot;,&quot;action&quot;:null,&quot;class&quot;:null}" data-component-name="ButtonCreateButton"><a class="button primary" href="https://www.datatinkerer.io/leaderboard?&amp;utm_source=post"><span>Refer a friend</span></a></p><p>Now, with that out of the way, let&#8217;s get to change data capture at Shopify!</p>
      <p>
          <a href="https://www.datatinkerer.io/p/how-shopify-uses-change-data-capture-to-serve-millions-of-merchants">
              Read more
          </a>
      </p>
   ]]></content:encoded></item><item><title><![CDATA[How Grab Shrunk Real-Time Queries from 5 Minutes to 1 with FlinkSQL and Kafka]]></title><description><![CDATA[With SQL as the interface, analysts and engineers can now explore streams and deploy pipelines in under 10 minutes.]]></description><link>https://www.datatinkerer.io/p/how-grab-shrunk-real-time-queries-from-five-minutes-to-one</link><guid isPermaLink="false">https://www.datatinkerer.io/p/how-grab-shrunk-real-time-queries-from-five-minutes-to-one</guid><dc:creator><![CDATA[Data Tinkerer]]></dc:creator><pubDate>Thu, 21 Aug 2025 06:45:35 GMT</pubDate><enclosure url="https://images.unsplash.com/photo-1587476351660-e9fa4bb8b26c?crop=entropy&amp;cs=tinysrgb&amp;fit=max&amp;fm=jpg&amp;ixid=M3wzMDAzMzh8MHwxfHNlYXJjaHwyfHxncmFifGVufDB8fHx8MTc1NTQ5OTAzOHww&amp;ixlib=rb-4.1.0&amp;q=80&amp;w=1080" length="0" type="image/jpeg"/><content:encoded><![CDATA[<p>Fellow Data Tinkerers!</p><p>Today we will look at how Grab made real-time processing faster for its users. </p><p>But before that, I wanted to share with you what you could unlock if you share Data Tinkerer with just <strong>1 more person</strong>.</p><div class="captioned-image-container"><figure><a class="image-link image2 is-viewable-img" target="_blank" href="https://substackcdn.com/image/fetch/$s_!4JBO!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fc192a623-a624-4963-a1d5-a293bf3f7e08_800x402.gif" data-component-name="Image2ToDOM"><div class="image2-inset"><picture><source type="image/webp" srcset="https://substackcdn.com/image/fetch/$s_!4JBO!,w_424,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fc192a623-a624-4963-a1d5-a293bf3f7e08_800x402.gif 424w, https://substackcdn.com/image/fetch/$s_!4JBO!,w_848,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fc192a623-a624-4963-a1d5-a293bf3f7e08_800x402.gif 848w, https://substackcdn.com/image/fetch/$s_!4JBO!,w_1272,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fc192a623-a624-4963-a1d5-a293bf3f7e08_800x402.gif 1272w, https://substackcdn.com/image/fetch/$s_!4JBO!,w_1456,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fc192a623-a624-4963-a1d5-a293bf3f7e08_800x402.gif 1456w" sizes="100vw"><img src="https://substackcdn.com/image/fetch/$s_!4JBO!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fc192a623-a624-4963-a1d5-a293bf3f7e08_800x402.gif" width="800" height="402" data-attrs="{&quot;src&quot;:&quot;https://substack-post-media.s3.amazonaws.com/public/images/c192a623-a624-4963-a1d5-a293bf3f7e08_800x402.gif&quot;,&quot;srcNoWatermark&quot;:null,&quot;fullscreen&quot;:null,&quot;imageSize&quot;:null,&quot;height&quot;:402,&quot;width&quot;:800,&quot;resizeWidth&quot;:null,&quot;bytes&quot;:3369150,&quot;alt&quot;:null,&quot;title&quot;:null,&quot;type&quot;:&quot;image/gif&quot;,&quot;href&quot;:null,&quot;belowTheFold&quot;:false,&quot;topImage&quot;:true,&quot;internalRedirect&quot;:&quot;https://www.datatinkerer.io/i/171226398?img=https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fc192a623-a624-4963-a1d5-a293bf3f7e08_800x402.gif&quot;,&quot;isProcessing&quot;:false,&quot;align&quot;:null,&quot;offset&quot;:false}" class="sizing-normal" alt="" srcset="https://substackcdn.com/image/fetch/$s_!4JBO!,w_424,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fc192a623-a624-4963-a1d5-a293bf3f7e08_800x402.gif 424w, https://substackcdn.com/image/fetch/$s_!4JBO!,w_848,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fc192a623-a624-4963-a1d5-a293bf3f7e08_800x402.gif 848w, https://substackcdn.com/image/fetch/$s_!4JBO!,w_1272,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fc192a623-a624-4963-a1d5-a293bf3f7e08_800x402.gif 1272w, https://substackcdn.com/image/fetch/$s_!4JBO!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fc192a623-a624-4963-a1d5-a293bf3f7e08_800x402.gif 1456w" sizes="100vw" fetchpriority="high"></picture><div class="image-link-expand"><div class="pencraft pc-display-flex pc-gap-8 pc-reset"><button class="pencraft pc-reset pencraft icon-container restack-image buttonBase-GK1x3M" tabindex="0" type="button"><svg aria-hidden="true" width="20" height="20" viewBox="0 0 20 20" fill="none" stroke-width="1.5" stroke="var(--color-fg-primary)" stroke-linecap="round" stroke-linejoin="round" xmlns="http://www.w3.org/2000/svg" class="icon-noB79L"><g><path d="M2.53001 7.81595C3.49179 4.73911 6.43281 2.5 9.91173 2.5C13.1684 2.5 15.9537 4.46214 17.0852 7.23684L17.6179 8.67647M17.6179 8.67647L18.5002 4.26471M17.6179 8.67647L13.6473 6.91176M17.4995 12.1841C16.5378 15.2609 13.5967 17.5 10.1178 17.5C6.86118 17.5 4.07589 15.5379 2.94432 12.7632L2.41165 11.3235M2.41165 11.3235L1.5293 15.7353M2.41165 11.3235L6.38224 13.0882"></path></g></svg></button><button class="pencraft pc-reset pencraft icon-container view-image buttonBase-GK1x3M" tabindex="0" type="button"><svg xmlns="http://www.w3.org/2000/svg" width="20" height="20" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" class="lucide lucide-maximize2 lucide-maximize-2 icon-noB79L"><polyline points="15 3 21 3 21 9"></polyline><polyline points="9 21 3 21 3 15"></polyline><line x1="21" x2="14" y1="3" y2="10"></line><line x1="3" x2="10" y1="21" y2="14"></line></svg></button></div></div></div></a></figure></div><p>There are 100+ resources to learn all things data (science, engineering, analysis). It includes videos, courses, projects and can be filtered by tech stack (Python, SQL, Spark and etc), skill level (Beginner, Intermediate and so on) provider name or free/paid. So if you know other people who like staying up to date on all things data, please share Data Tinkerer with them!</p><p class="button-wrapper" data-attrs="{&quot;url&quot;:&quot;https://www.datatinkerer.io/leaderboard?&amp;utm_source=post&quot;,&quot;text&quot;:&quot;Refer a friend&quot;,&quot;action&quot;:null,&quot;class&quot;:null}" data-component-name="ButtonCreateButton"><a class="button primary" href="https://www.datatinkerer.io/leaderboard?&amp;utm_source=post"><span>Refer a friend</span></a></p><p>Now, with that out of the way, let&#8217;s get to real-time processing at Grab!</p>
      <p>
          <a href="https://www.datatinkerer.io/p/how-grab-shrunk-real-time-queries-from-five-minutes-to-one">
              Read more
          </a>
      </p>
   ]]></content:encoded></item><item><title><![CDATA[How Expedia Monitors 1000+ A/B Tests in Real Time with Flink and Kafka]]></title><description><![CDATA[A look inside the pipeline that spots underperforming experiments in minutes and not days]]></description><link>https://www.datatinkerer.io/p/how-expedia-monitors-1000-ab-tests-in-real-time-with-flink-and-kafka</link><guid isPermaLink="false">https://www.datatinkerer.io/p/how-expedia-monitors-1000-ab-tests-in-real-time-with-flink-and-kafka</guid><dc:creator><![CDATA[Data Tinkerer]]></dc:creator><pubDate>Thu, 24 Jul 2025 07:55:43 GMT</pubDate><enclosure url="https://substackcdn.com/image/fetch/$s_!jZ_b!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F961833e9-d13b-4216-a74d-8aebaa3c9fc1_1024x1024.png" length="0" type="image/jpeg"/><content:encoded><![CDATA[<p>Fellow Data Tinkerers!</p><p>Today we will look at how Expedia Group monitors A/B tests at a large scale</p><p>But before that, I wanted to share an example of what you could unlock if you share Data Tinkerer with just <strong>2 other people </strong>and they subscribe to the newsletter.</p><div class="captioned-image-container"><figure><a class="image-link image2 is-viewable-img" target="_blank" href="https://substackcdn.com/image/fetch/$s_!b1ZD!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F5d63d619-0614-4aba-ae5d-81e6863b4663_1650x1275.jpeg" data-component-name="Image2ToDOM"><div class="image2-inset"><picture><source type="image/webp" srcset="https://substackcdn.com/image/fetch/$s_!b1ZD!,w_424,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F5d63d619-0614-4aba-ae5d-81e6863b4663_1650x1275.jpeg 424w, https://substackcdn.com/image/fetch/$s_!b1ZD!,w_848,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F5d63d619-0614-4aba-ae5d-81e6863b4663_1650x1275.jpeg 848w, https://substackcdn.com/image/fetch/$s_!b1ZD!,w_1272,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F5d63d619-0614-4aba-ae5d-81e6863b4663_1650x1275.jpeg 1272w, https://substackcdn.com/image/fetch/$s_!b1ZD!,w_1456,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F5d63d619-0614-4aba-ae5d-81e6863b4663_1650x1275.jpeg 1456w" sizes="100vw"><img src="https://substackcdn.com/image/fetch/$s_!b1ZD!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F5d63d619-0614-4aba-ae5d-81e6863b4663_1650x1275.jpeg" width="1456" height="1125" data-attrs="{&quot;src&quot;:&quot;https://substack-post-media.s3.amazonaws.com/public/images/5d63d619-0614-4aba-ae5d-81e6863b4663_1650x1275.jpeg&quot;,&quot;srcNoWatermark&quot;:null,&quot;fullscreen&quot;:null,&quot;imageSize&quot;:null,&quot;height&quot;:1125,&quot;width&quot;:1456,&quot;resizeWidth&quot;:null,&quot;bytes&quot;:1291918,&quot;alt&quot;:null,&quot;title&quot;:null,&quot;type&quot;:&quot;image/jpeg&quot;,&quot;href&quot;:null,&quot;belowTheFold&quot;:false,&quot;topImage&quot;:true,&quot;internalRedirect&quot;:&quot;https://www.datatinkerer.io/i/169094273?img=https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F5d63d619-0614-4aba-ae5d-81e6863b4663_1650x1275.jpeg&quot;,&quot;isProcessing&quot;:false,&quot;align&quot;:null,&quot;offset&quot;:false}" class="sizing-normal" alt="" srcset="https://substackcdn.com/image/fetch/$s_!b1ZD!,w_424,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F5d63d619-0614-4aba-ae5d-81e6863b4663_1650x1275.jpeg 424w, https://substackcdn.com/image/fetch/$s_!b1ZD!,w_848,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F5d63d619-0614-4aba-ae5d-81e6863b4663_1650x1275.jpeg 848w, https://substackcdn.com/image/fetch/$s_!b1ZD!,w_1272,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F5d63d619-0614-4aba-ae5d-81e6863b4663_1650x1275.jpeg 1272w, https://substackcdn.com/image/fetch/$s_!b1ZD!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F5d63d619-0614-4aba-ae5d-81e6863b4663_1650x1275.jpeg 1456w" sizes="100vw" fetchpriority="high"></picture><div class="image-link-expand"><div class="pencraft pc-display-flex pc-gap-8 pc-reset"><button class="pencraft pc-reset pencraft icon-container restack-image buttonBase-tqxFUX" tabindex="0" type="button"><svg aria-hidden="true" width="20" height="20" viewBox="0 0 20 20" fill="none" stroke-width="1.5" stroke="var(--color-fg-primary)" stroke-linecap="round" stroke-linejoin="round" xmlns="http://www.w3.org/2000/svg" class="icon-noB79L"><g><path d="M2.53001 7.81595C3.49179 4.73911 6.43281 2.5 9.91173 2.5C13.1684 2.5 15.9537 4.46214 17.0852 7.23684L17.6179 8.67647M17.6179 8.67647L18.5002 4.26471M17.6179 8.67647L13.6473 6.91176M17.4995 12.1841C16.5378 15.2609 13.5967 17.5 10.1178 17.5C6.86118 17.5 4.07589 15.5379 2.94432 12.7632L2.41165 11.3235M2.41165 11.3235L1.5293 15.7353M2.41165 11.3235L6.38224 13.0882"></path></g></svg></button><button class="pencraft pc-reset pencraft icon-container view-image buttonBase-tqxFUX" tabindex="0" type="button"><svg xmlns="http://www.w3.org/2000/svg" width="20" height="20" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" class="lucide lucide-maximize2 lucide-maximize-2 icon-noB79L"><polyline points="15 3 21 3 21 9"></polyline><polyline points="9 21 3 21 3 15"></polyline><line x1="21" x2="14" y1="3" y2="10"></line><line x1="3" x2="10" y1="21" y2="14"></line></svg></button></div></div></div></a></figure></div><p>There are 100+ more cheat sheets covering everything from Python, R, SQL, Spark to Power BI, Tableau, Git and many more. So if you know other people who like staying up to date on all things data, please share Data Tinkerer with them!</p><p class="button-wrapper" data-attrs="{&quot;url&quot;:&quot;https://www.datatinkerer.io/leaderboard?&amp;referrer_token=4tlsmi&amp;utm_source=post&quot;,&quot;text&quot;:&quot;Refer a friend&quot;,&quot;action&quot;:null,&quot;class&quot;:&quot;button-wrapper&quot;}" data-component-name="ButtonCreateButton"><a class="button primary button-wrapper" href="https://www.datatinkerer.io/leaderboard?&amp;referrer_token=4tlsmi&amp;utm_source=post"><span>Refer a friend</span></a></p><p>Now, with that out of the way, let&#8217;s get the real-time monitoring of A/B tests by Expedia</p>
      <p>
          <a href="https://www.datatinkerer.io/p/how-expedia-monitors-1000-ab-tests-in-real-time-with-flink-and-kafka">
              Read more
          </a>
      </p>
   ]]></content:encoded></item><item><title><![CDATA[How Bolt Reconciles €2B in Revenue Using Airflow, Spark and dbt]]></title><description><![CDATA[A look under the hood of a multi-country finance pipeline that ingests raw data, models discrepancies and reconciles cash flows at scale.]]></description><link>https://www.datatinkerer.io/p/how-bolt-reconciles-2b-in-revenue-using-airflow-spark-and-dbt</link><guid isPermaLink="false">https://www.datatinkerer.io/p/how-bolt-reconciles-2b-in-revenue-using-airflow-spark-and-dbt</guid><dc:creator><![CDATA[Data Tinkerer]]></dc:creator><pubDate>Thu, 03 Jul 2025 09:53:52 GMT</pubDate><enclosure url="https://images.unsplash.com/photo-1715351123666-6a9c4f180c54?crop=entropy&amp;cs=tinysrgb&amp;fit=max&amp;fm=jpg&amp;ixid=M3wzMDAzMzh8MHwxfHNlYXJjaHw1MXx8Ym9sdHxlbnwwfHx8fDE3NTE0MzU2NjN8MA&amp;ixlib=rb-4.1.0&amp;q=80&amp;w=1080" length="0" type="image/jpeg"/><content:encoded><![CDATA[<p>Fellow Data Tinkerers!</p><p>Today we will look at how Bolt tracks payments at scale.</p><p>But before that, I wanted to share an example of what you could unlock if you share Data Tinkerer with just <strong>2 other people </strong>and they subscribe to the newsletter.</p><div class="captioned-image-container"><figure><a class="image-link image2 is-viewable-img" target="_blank" href="https://substackcdn.com/image/fetch/$s_!ufW-!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fea22c22d-33fa-419b-a1db-9e654a2510d6_1650x1275.jpeg" data-component-name="Image2ToDOM"><div class="image2-inset"><picture><source type="image/webp" srcset="https://substackcdn.com/image/fetch/$s_!ufW-!,w_424,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fea22c22d-33fa-419b-a1db-9e654a2510d6_1650x1275.jpeg 424w, https://substackcdn.com/image/fetch/$s_!ufW-!,w_848,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fea22c22d-33fa-419b-a1db-9e654a2510d6_1650x1275.jpeg 848w, https://substackcdn.com/image/fetch/$s_!ufW-!,w_1272,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fea22c22d-33fa-419b-a1db-9e654a2510d6_1650x1275.jpeg 1272w, https://substackcdn.com/image/fetch/$s_!ufW-!,w_1456,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fea22c22d-33fa-419b-a1db-9e654a2510d6_1650x1275.jpeg 1456w" sizes="100vw"><img src="https://substackcdn.com/image/fetch/$s_!ufW-!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fea22c22d-33fa-419b-a1db-9e654a2510d6_1650x1275.jpeg" width="1456" height="1125" data-attrs="{&quot;src&quot;:&quot;https://substack-post-media.s3.amazonaws.com/public/images/ea22c22d-33fa-419b-a1db-9e654a2510d6_1650x1275.jpeg&quot;,&quot;srcNoWatermark&quot;:null,&quot;fullscreen&quot;:null,&quot;imageSize&quot;:null,&quot;height&quot;:1125,&quot;width&quot;:1456,&quot;resizeWidth&quot;:null,&quot;bytes&quot;:1357928,&quot;alt&quot;:null,&quot;title&quot;:null,&quot;type&quot;:&quot;image/jpeg&quot;,&quot;href&quot;:null,&quot;belowTheFold&quot;:false,&quot;topImage&quot;:true,&quot;internalRedirect&quot;:&quot;https://www.datatinkerer.io/i/167406813?img=https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fea22c22d-33fa-419b-a1db-9e654a2510d6_1650x1275.jpeg&quot;,&quot;isProcessing&quot;:false,&quot;align&quot;:null,&quot;offset&quot;:false}" class="sizing-normal" alt="" srcset="https://substackcdn.com/image/fetch/$s_!ufW-!,w_424,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fea22c22d-33fa-419b-a1db-9e654a2510d6_1650x1275.jpeg 424w, https://substackcdn.com/image/fetch/$s_!ufW-!,w_848,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fea22c22d-33fa-419b-a1db-9e654a2510d6_1650x1275.jpeg 848w, https://substackcdn.com/image/fetch/$s_!ufW-!,w_1272,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fea22c22d-33fa-419b-a1db-9e654a2510d6_1650x1275.jpeg 1272w, https://substackcdn.com/image/fetch/$s_!ufW-!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fea22c22d-33fa-419b-a1db-9e654a2510d6_1650x1275.jpeg 1456w" sizes="100vw" fetchpriority="high"></picture><div class="image-link-expand"><div class="pencraft pc-display-flex pc-gap-8 pc-reset"><button class="pencraft pc-reset pencraft icon-container restack-image buttonBase-tqxFUX" tabindex="0" type="button"><svg aria-hidden="true" width="20" height="20" viewBox="0 0 20 20" fill="none" stroke-width="1.5" stroke="var(--color-fg-primary)" stroke-linecap="round" stroke-linejoin="round" xmlns="http://www.w3.org/2000/svg" class="icon-noB79L"><g><path d="M2.53001 7.81595C3.49179 4.73911 6.43281 2.5 9.91173 2.5C13.1684 2.5 15.9537 4.46214 17.0852 7.23684L17.6179 8.67647M17.6179 8.67647L18.5002 4.26471M17.6179 8.67647L13.6473 6.91176M17.4995 12.1841C16.5378 15.2609 13.5967 17.5 10.1178 17.5C6.86118 17.5 4.07589 15.5379 2.94432 12.7632L2.41165 11.3235M2.41165 11.3235L1.5293 15.7353M2.41165 11.3235L6.38224 13.0882"></path></g></svg></button><button class="pencraft pc-reset pencraft icon-container view-image buttonBase-tqxFUX" tabindex="0" type="button"><svg xmlns="http://www.w3.org/2000/svg" width="20" height="20" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" class="lucide lucide-maximize2 lucide-maximize-2 icon-noB79L"><polyline points="15 3 21 3 21 9"></polyline><polyline points="9 21 3 21 3 15"></polyline><line x1="21" x2="14" y1="3" y2="10"></line><line x1="3" x2="10" y1="21" y2="14"></line></svg></button></div></div></div></a></figure></div><p>There are 100+ more cheat sheets covering everything from Python, R, SQL, Spark to Power BI, Tableau, Git and many more. So if you know other people who like staying up to date on all things data, please share Data Tinkerer with them!</p><p class="button-wrapper" data-attrs="{&quot;url&quot;:&quot;https://www.datatinkerer.io/leaderboard?&amp;referrer_token=4tlsmi&amp;utm_source=post&quot;,&quot;text&quot;:&quot;Refer a friend&quot;,&quot;action&quot;:null,&quot;class&quot;:&quot;button-wrapper&quot;}" data-component-name="ButtonCreateButton"><a class="button primary button-wrapper" href="https://www.datatinkerer.io/leaderboard?&amp;referrer_token=4tlsmi&amp;utm_source=post"><span>Refer a friend</span></a></p><p>Now, with that out of the way, let&#8217;s get to how Bolt deals with processing payments from millions of customers</p>
      <p>
          <a href="https://www.datatinkerer.io/p/how-bolt-reconciles-2b-in-revenue-using-airflow-spark-and-dbt">
              Read more
          </a>
      </p>
   ]]></content:encoded></item><item><title><![CDATA[How Flipkart Scaled Delivery Date Calculation 10x While Slashing Latency by 90%]]></title><description><![CDATA[Optimising for 100 items in 100ms without breaking the backend (or the bank)]]></description><link>https://www.datatinkerer.io/p/how-flipkart-scaled-delivery-date-calculation-10x-while-slashing-latency-by-90-percent</link><guid isPermaLink="false">https://www.datatinkerer.io/p/how-flipkart-scaled-delivery-date-calculation-10x-while-slashing-latency-by-90-percent</guid><dc:creator><![CDATA[Data Tinkerer]]></dc:creator><pubDate>Fri, 13 Jun 2025 09:10:50 GMT</pubDate><enclosure url="https://substackcdn.com/image/fetch/f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F5e325e31-1188-4171-a6d4-9d88b490de17_1024x1536.png" length="0" type="image/jpeg"/><content:encoded><![CDATA[<p>Fellow Data Tinkerers!</p><p>Today we will look at how Flipkart solved the problem of calculating delivery date.</p><p>But before that, I wanted to share an example of what you could unlock if you share Data Tinkerer with just <strong>2 other people</strong>.</p><div class="captioned-image-container"><figure><a class="image-link image2 is-viewable-img" target="_blank" href="https://substackcdn.com/image/fetch/$s_!-CMo!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fadf294af-e15c-41c8-9dfd-c0541054526b_3300x2550.jpeg" data-component-name="Image2ToDOM"><div class="image2-inset"><picture><source type="image/webp" srcset="https://substackcdn.com/image/fetch/$s_!-CMo!,w_424,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fadf294af-e15c-41c8-9dfd-c0541054526b_3300x2550.jpeg 424w, https://substackcdn.com/image/fetch/$s_!-CMo!,w_848,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fadf294af-e15c-41c8-9dfd-c0541054526b_3300x2550.jpeg 848w, https://substackcdn.com/image/fetch/$s_!-CMo!,w_1272,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fadf294af-e15c-41c8-9dfd-c0541054526b_3300x2550.jpeg 1272w, https://substackcdn.com/image/fetch/$s_!-CMo!,w_1456,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fadf294af-e15c-41c8-9dfd-c0541054526b_3300x2550.jpeg 1456w" sizes="100vw"><img src="https://substackcdn.com/image/fetch/$s_!-CMo!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fadf294af-e15c-41c8-9dfd-c0541054526b_3300x2550.jpeg" width="1456" height="1125" data-attrs="{&quot;src&quot;:&quot;https://substack-post-media.s3.amazonaws.com/public/images/adf294af-e15c-41c8-9dfd-c0541054526b_3300x2550.jpeg&quot;,&quot;srcNoWatermark&quot;:null,&quot;fullscreen&quot;:null,&quot;imageSize&quot;:null,&quot;height&quot;:1125,&quot;width&quot;:1456,&quot;resizeWidth&quot;:null,&quot;bytes&quot;:2910331,&quot;alt&quot;:null,&quot;title&quot;:null,&quot;type&quot;:&quot;image/jpeg&quot;,&quot;href&quot;:null,&quot;belowTheFold&quot;:false,&quot;topImage&quot;:true,&quot;internalRedirect&quot;:&quot;https://www.datatinkerer.io/i/165748507?img=https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fadf294af-e15c-41c8-9dfd-c0541054526b_3300x2550.jpeg&quot;,&quot;isProcessing&quot;:false,&quot;align&quot;:null,&quot;offset&quot;:false}" class="sizing-normal" alt="" srcset="https://substackcdn.com/image/fetch/$s_!-CMo!,w_424,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fadf294af-e15c-41c8-9dfd-c0541054526b_3300x2550.jpeg 424w, https://substackcdn.com/image/fetch/$s_!-CMo!,w_848,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fadf294af-e15c-41c8-9dfd-c0541054526b_3300x2550.jpeg 848w, https://substackcdn.com/image/fetch/$s_!-CMo!,w_1272,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fadf294af-e15c-41c8-9dfd-c0541054526b_3300x2550.jpeg 1272w, https://substackcdn.com/image/fetch/$s_!-CMo!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fadf294af-e15c-41c8-9dfd-c0541054526b_3300x2550.jpeg 1456w" sizes="100vw" fetchpriority="high"></picture><div class="image-link-expand"><div class="pencraft pc-display-flex pc-gap-8 pc-reset"><button class="pencraft pc-reset pencraft icon-container restack-image buttonBase-GK1x3M" tabindex="0" type="button"><svg aria-hidden="true" width="20" height="20" viewBox="0 0 20 20" fill="none" stroke-width="1.5" stroke="var(--color-fg-primary)" stroke-linecap="round" stroke-linejoin="round" xmlns="http://www.w3.org/2000/svg" class="icon-noB79L"><g><path d="M2.53001 7.81595C3.49179 4.73911 6.43281 2.5 9.91173 2.5C13.1684 2.5 15.9537 4.46214 17.0852 7.23684L17.6179 8.67647M17.6179 8.67647L18.5002 4.26471M17.6179 8.67647L13.6473 6.91176M17.4995 12.1841C16.5378 15.2609 13.5967 17.5 10.1178 17.5C6.86118 17.5 4.07589 15.5379 2.94432 12.7632L2.41165 11.3235M2.41165 11.3235L1.5293 15.7353M2.41165 11.3235L6.38224 13.0882"></path></g></svg></button><button class="pencraft pc-reset pencraft icon-container view-image buttonBase-GK1x3M" tabindex="0" type="button"><svg xmlns="http://www.w3.org/2000/svg" width="20" height="20" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" class="lucide lucide-maximize2 lucide-maximize-2 icon-noB79L"><polyline points="15 3 21 3 21 9"></polyline><polyline points="9 21 3 21 3 15"></polyline><line x1="21" x2="14" y1="3" y2="10"></line><line x1="3" x2="10" y1="21" y2="14"></line></svg></button></div></div></div></a></figure></div><p>There are 100+ more cheat sheets covering everything from Python, R, SQL, Spark to Power BI, Tableau, Git and many more. So if you know other people who like staying up to date on all things data, please share Data Tinkerer with them!</p><p class="button-wrapper" data-attrs="{&quot;url&quot;:&quot;https://www.datatinkerer.io/leaderboard?&amp;referrer_token=4tlsmi&amp;utm_source=post&quot;,&quot;text&quot;:&quot;Refer a friend&quot;,&quot;action&quot;:null,&quot;class&quot;:&quot;button-wrapper&quot;}" data-component-name="ButtonCreateButton"><a class="button primary button-wrapper" href="https://www.datatinkerer.io/leaderboard?&amp;referrer_token=4tlsmi&amp;utm_source=post"><span>Refer a friend</span></a></p><p>Now, with that out of the way, let&#8217;s get to Flipkart&#8217;s challenge of calculating delivery date at scale</p>
      <p>
          <a href="https://www.datatinkerer.io/p/how-flipkart-scaled-delivery-date-calculation-10x-while-slashing-latency-by-90-percent">
              Read more
          </a>
      </p>
   ]]></content:encoded></item><item><title><![CDATA[How Notion Brought Order to Its Data Chaos (And Why Their First Catalog Failed)]]></title><description><![CDATA[A behind-the-scenes look at the real challenges, missed steps and what finally made their data catalog work.]]></description><link>https://www.datatinkerer.io/p/how-notion-brought-order-to-its-data-chaos</link><guid isPermaLink="false">https://www.datatinkerer.io/p/how-notion-brought-order-to-its-data-chaos</guid><dc:creator><![CDATA[Data Tinkerer]]></dc:creator><pubDate>Thu, 22 May 2025 09:07:34 GMT</pubDate><enclosure url="https://substackcdn.com/image/fetch/$s_!6QIL!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F072d6737-aa00-471d-9ee2-b95adfcb8012_2520x1323.avif" length="0" type="image/jpeg"/><content:encoded><![CDATA[<p>Fellow Data Tinkerers!</p><p>Quick note before this week&#8217;s deep dive. Thanks for reading and subscribing, I really do mean it. If you&#8217;ve got feedback, just hit reply. I read every response.</p><p>Data Tinkerer has always been about sharing what actually works in data, beyond just tools and tech. The deep dives will keep coming but I want to start spotlighting the stuff we don&#8217;t talk about enough: the day-to-day challenges, business outcomes, the challenges and the learnings.</p><p>I want to feature stories from people in data roles: senior data engineers, lead analysts, heads of data, you name it. If you&#8217;ve got a story, lesson, recent technical win or even a battle scar from the data trenches, let&#8217;s get it in front of almost 1,000 smart and engaged peers.</p><p>You don&#8217;t need to be a &#8220;writer&#8221;, I&#8217;ll help your story shine. Plus, guest contributors get a shoutout in the newsletter and on LinkedIn (If you want).</p><p><strong>Keen to share your data story? Just reply to this email or message me on Substack and we&#8217;ll tee it up.</strong></p><div class="directMessage button" data-attrs="{&quot;userId&quot;:291590442,&quot;userName&quot;:&quot;Data Tinkerer&quot;,&quot;canDm&quot;:null,&quot;dmUpgradeOptions&quot;:null,&quot;isEditorNode&quot;:true}" data-component-name="DirectMessageToDOM"></div><p>Now &#8230;</p>
      <p>
          <a href="https://www.datatinkerer.io/p/how-notion-brought-order-to-its-data-chaos">
              Read more
          </a>
      </p>
   ]]></content:encoded></item><item><title><![CDATA[How Canva Rebuilt Its Data Pipelines for Billions of Events per Month]]></title><description><![CDATA[What it takes to track usage, pay creators fairly and not drown in incident recovery hell.]]></description><link>https://www.datatinkerer.io/p/how-canva-rebuilt-its-data-pipelines-for-billions-of-events-per-month</link><guid isPermaLink="false">https://www.datatinkerer.io/p/how-canva-rebuilt-its-data-pipelines-for-billions-of-events-per-month</guid><dc:creator><![CDATA[Data Tinkerer]]></dc:creator><pubDate>Thu, 01 May 2025 07:39:39 GMT</pubDate><enclosure url="https://substackcdn.com/image/fetch/f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fc4515c9b-2ca5-4dec-a2b4-98c5358d205a_1920x1080.jpeg" length="0" type="image/jpeg"/><content:encoded><![CDATA[<p>Fellow Data Tinkerers!</p><p>Today we will look at how Canva solved the surprisingly messy problem of counting at scale</p><p>But before that, I wanted to share an example of what you could unlock if you share Data Tinkerer with just <strong>2 other people</strong>.</p><div class="captioned-image-container"><figure><a class="image-link image2 is-viewable-img" target="_blank" href="https://substackcdn.com/image/fetch/$s_!46-k!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fd46a3f23-4582-459f-8820-6b06a7d85138_3300x2550.jpeg" data-component-name="Image2ToDOM"><div class="image2-inset"><picture><source type="image/webp" srcset="https://substackcdn.com/image/fetch/$s_!46-k!,w_424,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fd46a3f23-4582-459f-8820-6b06a7d85138_3300x2550.jpeg 424w, https://substackcdn.com/image/fetch/$s_!46-k!,w_848,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fd46a3f23-4582-459f-8820-6b06a7d85138_3300x2550.jpeg 848w, https://substackcdn.com/image/fetch/$s_!46-k!,w_1272,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fd46a3f23-4582-459f-8820-6b06a7d85138_3300x2550.jpeg 1272w, https://substackcdn.com/image/fetch/$s_!46-k!,w_1456,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fd46a3f23-4582-459f-8820-6b06a7d85138_3300x2550.jpeg 1456w" sizes="100vw"><img src="https://substackcdn.com/image/fetch/$s_!46-k!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fd46a3f23-4582-459f-8820-6b06a7d85138_3300x2550.jpeg" width="1456" height="1125" data-attrs="{&quot;src&quot;:&quot;https://substack-post-media.s3.amazonaws.com/public/images/d46a3f23-4582-459f-8820-6b06a7d85138_3300x2550.jpeg&quot;,&quot;srcNoWatermark&quot;:null,&quot;fullscreen&quot;:null,&quot;imageSize&quot;:null,&quot;height&quot;:1125,&quot;width&quot;:1456,&quot;resizeWidth&quot;:null,&quot;bytes&quot;:2470943,&quot;alt&quot;:null,&quot;title&quot;:null,&quot;type&quot;:&quot;image/jpeg&quot;,&quot;href&quot;:null,&quot;belowTheFold&quot;:false,&quot;topImage&quot;:true,&quot;internalRedirect&quot;:&quot;https://www.datatinkerer.io/i/162515057?img=https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fd46a3f23-4582-459f-8820-6b06a7d85138_3300x2550.jpeg&quot;,&quot;isProcessing&quot;:false,&quot;align&quot;:null,&quot;offset&quot;:false}" class="sizing-normal" alt="" srcset="https://substackcdn.com/image/fetch/$s_!46-k!,w_424,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fd46a3f23-4582-459f-8820-6b06a7d85138_3300x2550.jpeg 424w, https://substackcdn.com/image/fetch/$s_!46-k!,w_848,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fd46a3f23-4582-459f-8820-6b06a7d85138_3300x2550.jpeg 848w, https://substackcdn.com/image/fetch/$s_!46-k!,w_1272,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fd46a3f23-4582-459f-8820-6b06a7d85138_3300x2550.jpeg 1272w, https://substackcdn.com/image/fetch/$s_!46-k!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fd46a3f23-4582-459f-8820-6b06a7d85138_3300x2550.jpeg 1456w" sizes="100vw" fetchpriority="high"></picture><div class="image-link-expand"><div class="pencraft pc-display-flex pc-gap-8 pc-reset"><button class="pencraft pc-reset pencraft icon-container restack-image buttonBase-GK1x3M" tabindex="0" type="button"><svg aria-hidden="true" width="20" height="20" viewBox="0 0 20 20" fill="none" stroke-width="1.5" stroke="var(--color-fg-primary)" stroke-linecap="round" stroke-linejoin="round" xmlns="http://www.w3.org/2000/svg" class="icon-noB79L"><g><path d="M2.53001 7.81595C3.49179 4.73911 6.43281 2.5 9.91173 2.5C13.1684 2.5 15.9537 4.46214 17.0852 7.23684L17.6179 8.67647M17.6179 8.67647L18.5002 4.26471M17.6179 8.67647L13.6473 6.91176M17.4995 12.1841C16.5378 15.2609 13.5967 17.5 10.1178 17.5C6.86118 17.5 4.07589 15.5379 2.94432 12.7632L2.41165 11.3235M2.41165 11.3235L1.5293 15.7353M2.41165 11.3235L6.38224 13.0882"></path></g></svg></button><button class="pencraft pc-reset pencraft icon-container view-image buttonBase-GK1x3M" tabindex="0" type="button"><svg xmlns="http://www.w3.org/2000/svg" width="20" height="20" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" class="lucide lucide-maximize2 lucide-maximize-2 icon-noB79L"><polyline points="15 3 21 3 21 9"></polyline><polyline points="9 21 3 21 3 15"></polyline><line x1="21" x2="14" y1="3" y2="10"></line><line x1="3" x2="10" y1="21" y2="14"></line></svg></button></div></div></div></a></figure></div><p>There are 100+ more cheat sheets covering everything from Python, R, SQL, Spark to Power BI, Tableau, Git and many more. So if you know other people who like staying up to date on all things data, please share Data Tinkerer with them!</p><p class="button-wrapper" data-attrs="{&quot;url&quot;:&quot;https://www.datatinkerer.io/leaderboard?&amp;referrer_token=4tlsmi&amp;utm_source=post&quot;,&quot;text&quot;:&quot;Refer a friend&quot;,&quot;action&quot;:null,&quot;class&quot;:&quot;button-wrapper&quot;}" data-component-name="ButtonCreateButton"><a class="button primary button-wrapper" href="https://www.datatinkerer.io/leaderboard?&amp;referrer_token=4tlsmi&amp;utm_source=post"><span>Refer a friend</span></a></p><p>Now, with that out of the way, let&#8217;s get to Canva&#8217;s work to build a scalable data pipeline</p>
      <p>
          <a href="https://www.datatinkerer.io/p/how-canva-rebuilt-its-data-pipelines-for-billions-of-events-per-month">
              Read more
          </a>
      </p>
   ]]></content:encoded></item><item><title><![CDATA[How Airtable Made Archive Validation Work at Petabyte Scale]]></title><description><![CDATA[They handled billions of rows using joins, hashes and a lot of buckets.]]></description><link>https://www.datatinkerer.io/p/how-airtable-made-archive-validation-work-at-petabyte-scale</link><guid isPermaLink="false">https://www.datatinkerer.io/p/how-airtable-made-archive-validation-work-at-petabyte-scale</guid><dc:creator><![CDATA[Data Tinkerer]]></dc:creator><pubDate>Thu, 10 Apr 2025 06:22:40 GMT</pubDate><enclosure url="https://substackcdn.com/image/fetch/$s_!T_KN!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F682b198e-3ef1-40ee-b823-b52f80699458_1024x1024.png" length="0" type="image/jpeg"/><content:encoded><![CDATA[<div class="captioned-image-container"><figure><a class="image-link image2 is-viewable-img" target="_blank" href="https://substackcdn.com/image/fetch/$s_!T_KN!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F682b198e-3ef1-40ee-b823-b52f80699458_1024x1024.png" data-component-name="Image2ToDOM"><div class="image2-inset"><picture><source type="image/webp" srcset="https://substackcdn.com/image/fetch/$s_!T_KN!,w_424,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F682b198e-3ef1-40ee-b823-b52f80699458_1024x1024.png 424w, https://substackcdn.com/image/fetch/$s_!T_KN!,w_848,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F682b198e-3ef1-40ee-b823-b52f80699458_1024x1024.png 848w, https://substackcdn.com/image/fetch/$s_!T_KN!,w_1272,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F682b198e-3ef1-40ee-b823-b52f80699458_1024x1024.png 1272w, https://substackcdn.com/image/fetch/$s_!T_KN!,w_1456,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F682b198e-3ef1-40ee-b823-b52f80699458_1024x1024.png 1456w" sizes="100vw"><img src="https://substackcdn.com/image/fetch/$s_!T_KN!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F682b198e-3ef1-40ee-b823-b52f80699458_1024x1024.png" width="1024" height="1024" data-attrs="{&quot;src&quot;:&quot;https://substack-post-media.s3.amazonaws.com/public/images/682b198e-3ef1-40ee-b823-b52f80699458_1024x1024.png&quot;,&quot;srcNoWatermark&quot;:null,&quot;fullscreen&quot;:null,&quot;imageSize&quot;:null,&quot;height&quot;:1024,&quot;width&quot;:1024,&quot;resizeWidth&quot;:null,&quot;bytes&quot;:1708241,&quot;alt&quot;:null,&quot;title&quot;:null,&quot;type&quot;:&quot;image/png&quot;,&quot;href&quot;:null,&quot;belowTheFold&quot;:false,&quot;topImage&quot;:true,&quot;internalRedirect&quot;:&quot;https://www.datatinkerer.io/i/160983286?img=https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F682b198e-3ef1-40ee-b823-b52f80699458_1024x1024.png&quot;,&quot;isProcessing&quot;:false,&quot;align&quot;:null,&quot;offset&quot;:false}" class="sizing-normal" alt="" srcset="https://substackcdn.com/image/fetch/$s_!T_KN!,w_424,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F682b198e-3ef1-40ee-b823-b52f80699458_1024x1024.png 424w, https://substackcdn.com/image/fetch/$s_!T_KN!,w_848,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F682b198e-3ef1-40ee-b823-b52f80699458_1024x1024.png 848w, https://substackcdn.com/image/fetch/$s_!T_KN!,w_1272,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F682b198e-3ef1-40ee-b823-b52f80699458_1024x1024.png 1272w, https://substackcdn.com/image/fetch/$s_!T_KN!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F682b198e-3ef1-40ee-b823-b52f80699458_1024x1024.png 1456w" sizes="100vw" fetchpriority="high"></picture><div class="image-link-expand"><div class="pencraft pc-display-flex pc-gap-8 pc-reset"><button tabindex="0" type="button" class="pencraft pc-reset pencraft icon-container restack-image buttonBase-GK1x3M"><svg aria-hidden="true" width="20" height="20" viewBox="0 0 20 20" fill="none" stroke-width="1.5" stroke="var(--color-fg-primary)" stroke-linecap="round" stroke-linejoin="round" xmlns="http://www.w3.org/2000/svg" class="icon-noB79L"><g><path d="M2.53001 7.81595C3.49179 4.73911 6.43281 2.5 9.91173 2.5C13.1684 2.5 15.9537 4.46214 17.0852 7.23684L17.6179 8.67647M17.6179 8.67647L18.5002 4.26471M17.6179 8.67647L13.6473 6.91176M17.4995 12.1841C16.5378 15.2609 13.5967 17.5 10.1178 17.5C6.86118 17.5 4.07589 15.5379 2.94432 12.7632L2.41165 11.3235M2.41165 11.3235L1.5293 15.7353M2.41165 11.3235L6.38224 13.0882"></path></g></svg></button><button tabindex="0" type="button" class="pencraft pc-reset pencraft icon-container view-image buttonBase-GK1x3M"><svg xmlns="http://www.w3.org/2000/svg" width="20" height="20" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" class="lucide lucide-maximize2 lucide-maximize-2 icon-noB79L"><polyline points="15 3 21 3 21 9"></polyline><polyline points="9 21 3 21 3 15"></polyline><line x1="21" x2="14" y1="3" y2="10"></line><line x1="3" x2="10" y1="21" y2="14"></line></svg></button></div></div></div></a></figure></div><h3>TL;DR</h3>
      <p>
          <a href="https://www.datatinkerer.io/p/how-airtable-made-archive-validation-work-at-petabyte-scale">
              Read more
          </a>
      </p>
   ]]></content:encoded></item><item><title><![CDATA[How HubSpot Optimized Logging to Save Millions]]></title><description><![CDATA[By refining log storage and retention, HubSpot reduced costs by 55.7% and improved query performance by 50x]]></description><link>https://www.datatinkerer.io/p/how-hubspot-optimized-logging-to-save-millions</link><guid isPermaLink="false">https://www.datatinkerer.io/p/how-hubspot-optimized-logging-to-save-millions</guid><dc:creator><![CDATA[Data Tinkerer]]></dc:creator><pubDate>Thu, 20 Mar 2025 03:55:16 GMT</pubDate><enclosure url="https://substackcdn.com/image/fetch/$s_!Ozzx!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fd038ac92-6001-4b71-88f3-b1cf0fe98b56_1146x703.png" length="0" type="image/jpeg"/><content:encoded><![CDATA[<h3>TL;DR</h3><div><hr></div><h4><strong>Situation</strong></h4><p>HubSpot's backend performance team identified that Amazon S3 storage costs accounted for approximately 45% to 50% of daily expenses, with the 'hubspot-live-logs-prod' bucket alone responsible for 20% of these costs.</p><h4><strong>Task</strong></h4><p>The team aimed to reduce storage costs by addressing the inefficiencies in their logging system, particularly focusing on the large volumes of raw JSON logs that were not being efficiently compacted.</p><h4><strong>Action</strong></h4><ul><li><p><strong>Log Retention Review</strong>: They discovered that raw JSON logs were retained for 730 days, while compressed ORC logs were kept for 460 days. Aligning the retention period to 460 days for both formats reduced unnecessary storage.&#8203;</p></li><li><p><strong>Improved Compression</strong>: By enhancing their Spark compaction process, they increased the conversion rate of raw JSON logs to the more storage-efficient ORC format, achieving a compression ratio where ORC logs were about 5% the size of the original JSON logs.</p></li></ul><h4><strong>Result</strong></h4><p>These measures led to a 55.7% reduction in monthly JSON log storage costs, translating to annual savings in the seven-figure range. Additionally, engineers experienced faster log query times, with some reporting reductions from 30 minutes to just 36 seconds.</p><h4><strong>Use Cases</strong></h4><p>Cost monitoring, Log retention, Log volume reduction</p><h4><strong>Tech Stack/Framework</strong></h4><p>AWS Athena, Amazon S3, Apache Spark, Apache Mesos, Redash</p><div><hr></div><h3>Explained Further</h3><div><hr></div><h4><strong>Saving Millions on Logging</strong></h4>
      <p>
          <a href="https://www.datatinkerer.io/p/how-hubspot-optimized-logging-to-save-millions">
              Read more
          </a>
      </p>
   ]]></content:encoded></item><item><title><![CDATA[Scaling Apache Flink: How Reddit Cut Memory Usage by 60%]]></title><description><![CDATA[Optimizing real-time ad validation with field filtering, tiered storage, and infrastructure enhancements.]]></description><link>https://www.datatinkerer.io/p/scaling-apache-flink-how-reddit-cut-memory-usage-by-60-percent</link><guid isPermaLink="false">https://www.datatinkerer.io/p/scaling-apache-flink-how-reddit-cut-memory-usage-by-60-percent</guid><dc:creator><![CDATA[Data Tinkerer]]></dc:creator><pubDate>Wed, 19 Feb 2025 06:18:34 GMT</pubDate><enclosure url="https://images.unsplash.com/photo-1616509091215-57bbece93654?crop=entropy&amp;cs=tinysrgb&amp;fit=max&amp;fm=jpg&amp;ixid=M3wzMDAzMzh8MHwxfHNlYXJjaHwxfHxyZWRkaXR8ZW58MHx8fHwxNzM5OTQ0NTgwfDA&amp;ixlib=rb-4.0.3&amp;q=80&amp;w=1080" length="0" type="image/jpeg"/><content:encoded><![CDATA[<div class="captioned-image-container"><figure><a class="image-link image2 is-viewable-img" target="_blank" href="https://images.unsplash.com/photo-1616509091215-57bbece93654?crop=entropy&amp;cs=tinysrgb&amp;fit=max&amp;fm=jpg&amp;ixid=M3wzMDAzMzh8MHwxfHNlYXJjaHwxfHxyZWRkaXR8ZW58MHx8fHwxNzM5OTQ0NTgwfDA&amp;ixlib=rb-4.0.3&amp;q=80&amp;w=1080" data-component-name="Image2ToDOM"><div class="image2-inset"><picture><source type="image/webp" srcset="https://images.unsplash.com/photo-1616509091215-57bbece93654?crop=entropy&amp;cs=tinysrgb&amp;fit=max&amp;fm=jpg&amp;ixid=M3wzMDAzMzh8MHwxfHNlYXJjaHwxfHxyZWRkaXR8ZW58MHx8fHwxNzM5OTQ0NTgwfDA&amp;ixlib=rb-4.0.3&amp;q=80&amp;w=1080 424w, https://images.unsplash.com/photo-1616509091215-57bbece93654?crop=entropy&amp;cs=tinysrgb&amp;fit=max&amp;fm=jpg&amp;ixid=M3wzMDAzMzh8MHwxfHNlYXJjaHwxfHxyZWRkaXR8ZW58MHx8fHwxNzM5OTQ0NTgwfDA&amp;ixlib=rb-4.0.3&amp;q=80&amp;w=1080 848w, https://images.unsplash.com/photo-1616509091215-57bbece93654?crop=entropy&amp;cs=tinysrgb&amp;fit=max&amp;fm=jpg&amp;ixid=M3wzMDAzMzh8MHwxfHNlYXJjaHwxfHxyZWRkaXR8ZW58MHx8fHwxNzM5OTQ0NTgwfDA&amp;ixlib=rb-4.0.3&amp;q=80&amp;w=1080 1272w, https://images.unsplash.com/photo-1616509091215-57bbece93654?crop=entropy&amp;cs=tinysrgb&amp;fit=max&amp;fm=jpg&amp;ixid=M3wzMDAzMzh8MHwxfHNlYXJjaHwxfHxyZWRkaXR8ZW58MHx8fHwxNzM5OTQ0NTgwfDA&amp;ixlib=rb-4.0.3&amp;q=80&amp;w=1080 1456w" sizes="100vw"><img src="https://images.unsplash.com/photo-1616509091215-57bbece93654?crop=entropy&amp;cs=tinysrgb&amp;fit=max&amp;fm=jpg&amp;ixid=M3wzMDAzMzh8MHwxfHNlYXJjaHwxfHxyZWRkaXR8ZW58MHx8fHwxNzM5OTQ0NTgwfDA&amp;ixlib=rb-4.0.3&amp;q=80&amp;w=1080" width="5184" height="3888" data-attrs="{&quot;src&quot;:&quot;https://images.unsplash.com/photo-1616509091215-57bbece93654?crop=entropy&amp;cs=tinysrgb&amp;fit=max&amp;fm=jpg&amp;ixid=M3wzMDAzMzh8MHwxfHNlYXJjaHwxfHxyZWRkaXR8ZW58MHx8fHwxNzM5OTQ0NTgwfDA&amp;ixlib=rb-4.0.3&amp;q=80&amp;w=1080&quot;,&quot;srcNoWatermark&quot;:null,&quot;fullscreen&quot;:null,&quot;imageSize&quot;:null,&quot;height&quot;:3888,&quot;width&quot;:5184,&quot;resizeWidth&quot;:null,&quot;bytes&quot;:null,&quot;alt&quot;:&quot;red and white 8 logo&quot;,&quot;title&quot;:null,&quot;type&quot;:&quot;image/jpg&quot;,&quot;href&quot;:null,&quot;belowTheFold&quot;:false,&quot;topImage&quot;:true,&quot;internalRedirect&quot;:null,&quot;isProcessing&quot;:false,&quot;align&quot;:null,&quot;offset&quot;:false}" class="sizing-normal" alt="red and white 8 logo" title="red and white 8 logo" srcset="https://images.unsplash.com/photo-1616509091215-57bbece93654?crop=entropy&amp;cs=tinysrgb&amp;fit=max&amp;fm=jpg&amp;ixid=M3wzMDAzMzh8MHwxfHNlYXJjaHwxfHxyZWRkaXR8ZW58MHx8fHwxNzM5OTQ0NTgwfDA&amp;ixlib=rb-4.0.3&amp;q=80&amp;w=1080 424w, https://images.unsplash.com/photo-1616509091215-57bbece93654?crop=entropy&amp;cs=tinysrgb&amp;fit=max&amp;fm=jpg&amp;ixid=M3wzMDAzMzh8MHwxfHNlYXJjaHwxfHxyZWRkaXR8ZW58MHx8fHwxNzM5OTQ0NTgwfDA&amp;ixlib=rb-4.0.3&amp;q=80&amp;w=1080 848w, https://images.unsplash.com/photo-1616509091215-57bbece93654?crop=entropy&amp;cs=tinysrgb&amp;fit=max&amp;fm=jpg&amp;ixid=M3wzMDAzMzh8MHwxfHNlYXJjaHwxfHxyZWRkaXR8ZW58MHx8fHwxNzM5OTQ0NTgwfDA&amp;ixlib=rb-4.0.3&amp;q=80&amp;w=1080 1272w, https://images.unsplash.com/photo-1616509091215-57bbece93654?crop=entropy&amp;cs=tinysrgb&amp;fit=max&amp;fm=jpg&amp;ixid=M3wzMDAzMzh8MHwxfHNlYXJjaHwxfHxyZWRkaXR8ZW58MHx8fHwxNzM5OTQ0NTgwfDA&amp;ixlib=rb-4.0.3&amp;q=80&amp;w=1080 1456w" sizes="100vw" fetchpriority="high"></picture><div class="image-link-expand"><div class="pencraft pc-display-flex pc-gap-8 pc-reset"><button class="pencraft pc-reset pencraft icon-container restack-image buttonBase-GK1x3M" tabindex="0" type="button"><svg aria-hidden="true" width="20" height="20" viewBox="0 0 20 20" fill="none" stroke-width="1.5" stroke="var(--color-fg-primary)" stroke-linecap="round" stroke-linejoin="round" xmlns="http://www.w3.org/2000/svg" class="icon-noB79L"><g><path d="M2.53001 7.81595C3.49179 4.73911 6.43281 2.5 9.91173 2.5C13.1684 2.5 15.9537 4.46214 17.0852 7.23684L17.6179 8.67647M17.6179 8.67647L18.5002 4.26471M17.6179 8.67647L13.6473 6.91176M17.4995 12.1841C16.5378 15.2609 13.5967 17.5 10.1178 17.5C6.86118 17.5 4.07589 15.5379 2.94432 12.7632L2.41165 11.3235M2.41165 11.3235L1.5293 15.7353M2.41165 11.3235L6.38224 13.0882"></path></g></svg></button><button class="pencraft pc-reset pencraft icon-container view-image buttonBase-GK1x3M" tabindex="0" type="button"><svg xmlns="http://www.w3.org/2000/svg" width="20" height="20" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" class="lucide lucide-maximize2 lucide-maximize-2 icon-noB79L"><polyline points="15 3 21 3 21 9"></polyline><polyline points="9 21 3 21 3 15"></polyline><line x1="21" x2="14" y1="3" y2="10"></line><line x1="3" x2="10" y1="21" y2="14"></line></svg></button></div></div></div></a><figcaption class="image-caption">Photo by <a href="true">Brett Jordan</a> on <a href="https://unsplash.com">Unsplash</a></figcaption></figure></div><div><hr></div><h4><strong>Situation</strong></h4><p>Reddit's advertising platform processes thousands of ad engagement events per second, necessitating real-time validation and enrichment to ensure accurate reporting and prevent budget overdelivery.</p><h4><strong>Task</strong></h4><p>Develop a scalable, real-time ad event validation system capable of efficiently handling high event volumes while maintaining performance and reliability.</p><h4><strong>Action</strong></h4><p>The engineering team developed the Ad Events Validator (AEV) utilizing Apache Flink to correlate ad server events with user engagement events. To overcome issues related to large state sizes and resource demands, they implemented:</p><ul><li><p><strong>Field Filtering:</strong> Conducted a thorough analysis of downstream data consumption, establishing an allowlist that significantly reduced the event payload size by 90%, leading to CPU and memory usage reductions of 25% and 60%, respectively.</p></li><li><p><strong>Tiered State Storage:</strong> Integrated Apache Cassandra for external state storage, effectively reducing in-memory state size and enhancing the efficiency of checkpointing and system recovery processes.</p></li></ul><h4><strong>Result</strong></h4><p>These strategic enhancements resulted in a more scalable and cost-efficient AEV system, improving overall performance and operational effectiveness.</p><h4><strong>Use Cases</strong></h4><p>Real-Time Event Validation, Data Enrichment, Resource Optimization</p><h4><strong>Tech Stack/Framework</strong></h4><p>Apache Flink, Apache Kafka, Apache Cassandra</p><div><hr></div><h3>Explained Further</h3><div><hr></div><h4><strong>Background</strong></h4><p>Reddit processes thousands of ad engagement events per second. These events require validation and enrichment before being sent to downstream systems. Key components of this validation process include applying a standardized look-back window and filtering out suspected invalid traffic.</p><p>In addition to a batch validation pipeline, a near real-time pipeline improves budget spend accuracy and provides advertisers with real-time insights into campaign performance. This real-time component, known as the <strong>Ad Events Validator (AEV)</strong>, is built using Apache Flink. AEV matches ad server events with engagement events and writes the validated results to a separate Kafka topic for downstream consumption. </p><div class="captioned-image-container"><figure><a class="image-link image2 is-viewable-img" target="_blank" href="https://substackcdn.com/image/fetch/$s_!6utL!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F441259f5-fa8b-48c2-a11e-6a8df3128807_1080x441.webp" data-component-name="Image2ToDOM"><div class="image2-inset"><picture><source type="image/webp" srcset="https://substackcdn.com/image/fetch/$s_!6utL!,w_424,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F441259f5-fa8b-48c2-a11e-6a8df3128807_1080x441.webp 424w, https://substackcdn.com/image/fetch/$s_!6utL!,w_848,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F441259f5-fa8b-48c2-a11e-6a8df3128807_1080x441.webp 848w, https://substackcdn.com/image/fetch/$s_!6utL!,w_1272,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F441259f5-fa8b-48c2-a11e-6a8df3128807_1080x441.webp 1272w, https://substackcdn.com/image/fetch/$s_!6utL!,w_1456,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F441259f5-fa8b-48c2-a11e-6a8df3128807_1080x441.webp 1456w" sizes="auto, 100vw"><img src="https://substackcdn.com/image/fetch/$s_!6utL!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F441259f5-fa8b-48c2-a11e-6a8df3128807_1080x441.webp" width="1080" height="441" data-attrs="{&quot;src&quot;:&quot;https://substack-post-media.s3.amazonaws.com/public/images/441259f5-fa8b-48c2-a11e-6a8df3128807_1080x441.webp&quot;,&quot;srcNoWatermark&quot;:null,&quot;fullscreen&quot;:null,&quot;imageSize&quot;:null,&quot;height&quot;:441,&quot;width&quot;:1080,&quot;resizeWidth&quot;:null,&quot;bytes&quot;:16750,&quot;alt&quot;:null,&quot;title&quot;:null,&quot;type&quot;:&quot;image/webp&quot;,&quot;href&quot;:null,&quot;belowTheFold&quot;:true,&quot;topImage&quot;:false,&quot;internalRedirect&quot;:null,&quot;isProcessing&quot;:false,&quot;align&quot;:null,&quot;offset&quot;:false}" class="sizing-normal" alt="" srcset="https://substackcdn.com/image/fetch/$s_!6utL!,w_424,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F441259f5-fa8b-48c2-a11e-6a8df3128807_1080x441.webp 424w, https://substackcdn.com/image/fetch/$s_!6utL!,w_848,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F441259f5-fa8b-48c2-a11e-6a8df3128807_1080x441.webp 848w, https://substackcdn.com/image/fetch/$s_!6utL!,w_1272,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F441259f5-fa8b-48c2-a11e-6a8df3128807_1080x441.webp 1272w, https://substackcdn.com/image/fetch/$s_!6utL!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F441259f5-fa8b-48c2-a11e-6a8df3128807_1080x441.webp 1456w" sizes="auto, 100vw" loading="lazy"></picture><div class="image-link-expand"><div class="pencraft pc-display-flex pc-gap-8 pc-reset"><button class="pencraft pc-reset pencraft icon-container restack-image buttonBase-GK1x3M" tabindex="0" type="button"><svg aria-hidden="true" width="20" height="20" viewBox="0 0 20 20" fill="none" stroke-width="1.5" stroke="var(--color-fg-primary)" stroke-linecap="round" stroke-linejoin="round" xmlns="http://www.w3.org/2000/svg" class="icon-noB79L"><g><path d="M2.53001 7.81595C3.49179 4.73911 6.43281 2.5 9.91173 2.5C13.1684 2.5 15.9537 4.46214 17.0852 7.23684L17.6179 8.67647M17.6179 8.67647L18.5002 4.26471M17.6179 8.67647L13.6473 6.91176M17.4995 12.1841C16.5378 15.2609 13.5967 17.5 10.1178 17.5C6.86118 17.5 4.07589 15.5379 2.94432 12.7632L2.41165 11.3235M2.41165 11.3235L1.5293 15.7353M2.41165 11.3235L6.38224 13.0882"></path></g></svg></button><button class="pencraft pc-reset pencraft icon-container view-image buttonBase-GK1x3M" tabindex="0" type="button"><svg xmlns="http://www.w3.org/2000/svg" width="20" height="20" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" class="lucide lucide-maximize2 lucide-maximize-2 icon-noB79L"><polyline points="15 3 21 3 21 9"></polyline><polyline points="9 21 3 21 3 15"></polyline><line x1="21" x2="14" y1="3" y2="10"></line><line x1="3" x2="10" y1="21" y2="14"></line></svg></button></div></div></div></a><figcaption class="image-caption">Overview of the real-time ad engagement event validation system (Source: Reddit)</figcaption></figure></div><p>Building and maintaining AEV though, presented several challenges to the Reddit team</p><div><hr></div><h4><strong>1st Challenge: Addressing High State Size Issues</strong></h4>
      <p>
          <a href="https://www.datatinkerer.io/p/scaling-apache-flink-how-reddit-cut-memory-usage-by-60-percent">
              Read more
          </a>
      </p>
   ]]></content:encoded></item><item><title><![CDATA[ML Training Too Slow? Yelp’s 1,400x Speed Boost Fixes That]]></title><description><![CDATA[Discover the data pipeline and GPU optimisations that made it happen]]></description><link>https://www.datatinkerer.io/p/ml-training-too-slow-yelps-1400x-speed-boost-fixes-that</link><guid isPermaLink="false">https://www.datatinkerer.io/p/ml-training-too-slow-yelps-1400x-speed-boost-fixes-that</guid><dc:creator><![CDATA[Data Tinkerer]]></dc:creator><pubDate>Wed, 12 Feb 2025 05:46:32 GMT</pubDate><enclosure url="https://images.unsplash.com/photo-1730818876892-7e7c6ddf3dc6?crop=entropy&amp;cs=tinysrgb&amp;fit=max&amp;fm=jpg&amp;ixid=M3wzMDAzMzh8MHwxfHNlYXJjaHwzfHx5ZWxwfGVufDB8fHx8MTczOTMzODU5MHww&amp;ixlib=rb-4.0.3&amp;q=80&amp;w=1080" length="0" type="image/jpeg"/><content:encoded><![CDATA[<div class="captioned-image-container"><figure><a class="image-link image2 is-viewable-img" target="_blank" href="https://images.unsplash.com/photo-1730818876892-7e7c6ddf3dc6?crop=entropy&amp;cs=tinysrgb&amp;fit=max&amp;fm=jpg&amp;ixid=M3wzMDAzMzh8MHwxfHNlYXJjaHwzfHx5ZWxwfGVufDB8fHx8MTczOTMzODU5MHww&amp;ixlib=rb-4.0.3&amp;q=80&amp;w=1080" data-component-name="Image2ToDOM"><div class="image2-inset"><picture><source type="image/webp" srcset="https://images.unsplash.com/photo-1730818876892-7e7c6ddf3dc6?crop=entropy&amp;cs=tinysrgb&amp;fit=max&amp;fm=jpg&amp;ixid=M3wzMDAzMzh8MHwxfHNlYXJjaHwzfHx5ZWxwfGVufDB8fHx8MTczOTMzODU5MHww&amp;ixlib=rb-4.0.3&amp;q=80&amp;w=1080 424w, https://images.unsplash.com/photo-1730818876892-7e7c6ddf3dc6?crop=entropy&amp;cs=tinysrgb&amp;fit=max&amp;fm=jpg&amp;ixid=M3wzMDAzMzh8MHwxfHNlYXJjaHwzfHx5ZWxwfGVufDB8fHx8MTczOTMzODU5MHww&amp;ixlib=rb-4.0.3&amp;q=80&amp;w=1080 848w, https://images.unsplash.com/photo-1730818876892-7e7c6ddf3dc6?crop=entropy&amp;cs=tinysrgb&amp;fit=max&amp;fm=jpg&amp;ixid=M3wzMDAzMzh8MHwxfHNlYXJjaHwzfHx5ZWxwfGVufDB8fHx8MTczOTMzODU5MHww&amp;ixlib=rb-4.0.3&amp;q=80&amp;w=1080 1272w, https://images.unsplash.com/photo-1730818876892-7e7c6ddf3dc6?crop=entropy&amp;cs=tinysrgb&amp;fit=max&amp;fm=jpg&amp;ixid=M3wzMDAzMzh8MHwxfHNlYXJjaHwzfHx5ZWxwfGVufDB8fHx8MTczOTMzODU5MHww&amp;ixlib=rb-4.0.3&amp;q=80&amp;w=1080 1456w" sizes="100vw"><img src="https://images.unsplash.com/photo-1730818876892-7e7c6ddf3dc6?crop=entropy&amp;cs=tinysrgb&amp;fit=max&amp;fm=jpg&amp;ixid=M3wzMDAzMzh8MHwxfHNlYXJjaHwzfHx5ZWxwfGVufDB8fHx8MTczOTMzODU5MHww&amp;ixlib=rb-4.0.3&amp;q=80&amp;w=1080" width="6240" height="4160" data-attrs="{&quot;src&quot;:&quot;https://images.unsplash.com/photo-1730818876892-7e7c6ddf3dc6?crop=entropy&amp;cs=tinysrgb&amp;fit=max&amp;fm=jpg&amp;ixid=M3wzMDAzMzh8MHwxfHNlYXJjaHwzfHx5ZWxwfGVufDB8fHx8MTczOTMzODU5MHww&amp;ixlib=rb-4.0.3&amp;q=80&amp;w=1080&quot;,&quot;srcNoWatermark&quot;:null,&quot;fullscreen&quot;:null,&quot;imageSize&quot;:null,&quot;height&quot;:4160,&quot;width&quot;:6240,&quot;resizeWidth&quot;:null,&quot;bytes&quot;:null,&quot;alt&quot;:&quot;A cell phone sitting on top of a wooden table&quot;,&quot;title&quot;:null,&quot;type&quot;:&quot;image/jpg&quot;,&quot;href&quot;:null,&quot;belowTheFold&quot;:false,&quot;topImage&quot;:true,&quot;internalRedirect&quot;:null,&quot;isProcessing&quot;:false,&quot;align&quot;:null,&quot;offset&quot;:false}" class="sizing-normal" alt="A cell phone sitting on top of a wooden table" title="A cell phone sitting on top of a wooden table" srcset="https://images.unsplash.com/photo-1730818876892-7e7c6ddf3dc6?crop=entropy&amp;cs=tinysrgb&amp;fit=max&amp;fm=jpg&amp;ixid=M3wzMDAzMzh8MHwxfHNlYXJjaHwzfHx5ZWxwfGVufDB8fHx8MTczOTMzODU5MHww&amp;ixlib=rb-4.0.3&amp;q=80&amp;w=1080 424w, https://images.unsplash.com/photo-1730818876892-7e7c6ddf3dc6?crop=entropy&amp;cs=tinysrgb&amp;fit=max&amp;fm=jpg&amp;ixid=M3wzMDAzMzh8MHwxfHNlYXJjaHwzfHx5ZWxwfGVufDB8fHx8MTczOTMzODU5MHww&amp;ixlib=rb-4.0.3&amp;q=80&amp;w=1080 848w, https://images.unsplash.com/photo-1730818876892-7e7c6ddf3dc6?crop=entropy&amp;cs=tinysrgb&amp;fit=max&amp;fm=jpg&amp;ixid=M3wzMDAzMzh8MHwxfHNlYXJjaHwzfHx5ZWxwfGVufDB8fHx8MTczOTMzODU5MHww&amp;ixlib=rb-4.0.3&amp;q=80&amp;w=1080 1272w, https://images.unsplash.com/photo-1730818876892-7e7c6ddf3dc6?crop=entropy&amp;cs=tinysrgb&amp;fit=max&amp;fm=jpg&amp;ixid=M3wzMDAzMzh8MHwxfHNlYXJjaHwzfHx5ZWxwfGVufDB8fHx8MTczOTMzODU5MHww&amp;ixlib=rb-4.0.3&amp;q=80&amp;w=1080 1456w" sizes="100vw" fetchpriority="high"></picture><div class="image-link-expand"><div class="pencraft pc-display-flex pc-gap-8 pc-reset"><button class="pencraft pc-reset pencraft icon-container restack-image buttonBase-GK1x3M" tabindex="0" type="button"><svg aria-hidden="true" width="20" height="20" viewBox="0 0 20 20" fill="none" stroke-width="1.5" stroke="var(--color-fg-primary)" stroke-linecap="round" stroke-linejoin="round" xmlns="http://www.w3.org/2000/svg" class="icon-noB79L"><g><path d="M2.53001 7.81595C3.49179 4.73911 6.43281 2.5 9.91173 2.5C13.1684 2.5 15.9537 4.46214 17.0852 7.23684L17.6179 8.67647M17.6179 8.67647L18.5002 4.26471M17.6179 8.67647L13.6473 6.91176M17.4995 12.1841C16.5378 15.2609 13.5967 17.5 10.1178 17.5C6.86118 17.5 4.07589 15.5379 2.94432 12.7632L2.41165 11.3235M2.41165 11.3235L1.5293 15.7353M2.41165 11.3235L6.38224 13.0882"></path></g></svg></button><button class="pencraft pc-reset pencraft icon-container view-image buttonBase-GK1x3M" tabindex="0" type="button"><svg xmlns="http://www.w3.org/2000/svg" width="20" height="20" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" class="lucide lucide-maximize2 lucide-maximize-2 icon-noB79L"><polyline points="15 3 21 3 21 9"></polyline><polyline points="9 21 3 21 3 15"></polyline><line x1="21" x2="14" y1="3" y2="10"></line><line x1="3" x2="10" y1="21" y2="14"></line></svg></button></div></div></div></a><figcaption class="image-caption">Photo by <a href="true">appshunter.io</a> on <a href="https://unsplash.com">Unsplash</a></figcaption></figure></div><h3>TL;DR</h3><div><hr></div><h4><strong>Situation</strong></h4><p>Yelp's ad revenue relies on predicting which ads users are likely to click on, using a model called "Wide and Deep Neural Network." Initially, training this model on 450 million data samples took 75 hours per cycle, which was too slow. Yelp wanted to handle 2 billion samples and reduce training time to under an hour per cycle.</p><h4><strong>Task</strong></h4><p>The goal was to speed up the training process by improving how data is stored and read, and by using multiple GPUs to handle more data at once.</p><h4><strong>Action</strong></h4><ul><li><p><strong>Data Storage</strong>: Yelp stored the training data in Parquet format on Amazon's S3 storage, which works well with their data processing system, Spark. They found that a tool called Petastorm was too slow for their needs, so they developed their own system called ArrowStreamServer. This new system reads and sends data more efficiently, reducing the time to process 9 million samples from over 13 minutes to about 19 seconds.</p></li><li><p><strong>Distributed Training</strong>: Yelp initially used a method called MirroredStrategy to train the model on multiple GPUs but found it didn't work well as they added more GPUs. They switched to a tool called Horovod, which allowed them to efficiently use up to 8 GPUs at once, significantly speeding up the training process.</p></li></ul><h4><strong>Result</strong></h4><p>By implementing these changes, Yelp achieved a total speed increase of about 1,400 times in their model training. This means they can now train their ad prediction models much faster, allowing them to handle more data and improve their ad services.</p><h4><strong>Use Cases</strong></h4><p>Large-Scale ML Training, ML Training Optimisation, Enhancing Data Pipeline Efficiency</p><h4><strong>Tech Stack/Framework</strong></h4><p>TensorFlow, Horovod, Keras, PyArrow, Amazon S3, Apache Spark</p><div><hr></div><h3>Explained Further</h3><div><hr></div><h4><strong>The Challenge</strong></h4>
      <p>
          <a href="https://www.datatinkerer.io/p/ml-training-too-slow-yelps-1400x-speed-boost-fixes-that">
              Read more
          </a>
      </p>
   ]]></content:encoded></item></channel></rss>