It’s time for another round-up on all things data and AI!

But before that, I wanted to share with you what you could unlock if you share Data Tinkerer with just 1 more person.

There are 100+ resources to learn all things data (science, engineering, analysis). It includes videos, courses, projects and can be filtered by tech stack (Python, SQL, Spark and etc), skill level (Beginner, Intermediate and so on) provider name or free/paid. So if you know other people who like staying up to date on all things data, please share Data Tinkerer with them!

Refer a friend

Without further ado, let’s get to the round up for July!

Data science & AI

• Kimi K3 Architecture Notes (4 minute read)
  dissects Kimi K3’s sparse architecture, attention residuals, NoPE design and inference-efficiency trade-offs

  

• The Cheapest Way to Make Open Weight AI Models Better (27 minute read)
  explains why targeted post-training can improve open-weight models more economically than scaling pre-training alone.

• A Taxonomy of Self-evolving Agents (14 minute read)
  Shilong Liu proposes a practical taxonomy for agents that adapt their memory, tools, prompts or policies through experience.

  

• The LLM Critics Are Right. I Use LLMs Anyway. (19 minute read)
  accepts the strongest criticisms of language models while explaining the disciplined uses that still make them valuable.

• Closing the Verification Loop (12 minute read)
  Kieran Klaassen argues that agents become dependable only when verification is built into the loop rather than left to a final human check.

  

• How an AI Token Travels Through a Data Center (26 minute read)
  traces one token through gateways, schedulers, KV caches, accelerators and networks to explain the real mechanics and economics of inference.

• How to Implement a Unified Memory From Scratch (16 minute read)
  builds a unified memory layer from scratch and explains how agents can retain useful state without turning context into clutter.

  

Data engineering

• I Spent 10 Hours Learning Multithreading and Multiprocessing (14 minute read)
  compares multithreading and multiprocessing through hands-on examples so Python practitioners can choose the right concurrency model.

  

• The Data Anarchy Tax: Why your team is firefighting 45% of the time (9 minute read)
  Good article by on why fragmented ownership and unclear operating rules create a measurable data-anarchy tax that keeps teams in reactive work.

• The Most Expensive Mistake in System Design (12 minute read)
  explains why designing for imagined scale too early can make data systems harder and more expensive to operate than the problem requires.

  

• The Startup’s Postgres Survival Guide (11 minute read)
  Alexander Belanger lays out pragmatic Postgres choices for startups that need reliability without prematurely adopting a more complex data platform.

• Your Data Warehouse Isn’t Integrated Just Because the Tables Are in One Place (12 minute read)
  argues that co-locating tables is not integration when semantics, contracts and ownership remain fragmented.

  

• So, is data modeling dead? (16 minute read)
  Daniel Beach examines what AI-assisted development changes about data modelling and why deliberate structures still matter.

• Backfilling: The Most Underrated Feature of a Data Pipeline (10 minute read)
  Lucie Thimus explains why safe, repeatable backfills are a core capability for correcting history and evolving production pipelines.

• FastAPI for Model Serving: The Standard You Should Actually Use (12 minute read)
  shows how to structure a FastAPI service for model inference with production concerns such as validation and deployment in mind.

Data analysis and visualisation

• The Model Is Smart. Your Company Is the Problem (10 minute read)
  argues that failed AI analysis is often rooted in unclear metrics and organisational fragmentation rather than model capability.

• Going Bayesian Automates Your Manual Data Analysis (9 minute read)
  Eric J. Ma shows how robust likelihoods and hierarchical Bayesian models can replace manual outlier handling with a reproducible analytical process.

Other interesting reads

• How Tech Workers Are Feeling in 2026: A Workforce Splitting in Two (22 minute read)
  Interesting analysis by and using a large sentiment survey to show how AI is dividing tech workers between amplification and destabilisation while burnout rises.

  

• You Don’t Have to Be Smart If You Can Think Clearly (4 minute read)
  argues that methodical, clear thinking under uncertainty is more dependable than flashes of intuition when engineers face genuinely hard problems.

Quick favor - need your take

Was there any standout article or topic from July I missed? Feel free to drop a comment or hit reply, even a quick line helps.

If you are already subscribed and enjoyed the article, please give it a like and/or share it others, really appreciate it 🙏

Share

Keep learning

It’s time for another round-up on all things data and AI!

But before that, I wanted to share with you what you could unlock if you share Data Tinkerer with just 1 more person.

There are 100+ resources to learn all things data (science, engineering, analysis). It includes videos, courses, projects and can be filtered by tech stack (Python, SQL, Spark and etc), skill level (Beginner, Intermediate and so on) provider name or free/paid. So if you know other people who like staying up to date on all things data, please share Data Tinkerer with them!

Refer a friend

Without further ado, let’s get to the round up for June!

Data science & AI

• Stop Monitoring AI Systems Like Web Services (11 minute read)
  explains why production AI needs monitoring built around quality, behaviour, cost, feedback and changing data rather than ordinary web-service health metrics.

  

• When Is It Actually a RAG Problem? (10 minute read)
  provides a decision framework for separating genuine retrieval problems from issues better solved with tools, semantic layers or improved context.

• Using Local Coding Agents (34 minute read)
  tests open-weight models in local coding-agent setups and examines how close they come to replacing hosted coding subscriptions.

• Stateful Swarms make AI 100x More Intelligent per Dollar (24 minute read)
  examines how persistent memory changes multi-agent performance, coordination and cost compared with stateless agent architectures.

  

• When Trees Still Beat Tabular Foundation Models (4 minute read)
  examines the dataset characteristics that still favor gradient-boosted trees over increasingly capable tabular foundation models.

• Running Local Models Is Good Now (8 minute read)
  draws on hands-on experiments to explain why local models have become practical for coding, research and personalised development assistance.

• Scaling Laws, Carefully (25 minute read)
  Lilian Weng provides a careful guide to neural scaling laws, covering their mathematical foundations, practical value and important limitations.

Data engineering

• Vibe Coding Is Dangerous, Agentic Engineering Isn’t (15 minute read)
  argues that coding agents become useful when humans remain responsible for specifications, architecture, tests and review.

• How to Build a Simple, Bulletproof Data Pipeline (9 minute read)
  demonstrates how a deliberately simple batch pipeline can deliver robustness and maintainability without unnecessary real-time infrastructure.

  

• Five dbt Mistakes I See in Every Startup (11 minute read)
  examines five recurring dbt mistakes involving CI, contracts, incremental models, sources and environment separation.

• Broker-Visible vs Client-Local Parallelism (6 minute read)
  compares two approaches to messaging parallelism and explains how each affects coordination, throughput and failure handling.

  

• Basic Spark Concepts (5 minute read)
  explains the foundational Spark concepts practitioners need before tuning jobs or adopting more complicated distributed-processing patterns.

• What Is Apache Arrow Flight? (13 minute read)
  Daniel Beach builds an Arrow Flight server and client to explain how Arrow and gRPC move columnar data without traditional serialisation overhead.

• In 2026, the Data Fundamentals Matter More Than Ever (12 minute read)
  argues that modelling, SQL and dependable data foundations remain the real constraints on AI adoption despite the focus on agents and new tools.

• How AI Changes Four Core Data Roles (7 minute read)
  examines how AI is changing the responsibilities and required skills of analysts, analytics engineers, data engineers and data scientists.

Data analysis and visualisation

• On Data Quality (12 minute read)
  Great article by arguing that data quality is conditional on use and should be evaluated according to the decisions and outcomes the data must support.

  

• Why General AI Models Still Cannot Be Trusted for Data Analysis (9 minute read)
  Interesting review by examining where general AI models continue to fail at analytical reasoning, validation and reliable interpretation of data.

• The Tableau Exodus Has Begun (7 minute read)
  Interesting observation by arguing that organisations leaving Tableau should use the migration to rethink metrics, workflows and the value of BI.

Other interesting reads

• Consulting in an AI-World is Fine. Big Consulting Isn’t (8 minute read)
  Interesting take by arguing that AI threatens the economics of large consulting firms more than the value of independent specialist expertise.

• What I’d Do As a Junior Candidate in Mid-2026 (6 minute read)
  practical advice by for junior candidates trying to develop credible skills and stand out in a difficult data job market.

Quick favor - need your take

Was there any standout article or topic from June I missed? Feel free to drop a comment or hit reply, even a quick line helps.

If you are already subscribed and enjoyed the article, please give it a like and/or share it others, really appreciate it 🙏

Share

Keep learning

It’s time for another round-up on all things data and AI!

But before that, I wanted to share with you what you could unlock if you share Data Tinkerer with just 1 more person.

There are 100+ resources to learn all things data (science, engineering, analysis). It includes videos, courses, projects and can be filtered by tech stack (Python, SQL, Spark and etc), skill level (Beginner, Intermediate and so on) provider name or free/paid. So if you know other people who like staying up to date on all things data, please share Data Tinkerer with them!

Refer a friend

Without further ado, let’s get to the round up for May!

Data science & AI

• Agentic AI Flywheels (15 minute read)
  explains how production feedback, observability and evals create a flywheel that helps AI agents improve with every release.

  

• Choosing the Right Graph (36 minute read)
  Detailed breakdown by comparing labeled property graphs and RDF knowledge graphs, offering a practical framework for choosing between them.

• How AI agent memory works (14 minute read)
  walks through how AI agents use context, embeddings, retrieval and memory stores to remember useful information across conversations.

  

• Building Your First Claude Skill (19 minute read)
  shows how to build a Claude Skill that checks whether an analysis is correct, answers the right question and can be trusted before it reaches stakeholders.

• What political censorship looks like inside an LLM’s weights (45+ minute read)
  This article finds that Qwen 3.5’s political censorship is a small, identifiable circuit layered over facts the model already knows.

• Agent Evaluation: A Detailed Guide (60+ minute read)
  explains how to evaluate AI agents using realistic tasks, outcome-based metrics and detailed analysis of their tool use and decision paths.

  

• Is logistic regression regression? (6 minute read)
  Richard Vale explains why logistic regression is genuinely regression because it predicts a probability, only becoming classification once a cutoff is applied.

• What Matters in Production RAG (13 minute read)
  explains how production RAG depends on thoughtful chunking, reliable index updates and retrieval tracing that shows why the system returned a particular answer.

• How Lyft Uses AI to Scale Translation Across 150+ Products (18 minute read)
  This article walks through how Lyft used an LLM drafting and evaluation loop to cut translation times from days to minutes across 150+ services.

Data engineering

• Quick insights on materialized views (11 minute read)
  explains how materialized views speed up queries, how incremental refresh keeps them current and when they make sense for real-time processing.

  

• Five Worlds of Data Engineering (12 minute read)
  Chris hillman explains why data engineering advice often clashes because teams operate across five very different worlds, each with its own tools, constraints and priorities.

• Why I Set Up Databricks and Then Walked Away From It (17 minute read)
  walks through why he dropped Databricks for a simpler open-source stack that better matched his company’s data scale, budget and team maturity.

• Does ELT vs. ETL Even Still Matter? (9 minute read)
  argues that ETL versus ELT matters far less than building reliable, affordable pipelines that help the business make better decisions.

• Complete End-To-End Build of ETL Pipeline in AWS (12 minute read)
  walks through building ETL with DuckDB and ECS Fargate on AWS for under 50 cents.

  

• The Cognitive Overload of AI Development (7 minute read)
  Interesting article by on how AI coding has turned senior engineers into exhausted code reviewers, forever one giant AI-generated pull request away from chicken farming.

• Event-Driven vs. Polling Architectures for Agent Triggers (17 minute read)
  explains why production agent triggers need a mix of events, reconciliation polling, idempotency and durable runtimes rather than relying on webhooks alone.

• Progressive Disclosure: The Core Pattern for Analytics Agents (10 minute read)
  explains why analytics agents should load only the schema and business context they need, when they need it, to reduce confusion and hallucinated queries.

• Strong views on PostgreSQL VIEWs (24 minute read)
  Radim Marek explains why PostgreSQL views remain useful abstractions, but hidden dependencies, rigid schemas and complex migrations make them easy to misuse.

• How Grab Reclaimed Hundreds of Data Engineering Hours With Multi-Agent AI (17 minute read)
  This article explains how Grab used specialist AI agents to cut repetitive data support from hours to minutes and return hundreds of engineering hours to roadmap work.

  

Data analysis and visualisation

• Of Hammers and Nails: What AI Can and Cannot Do for a Data Analyst (5 minute read)
  argues that AI makes analysts faster at coding and drafting, but still struggles with the judgement, context and consistency needed for reliable analysis.

• Dead on Arrival: The AI Dashboard Problem (17 minute read)
  argues that AI can build polished dashboards in minutes, but good analysis, domain knowledge and human judgement still decide whether anyone should trust them.

• How to Visualize Plan vs Actual (6 minute read)
  explains how to make plan-versus-actual gaps obvious through distinct visual styles, selective color and the right choice of bars or lines.

  

Other interesting reads

• Cloudflare And Meta Just Told You Which Jobs Get Cut Next (12 minute read)
  Vin Vashishta argues that AI-driven layoffs will move from reporting roles to coordination roles and finally anyone whose work cannot be tied to measurable business value.

• Notes from inside China’s AI labs (18 minute read)
  shares what he learned inside China’s leading AI labs, from their fast-moving research culture and open-model mindset to their hunger for more compute.

• The economics of superstar AI researchers (10 minute read)
  Interesting article looking at how small differences in research ability can lead to enormous pay gaps when one researcher’s work reaches billions of users and cannot be replicated by simply hiring more people.

Quick favor - need your take

Was there any standout article or topic from May I missed? Feel free to drop a comment or hit reply, even a quick line helps.

If you are already subscribed and enjoyed the article, please give it a like and/or share it others, really appreciate it 🙏

Share

Keep learning

Today we will look at how Grab used AI agents to cut repetitive data support from hours to minutes.

But before that, I wanted to share with you what you could unlock if you share Data Tinkerer with just 1 more person.

There are 100+ resources to learn all things data (science, engineering, analysis). It includes videos, courses, projects and can be filtered by tech stack (Python, SQL, Spark and etc), skill level (Beginner, Intermediate and so on) provider name or free/paid. So if you know other people who like staying up to date on all things data, please share Data Tinkerer with them!

Refer a friend

Now, with that out of the way, let’s get into how Grab turned repetitive data support into an AI agent workflow.

TL;DR

Situation

Grab’s Analytics Data Warehouse team supported 1,000+ monthly users and 15,000+ tables, but repetitive ‘quick questions’ were consuming ~40% of engineering time.

Task

The team needed to reduce manual support work while keeping answers accurate, safe and trusted for production data workflows.

Action

Grab built a multi-agent AI system with specialised agents for data investigation, code search, on-call checks, summarisation and enhancement requests.

Result

Resolution time dropped from hours to minutes, support backlog was largely removed and several FTEs worth of engineering capacity shifted back to roadmap work.

Use Cases

Data quality validation, internal support bot, BI troubleshooting assistant, tracing data lineage

Tech Stack/Framework

FastAPI, LangGraph, Redis, PostgreSQL, tiktoken, Trino, Apache Hive, Delta Lake, Apache Spark, Apache Airflow, RAG

Explained further

Context

Grab’s Analytics Data Warehouse team supports more than 1,000 users every month. Those users rely on a repository of over 15,000 tables, which powers roughly half of all queries across Grab’s data lake.

That scale is useful, obviously. It also creates a very familiar data platform problem: the team becomes the help desk for every ‘quick question’ that is not actually quick.

Someone asks what a column means. Someone else wants to know why a table looks wrong. Another person needs help tracing where a metric came from. Then there are quality checks, source investigations, pipeline questions and basic enhancement requests.

Individually, none of these requests look dramatic. Collectively, they were eating about 40% of the team’s time. That is roughly two days every week spent on repetitive support work instead of building the next layer of the platform.

The Grab team responded by building a multi-agent AI system that could handle simpler questions autonomously and work through more complex investigations with help from specialised agents. The result was a shift from reactive support back to proactive engineering work, with hundreds of hours of productivity unlocked every month.

The Friday Slack message every data engineer fears

It is 5:00 PM on a Friday. A Slack message lands:

“The vehicle_id in our production table looks gibberish. Is the pipeline broken?”

Every data team knows this message. It might sound small but it is not.

To answer it properly, an engineer needs to check the data catalogue, inspect the table, trace the lineage, validate the SQL, look at the code, check pipeline health and maybe search through Slack or Confluence for recent incidents. By the time the stakeholder receives a confident answer, what looked like a five-minute question has quietly consumed a large chunk of engineering time.

The Grab team studied these requests and found a pattern. The problems were different, but the investigation process was often the same. The team had to gather context, check metadata, inspect data, search code, validate assumptions and summarise the answer. That consistency was the opportunity.

Rather than trying to answer every question manually, the team built a multi-agent AI system that could automate the context hunt. Engineers would still stay in the loop where needed, especially for production changes and judgment-heavy decisions. But the repetitive investigation steps could be handled by agents.

Solution

The system was designed around one simple idea: separate the reasoning layer from the action layer.

The large language model handles interpretation and planning. The agents handle specialised work such as querying data, searching code, checking incidents, reviewing pipeline health and drafting code changes. This made the system easier to reason about, easier to debug and safer to operate.

Grab used FastAPI to handle incoming requests and LangGraph to manage the agent workflows. LangGraph was important because this was not a simple one-shot LLM call. The agents needed to loop, pass work to each other, ask for more information and maintain state across a multi-step investigation.

Redis was used for caching and real-time session needs. PostgreSQL served as persistent memory, storing conversation history and agent metadata.

The system also connected into Grab’s internal platforms:

Hubble, the central metadata management platform and data catalogue.

Genchi, the data quality observability platform used to enforce data contracts.

Lighthouse, the platform used to track execution status and monitor pipeline health.

How a Slack question turns into an agent workflow

The journey starts in Slack. When a user sends a request, the system routes it into one of two broad streams.

The first stream is for enhancement requests. These go to the Enhancement Agent, which works with engineering tools such as GitLab, Apache Spark and Airflow to propose and test code changes.

The second stream is for general questions. These go through the investigation pathway, where the system coordinates a small group of specialist agents. The Data Agent queries systems such as Trino, Hive or Delta Lake. The Code Search Agent inspects GitLab. The On-call Agent checks operational context from Confluence, Slack and observability tools.

This design matters because the system is not one giant agent trying to do everything. The ‘brain’ is decoupled from the ‘hands’. The LLM can reason about the request, but the actual work is handled by tools and agents with specific responsibilities.

Why one giant agent was the wrong answer

Grab could have built one large agent and asked it to handle every type of request. That would have been simpler at first glance: one model, one prompt, one inference path.

But that kind of monolithic design becomes hard to maintain as the system grows. Every new tool increases prompt complexity. Every change risks affecting unrelated behaviour. Debugging becomes messy because it is hard to know whether the failure came from classification, data access, code search, reasoning or summarisation.

The multi-agent design traded some latency and coordination complexity for maintainability and accuracy.

A single AI system has the advantage of being simpler to call and manage. But it is also harder to debug, harder to improve safely and more likely to behave like a generalist.

A multi-agent system is more modular. Each agent can be improved independently. The Data Agent can get better at query validation without touching the Code Search Agent. The On-call Agent can be expanded with better incident lookup without changing the Enhancement Agent. Failures are easier to isolate because each part has a narrower job.

For Grab, the choice was obvious. When replacing a manual process that used to take hours, a few minutes of agent coordination is not a serious downside. Accuracy and maintainability mattered more than shaving off a few seconds.

The two-lane system behind Grab’s agent workflow

When a question arrives through Slack, the system first decides which pathway it belongs to.

Pathway 1: Enhancement requests go to the Enhancement Agent.

Pathway 2: Investigation questions go through the Classifier, then one or more specialist agents, then the Summarizer.

The agent that drafts pipeline changes but does not ship them blindly

Some requests are not just questions. They require changes to data pipelines.

For example:

“Can you add a new column for customer_segment?”

Or:

“We need to change the aggregation logic for revenue.”

Previously, this meant an engineer had to gather requirements, inspect schemas, find the right code, check dependencies, write changes, test them and prepare a merge request.

The Enhancement Agent handles much of that heavy lifting.

It gathers context from schemas, lineage, dependencies and the existing codebase. It proposes code changes and creates a merge request. It runs the changes in a test environment. It also flags governance concerns such as PII classification, SLA impact and backward compatibility.

The workflow is intentionally semi-automated.

A user creates a JIRA request. The agent analyses the requirements and gathers extra context through dialogue with the engineer. It creates a Merge Request (MR) with suggested code. The engineer reviews the MR. If the changes are valid, the agent runs them in a test environment. The engineer checks the output against test cases. If everything passes, the engineer merges the MR.

That final approval stays with a human because production pipeline changes need judgment. The agent can do research, write code and run tests, but it should not be blindly changing production systems.

The agent squad for messy data questions

For questions such as “Why does this data look wrong?” or “Where does this metric come from?”, the system uses a coordinated investigation flow.

The Classifier is the first responder. It parses the question, extracts important details such as tables, scripts or columns, detects guardrail violations and decides which specialist agents are needed. It also provides the reasoning and task description for each agent.

For a question like “Why does this ID look wrong?”, the Classifier may route the request to the Data Agent, then the Code Search Agent, then the On-call Agent if production context is needed.

The Data Agent performs the data investigation. It enriches the prompt with table and column metadata, runs SQL queries with guardrails, validates schemas before scanning and retrieves sample data with exploratory comments. This helps confirm whether the user’s observation is real or based on a misunderstanding.

The Code Search Agent inspects the codebase. It traces transformations, follows table lineage across multiple steps and explains transformation logic in plain language. It can also highlight differences between the implementation and what documentation or stakeholders expect.

The On-call Agent checks production context. It searches Slack for outage announcements, checks observability platforms for pipeline status, reviews logs and retry policies, validates quality metrics such as null counts and duplicates, and produces initial incident notes when something looks wrong.

The Summarizer Agent pulls the pieces together. It handles conflicts, turns agent findings into a coherent answer and makes the response concise enough for Slack. It is the final step before the answer is reviewed or posted.

What this looks like in real data team workflows

The clearest way to understand the system is to walk through two common scenarios.

Scenario 1: Adding a new column

A stakeholder creates a JIRA ticket:

“Please add a customer_segment column to the rides table. Source data is available in the user_profiles table.”

In the traditional workflow, this would take a decent chunk of an engineer’s afternoon. There would be requirement checks, code discovery, schema validation, development, testing and MR preparation.

With the Enhancement Agent, the process becomes much faster.

The agent first reads the JIRA ticket to understand what column needs to be added, which target table is involved and where the source data lives.

Then it searches the codebase to find the relevant transformation files. It navigates the repository, identifies the right scripts and locates the parts that need modification.

Before making changes, it validates that the requested column exists in the upstream source table, checks that the column does not already exist in the target table, and confirms schema compatibility and data quality requirements.

It then generates the database schema changes. To do that, it references existing Data Definition Language (DDL) scripts, follows the standard format and creates the required schema modification scripts. These are included in the MR alongside the code changes.

Finally, it creates the MR with documentation. Once the MR is validated, users can interact with the bot to trigger Airflow test runs. They can also specify date ranges or parameters for testing.

The process from ticket to deployable MR is completed in minutes, but the engineer can still inspect the changes before anything moves forward.

Scenario 2: Investigating faulty-looking data

A user asks:

“Why is the ID in the vehicles table unreadable?”

In the old process, an engineer would manually search the catalogue, inspect lineage, write SQL, check logs and review pipeline status.

With agents, the investigation is broken into steps.

First, the Classifier parses the question and decides that all three specialist agents are needed. It plans the sequence as Data Agent, then Code Search Agent, then On-call Agent. Its reasoning is simple: the system needs to verify the data format, trace the transformation logic and check whether there are any production incidents.

The Data Agent starts by retrieving metadata and building a SQL query to inspect sample values. It confirms the user’s observation: the IDs do look unreadable, because they are in UUID format.

It then searches Grab’s data catalogue for dimension tables that could make the values more readable. It finds a suitable dimension table and builds a join query to test whether the UUIDs can be mapped to human-readable vehicle names.

The Data Agent’s conclusion is that the ID column contains valid UUID values. They can be joined with dim_vehicles to get readable vehicle names. The data is not corrupted.

Next, the Code Search Agent traces the lineage. It scans transformation logic in the codebase and finds that the ID is extracted as a raw UUID from a JSON payload from the source system. It queries source samples and confirms that the same pattern exists upstream.

Its conclusion is that the UUID format comes directly from the source system. It is not a bug introduced by Spark transformations.

Then the On-call Agent checks operational health. It reviews Airflow pipeline status, searches Slack for incidents and checks data quality metrics.

Its conclusion is that no production incident is detected. The pipeline is running successfully, quality metrics are normal and no recent complaints have been raised in communication channels.

The Summarizer Agent then turns these findings into a structured answer.

The user’s concern was that ID values looked unreadable. The Data Agent found that the IDs are valid UUIDs and can be joined with dim_vehicles for readable names. The Code Search Agent found that the UUIDs come directly from the source system. The On-call Agent found no production issue.

So the answer is clear: the data looks unfriendly, but it is valid. The pipeline is not broken.

The answer is posted to Slack quickly and marked as unreviewed. This gives users a fast initial response while making it clear that an engineer still needs to review it. The initial response time drops from hours to minutes.

After the answer is posted, anyone can continue the conversation with the agents, which restarts the loop when more clarification is needed.

What Grab had to fix before this worked in production

The prototype worked but making it work in production was harder.

Real users ask messy questions. Conversations get long. Agents use too many tools. Tool outputs can be huge. Some queries are risky. And because this system touches production data workflows, trust matters.

Grab had to optimise the architecture around six major challenges.

1. The agents were drowning in their own context

In multi-agent systems, context grows quickly. Each agent adds messages, tool calls, results and reasoning. If that context is passed forward without discipline, token usage balloons and performance drops.

Grab’s orchestrator maintains a rich execution state, including conversation and tooling history, agent progress and structured responses from each agent. But that state is managed carefully so agents receive the context they need without being flooded.

The system tracks tokens using tiktoken. When limits are exceeded, earlier messages are summarised while preserving information relevant to the original question. Recent messages and critical context remain unsummarised to protect accuracy.

The team also pruned RAG context. Instead of passing entire code files to the Code Search Agent, smaller LLMs extract the most relevant snippets and short descriptions. For database queries, filters are used to retrieve only the most relevant results.

The handoff pattern is also important. Each agent returns its response to a central orchestrator. The orchestrator cleans the context, removes unnecessary tokens and invokes the next agent.

The result is a system that can handle extended investigations without drowning in its own context.

2. Too many tools made the system slower, not smarter

The early version had more than 30 tools, many shaped like generic APIs. That created another problem: tool descriptions and outputs became part of the agent prompt, so the model had to process a lot of irrelevant material before doing useful work.

Grab redesigned tools around real usage. Tool descriptions were shortened. Outputs were aggressively trimmed. Only information needed for decision-making was included.

This reduced the amount of data agents had to process and improved responsiveness.

3. Adding guardrails around SQL and code changes

Agents with database access and code generation powers are useful, but also risky.

They could access sensitive PII, run expensive queries, execute dangerous SQL or generate breaking changes. Grab added multiple layers of safety.

First, the Classifier checks for PII requests and out-of-scope questions before any agent runs.

Second, the Data Agent validates SQL before execution. It checks PII column access, blocks risky DDL and DML operations, detects slow queries and validates schemas before running anything.

Third, all database queries have strict timeout limits.

Fourth, the Enhancement Agent cannot commit directly to master or main. All changes go through merge requests, require human review and run in a test environment first.

The point is not to pretend agents are perfectly safe. The point is to build the system assuming they are not.

4. Designing review into the system

Even with RAG and guardrails, agents can be wrong. A confident wrong answer is worse than no answer, especially in a data platform context.

Grab added human review after the summarised response is generated. Reviewers can approve, reject, refine, re-route to a specific sub-agent or annotate the response.

Approval posts the response with a note that it has been checked. Rejection prevents bad information from reaching users and logs the failure. Refinement lets reviewers add guidance and regenerate the answer. Re-routing sends the question back to a specific agent with more context. Annotation stores structured feedback for future improvement.

This review loop helped build trust while also creating a feedback dataset for improving the system.

5. Letting users move fast without hiding uncertainty

The initial design held back AI-generated answers until an engineer approved them. That was safe, but it created a bottleneck. During busy periods, users could still wait too long.

Grab changed the process. Responses can now be posted immediately, but they are clearly marked as unreviewed. Engineers can still review and modify them afterwards.

This gives users fast answers while setting the right expectation. It also keeps the feedback loop intact.

6. Turning annotations into system improvements

Collecting annotations is useful only if the team learns from them.

Grab turned annotations into an improvement engine. Random annotations are pulled into offline evaluations, so the system is tested against real failure cases rather than only synthetic examples.

The team analyses patterns to see whether the Classifier is routing incorrectly, whether a specific agent has quality issues, whether certain query types cause hallucinations or whether particular table schemas confuse the system.

Quality metrics track annotation rates over time. A sudden rise in rejection rates signals that something needs investigation.

Annotations then guide targeted improvements: better prompts, stronger guardrails, enhanced agents, better few-shot examples, fine-tuning opportunities and regression test suites built from real failures.

The operational impact: faster resolution and more engineering capacity

The multi-agent system changed the team’s operating model.

The bots now autonomously handle most standard user inquiries and a meaningful portion of common enhancement requests. Simple questions that previously required manual investigation can now be answered within minutes.

The resolution time dropped by an order of magnitude. The support backlog was effectively removed. More importantly, the team reclaimed several FTEs worth of engineering capacity.

That capacity shifted from reactive support into proactive roadmap delivery.

This is the real win. The team did not just make support faster. It changed what engineers could spend their time on. Instead of tracing the same definitions, checking the same logs and answering the same data questions, they could return to higher-value platform work. And downstream users got faster answers.

Three principles for applying AI agents in data teams

Grab’s move from manual firefighting to AI-assisted operations came down to three principles.

Multi-agent architecture: Specialists over generalists

Specialised agents worked better than one generalist system because each agent had a narrower job. The Data Agent could focus on SQL and metadata. The Code Search Agent could focus on transformation logic. The On-call Agent could focus on incidents and pipeline health. The Enhancement Agent could focus on code changes.

This made the system easier to maintain, easier to improve and easier to debug.

Strategic human oversight: Building trust through transparency

Human review was not treated as an afterthought. It was part of the product design.

The system could generate fast answers, but reviewers could approve, reject, refine, re-route or annotate them. That created a trust layer for users and a learning loop for the engineering team.

The “unreviewed” label also mattered. It let users benefit from speed without pretending the AI response had the same authority as a checked engineering answer.

Focus on augmentation: Automating repetitive tasks

The agents were not built to replace engineers. They were built to take over the repetitive parts of the work: gathering context, running safe queries, checking logs, tracing code and preparing first-pass answers.

That left humans to focus on the work that actually needs human judgment: architecture, prioritisation, production approvals and building new capabilities.

For Grab’s Analytics Data Warehouse team, that was the shift. Less time spent answering the same operational questions. More time spent building the platform users depend on.

The full scoop

To learn more about this, check Grab's Engineering Blog post on this topic

If you are already subscribed and enjoyed the article, please give it a like and/or share it others, really appreciate it 🙏

Share

Keep learning

Today we will look at how Lyft cut translation latency from days to minutes using AI

But before that, I wanted to share with you what you could unlock if you share Data Tinkerer with just 1 more person.

There are 100+ resources to learn all things data (science, engineering, analysis). It includes videos, courses, projects and can be filtered by tech stack (Python, SQL, Spark and etc), skill level (Beginner, Intermediate and so on) provider name or free/paid. So if you know other people who like staying up to date on all things data, please share Data Tinkerer with them!

Refer a friend

Now, with that out of the way, let’s get to how Lyft turned localisation into a 30-minute AI workflow!

TL;DR

Situation

Lyft’s human-led localisation was high quality but too slow for Québec’s Bill 96 launch and European expansion.

Task

The team needed translations in minutes across 11 locales and 150+ services without losing terminology, tone, compliance or placeholders.

Action

Lyft built an LLM Drafter/Evaluator loop with retries, context injection, deterministic guardrails, prompt versioning and locale-specific tuning.

Result

Translation latency dropped from days to minutes, with a 30-minute SLA target for 95% of batch translations and 95% needing no major human edits.

Use Cases

translation quality evaluation, batch translation, AI-assisted localisation

Tech Stack/Framework

LLMs, Translation Management Systems, Pydantic, prompt engineering

Explained further

Context

For years, Lyft’s localisation setup was built around human translation. Human linguists usually meant strong quality, consistent terminology and fewer awkward mistakes in the app. When Lyft only supported a small number of languages like Spanish, Portuguese and French, multi-day turnaround times and translation costs that grew with every new language were manageable.

Then the business need changed.

Lyft’s Québec launch created a different level of pressure. Bill 96 required French-first user experiences, which meant localisation could not sit in a multi-day queue while product teams waited around. At the same time, Lyft Urban Solutions, the Bikes & Scooters division, wanted to expand into European markets and needed six new languages.

The old workflow was still good at quality, but it was too slow for the pace of the business. So Lyft rebuilt its translation pipeline around AI, while keeping linguists in the loop for review and final quality control. The goal was to reduce translation latency from days to minutes without turning product copy into awkward machine-translated text.

This article breaks down how Lyft did it: the architecture, the Drafter/Evaluator loop, context injection, deterministic guardrails, prompt versioning and locale-specific tuning.

Inside Lyft’s translation pipeline

Before getting into the LLM workflow, it helps to understand the scale of Lyft’s localisation platform.

Lyft’s platform now serves 11 locales across more than 150 services. That means localisation is not just a small content operation sitting next to the app. It is part of the product infrastructure. Translated strings have to flow reliably across many teams, services and surfaces.

The batch translation pipeline handles 99% of Lyft’s app and web content and targets a 30-minute SLA for 95% of translations. Lyft also supports real-time translation, such as ride chat, though that uses a different architecture and is not covered here.

At the centre of the batch pipeline is a dual-path design.

When requesters submit source strings, the pipeline sends them to Smartling, Lyft’s Translation Management System (TMS), for human oversight. At the same time, the same strings go to LLM workers for rapid draft generation.

That split is important because the LLM path gives product teams translated strings quickly enough to unblock launches but the TMS path keeps human review as the quality control layer and system of record. AI translations can ship early, then linguist-reviewed versions replace them once approved.

The batch pipeline has three main phases:

1- Drafting: where translation starts

Requesters submit source strings with context. This includes where the text appears in the UI, what the text is trying to communicate and what tone is expected. This context matters because short UI strings are often ambiguous.

‘Start ride’ is not just two words. It could be a button. It could be an instruction. It could refer to a driver, a rider, a scooter or a bike. Without context, the translation system is guessing.

The Drafter uses that context to generate translation candidates.

2- Unblocking launches: shipping the best draft fast

The Evaluator reviews the candidates and chooses the best one, or rejects all of them.

The selected translation can then be shipped within minutes, which allows product teams to keep moving instead of waiting days for a human translation cycle.

This early-release version is not treated as the final truth. It is a fast, quality-checked draft.

3- Finalisation: human approval and final quality control

Professional linguists review the translations inside the Translation Management System, or TMS. Once approved, the linguist-reviewed version replaces the early-release version and becomes the system of record. If a translation is flagged, it is corrected before distribution.

This dual-path setup is the key design choice.

The LLM path gives speed. The TMS path gives quality control and governance. The AI output can unblock launches, but the human-reviewed version still wins in the end.

Why Lyft chose LLMs over traditional translation?

Lyft started by evaluating traditional Neural Machine Translation providers. These systems were fast, but they often struggled with Lyft-specific terminology and product context.

That is a real problem in localisation because generic correctness is rarely enough. A translation can be grammatically fine while still being wrong for the product. It might ignore official terminology, mishandle regional phrasing, change the tone or translate something that should have stayed in English.

The Lyft team moved toward LLMs because they are better at using context. Traditional NMT services often process each string in isolation. LLMs can take in extra metadata, such as where the string appears, what the intended tone is, what glossary terms apply and which placeholders need to survive untouched.

That extra context is especially useful for product UI.

Take this string:

"Your {vehicle_type} is arriving in {eta_minutes} minutes"

A human immediately sees a few things going on. The tone should probably be clear and calm. It likely appears in a time-sensitive user experience. It needs to sound natural but not too casual. And it may need region-specific wording.

LLMs can work with that additional context.

They can also evaluate translations, not just generate them. That matters because translation is rarely a single-answer problem. Multiple versions can be valid, but one may be clearer, more natural or more aligned with Lyft’s terminology.

This led to Lyft’s core architecture: one model drafts, another model evaluates, and the drafter refines based on feedback when needed.

Building a translation review loop

A simple machine translation pipeline would make one API call: send source text, receive translated text, ship it. But this would be an issue at scale for several reasons:

First, it has no nuance. The output may be technically correct, but not necessarily right for the brand, region or UI.

Second, it has no quality signal. There is no built-in way to know whether the translation is good enough before users see it.

Third, it has no recovery path. If the translation breaks a placeholder, ignores a glossary term or changes the meaning, the system has no structured way to repair it.

Lyft needed something closer to a human review loop: generate a few options, critique them, refine the result and only release a candidate that passes checks.

That became the Drafter and Evaluator pattern.

The “Drafter”: generating translation options

The Drafter’s job is to produce translation candidates. Lyft configures it to produce three distinct candidates for each source string. That is a simple but important design choice.

One translation often converges on the safest or most likely wording. That can be fine, but it may not be the best fit for the specific product context. By asking for three candidates, the system increases the chance that one version handles tone, terminology and regional phrasing better than the others.

For the Drafter, Lyft uses a fast, non-reasoning model. Translation generation is mostly a generative task, and standard models already perform well enough for this part of the workflow.

Speed matters because the system may need to retry. Cost matters because localisation touches a lot of strings. Using a frontier reasoning model for every draft would be overkill.

A simplified Drafter prompt looks like this:

DRAFTER_PROMPT = """
You are a professional translator for Lyft.
Translate into {language} for {country}.
Give {num_translations} translations of the following text.

GLOSSARY: {glossary}
PLACEHOLDERS (preserve exactly): {placeholders}

Text: {source_text}
"""

For example:

source_text = "Your {vehicle_type} is arriving in {eta_minutes} minutes"
language = "French"
country = "Canada"

The Drafter might return:

DrafterOutput(
    candidates=[
        TranslationCandidate(text="Votre {vehicle_type} arrive dans {eta_minutes} minutes"),
        TranslationCandidate(text="Votre {vehicle_type} sera là dans {eta_minutes} minutes"),
        TranslationCandidate(text="Votre {vehicle_type} arrivera d'ici {eta_minutes} minutes"),
    ]
)

That gives the pipeline clear contracts between components. The Drafter returns a known object shape. The Evaluator receives candidates in a predictable format. Parsing is less brittle.

The “Evaluator”: choosing the best translation

The Evaluator acts as a quality gate. It receives all candidates from the Drafter, checks them against a rubric and then either selects the best candidate or rejects the batch.

Lyft uses a reasoning-focused model for this step because evaluation requires comparison and judgment. The model has to check whether the source meaning is preserved, whether glossary terms are followed, whether the tone fits and whether placeholders survived the trip.

The Evaluator grades each candidate across four dimensions.

Accuracy and clarity: Does the translation preserve the full meaning of the source? Is it clear?

Fluency & adaptation: Does it read naturally to a native speaker? Is it appropriate for the target region?

Brand alignment: Does it use official Lyft terminology? Are brand names, proper nouns and airport codes preserved correctly?

Technical correctness: Are spelling, grammar, placeholders and required terms handled properly?

Each candidate receives a grade: pass or revise. If any candidate passes, the Evaluator picks the best one. If all candidates fail, it explains why.

Example output:

EvaluatorOutput(
    evaluations=[
        CandidateEvaluation(
            candidate_index=0,
            grade=Grade.PASS, 
            explanation="Accurate, natural phrasing."
        ),
        CandidateEvaluation(
            candidate_index=1,
            grade=Grade.PASS,
            explanation="Natural and conversational."
        ),
        CandidateEvaluation(
            candidate_index=2,
            grade=Grade.REVISE,
            explanation="'d'ici' implies uncertainty, inappropriate for ETA."
        ),
    ],
    best_candidate_index=0,
)

That last point is a good example of why evaluation matters. The third candidate may look reasonable at first glance, but the phrase introduces uncertainty that does not belong in an ETA message. It is the sort of subtle issue a simple string-by-string translation system can miss.

Why Separate Drafter and Evaluator?

Separating the Drafter and Evaluator gives Lyft several advantages.

The first is practical: spotting errors is often easier than producing the ideal translation on the first attempt. The Evaluator does not need to be the world’s greatest translator. It needs to be a strong reviewer with a clear rubric.

The second is bias reduction. A model judging its own work can be too forgiving. It already has the context of why it chose a phrase, which can make it less likely to challenge the result.

The third is cost control. Lyft can use a faster model for drafting and a more capable reasoning model for evaluation. This keeps the expensive judgment step focused where it matters.

How failed translations get a second (or third) chance

When the Evaluator rejects every candidate, the pipeline does not immediately give up. It captures the critique and feeds it into the Drafter’s next attempt.

Lyft limits this loop to three attempts. The team found that most of the gains come from the first one or two cycles, so three attempts create a useful balance between quality, latency and cost.

A retry critique might look like this:

critique_for_retry = """
All candidates failed glossary compliance. Key issues:
- "Ride" must be translated as "trajet" per Lyft Quebec glossary
- Do not use "course" which is European French
"""

The next Drafter prompt explicitly tells the model what failed and what needs to be fixed: glossary usage, placeholders, tone or formatting.

This feedback loop produces a success rate of more than 95% across most languages. For cases where the Drafter and Evaluator still cannot agree after three tries, the string falls back to the human workflow. Product teams can wait for linguist review in the TMS or request an expedited translation for urgent launches.

Giving the model Lyft’s translation rules

LLMs do not automatically know Lyft’s internal terminology rules. They do not know that {driver_name} is a code variable that must be preserved. They do not know that a certain term should use Lyft’s official translation in Québec French. They do not know which product names should stay in English.

So Lyft injects context into the prompts. The pipeline uses glossary terms, do-not-translate lists and reference data from the TMS.

Glossary terms include official translations for individual words and phrases. For example, “Driver” may need to be translated as “Chauffeur” in French, depending on Lyft’s approved terminology.

Do-not-translate lists cover brand names, product names and proper nouns that should remain in English.

These assets are maintained by linguists and periodically pulled into the Translation Pipeline. It gives both the Drafter and Evaluator the production-specific information they need.

Where rules matter more than prompts

Prompts can reduce errors but they cannot fully prevent them.

LLMs are probabilistic. They may translate placeholders, remove URLs, alter formatting or corrupt structured text. For production localisation, that is a serious problem.

Lyft observed several recurring issues. Placeholder translation was the most common failure mode. For example, {driver_name} could become {nom_du_conducteur}.

Lyft’s guardrail system works in two phases: pre-translation extraction and post-translation validation.

Pre-Translation: masking what must not change

Before content reaches the LLM, the pipeline extracts elements that must be preserved exactly. This includes variables and format strings such as {driver_name}, {eta}, %s, %1$s and %@.

It also includes URLs, email addresses, region codes, HTML tags and escape sequences such as \n, \t and \r. These elements are replaced with numbered tokens like __PH_0__, __PH_1__ and __PH_2__.

Example:

Original:
"Hey {first_name}! Your Lyft arrives at {eta}.\nTrack: https://lyft.com/r/abc"

Masked:
"Hey __PH_0__! Your Lyft arrives at __PH_1__.__PH_2__Track: __PH_3__"

Mapping:
    __PH_0__ → {first_name}
    __PH_1__ → {eta}
    __PH_2__ → \n
    __PH_3__ → https://lyft.com/r/abc

The mapping is included in the prompt so the LLM understands what each token represents and what needs to be preserved.

The numbering also handles reordering. Different languages may place variables in different parts of the sentence, and the token mapping allows that movement without losing the original placeholder.

Post-Translation: validating before release

After the LLM returns a translation, the pipeline validates it before accepting the result. Validation checks that every expected token appears exactly once, no unexpected tokens were introduced and structured content such as HTML tags remains balanced.

When validation fails, the exact error is passed into a retry prompt. For example, the model might be told that __PH_2__ is missing and that it represents \n. That gives the next attempt a concrete repair instruction rather than a vague ‘please do better’, which, to be fair, is a management style but not a validation strategy.

Once validation passes, the tokens are restored:

Translated:
"Salut __PH_0__! Votre Lyft arrive à __PH_1__.__PH_2__Suivre: __PH_3__"

Restored:
"Salut {first_name}! Votre Lyft arrive à {eta}.\nSuivre: https://lyft.com/r/abc"

Treating prompts like production code

In early development, Lyft treated prompts like configuration. The team tweaked wording, tested manually and deployed changes with limited review.

That created a problem. A small prompt change in the Evaluator could increase false rejections, and without proper testing it was difficult to detect the regression or roll back safely.

Lyft now treats prompts as production code.

Prompt templates live alongside the Translation Pipeline code. They go through review. Each prompt version includes a changelog that explains what changed and why. Prompt and model changes are tested against an evaluation suite before receiving production traffic.

The team evaluates changes against ground truth translations in the TMS. Divergences from linguist-approved versions are flagged and reviewed before rollout.

This is a useful lesson for LLM systems generally. Prompts influence system behaviour, so they deserve the same discipline as code: version control, review, testing and rollback.

Testing models without locking into one provider

Lyft does not commit the pipeline to a single model provider. The LLM layer is abstracted, and experimentation happens through Lyft’s configuration infrastructure.

That setup supports traffic splitting, so new models can receive 5%, 20% or 50% of requests. It supports shadow mode, where a candidate model runs in parallel without affecting production. It also supports per-locale overrides and instant rollback without a deploy.

Example config:

drafter:
  model: <fast-generation-model>
  fallback: <fallback-model>
  prompt: <prompt-id>
  shadow:
    model: <candidate-model>
    prompt: <prompt-id>
    traffic_percent: 10

evaluator:
  model: <reasoning-model>
  fallback: <fallback-model>
  prompt: <prompt-id>
  reasoning_effort: medium

locale_overrides:
  <locale_code>:
    evaluator:
      reasoning_effort: high

Through experimentation, Lyft found that faster and cheaper models perform comparably to frontier models for initial generation, while reasoning-focused models do much better at catching subtle evaluation errors.

Tuning prompts to each locale

Different locales need different handling.

Lyft saw this clearly with English variants such as en-GB and en-CA. These variants often require spelling, punctuation and vocabulary adjustments rather than full translation.

For example:

color → colour
trunk → boot

Without clear constraints, the model can overdo it. A simple ‘Try again’ button can become ‘Have another go’. That may be valid British English, but it changes the product voice. It may feel too playful or inconsistent with Lyft’s product voice.

To prevent this, Lyft uses locale-specific overrides that narrow the scope of transformation:

LOCALE_OVERRIDE_EN_GB = """
You are adapting American English text for British English speakers.

IMPORTANT: This is an ORTHOGRAPHIC adaptation, not a full translation.

Only change:
- Spelling (color → colour, center → centre, organize → organise)
- Punctuation conventions where required

DO NOT change:
- Tone or voice
- Sentence structure  
- Casual vs. formal register
- Idioms or expressions (unless they are specifically American and would confuse UK readers)

The goal is that a British reader sees familiar spelling, not that the text "sounds British."
"""

This reduced over-adaptation while keeping Lyft’s brand voice consistent across locales.

What Lyft’s localisation pipeline gets right

Lyft’s localisation pipeline shows how LLMs can be useful in production when the system is designed around their strengths and weaknesses.

The models generate candidates quickly. A separate evaluator reviews them against a rubric. Retry loops repair failures. Context injection keeps terminology aligned with Lyft’s glossary. Deterministic guardrails protect placeholders, URLs, formatting and structured content. Human linguists still review the final output inside the TMS.

That combination reduced translation latency from days to minutes while keeping quality control in place.

The most interesting part is the balance. Lyft uses AI to remove the slowest parts of the workflow, while still relying on deterministic checks and human review where they matter most. The system does not ask the model to be perfect. It gives the model a well-defined job, checks the output and escalates hard cases.

Through this setup, Lyft sees 95% of translations needing no significant changes after linguist review. The remaining 5% tend to be the hard cases: regional idioms, legal language and brand voice decisions.

That feels like the right division of labour. Let the machine handle the repeatable work at speed. Let deterministic systems catch mechanical failures. Let humans focus on the judgment calls where context, nuance and risk actually matter.

Also, keep the placeholders alive. Somewhere out there, {driver_name} is depending on it.

The full scoop

To learn more about this, check Lyft's Engineering Blog post on this topic.

If you are already subscribed and enjoyed the article, please give it a like and/or share it others, really appreciate it 🙏

Share

Keep learning

It’s time for another round-up on all things data and AI!

But before that, I wanted to share with you what you could unlock if you share Data Tinkerer with just 1 more person.

There are 100+ resources to learn all things data (science, engineering, analysis). It includes videos, courses, projects and can be filtered by tech stack (Python, SQL, Spark and etc), skill level (Beginner, Intermediate and so on) provider name or free/paid. So if you know other people who like staying up to date on all things data, please share Data Tinkerer with them!

Refer a friend

Without further ado, let’s get to the round up for April!

Read more

Today we will look at how decision making can be improved.

But before that, I wanted to share with you what you could unlock if you share Data Tinkerer with just 1 more person.

There are 100+ resources to learn all things data (science, engineering, analysis). It includes videos, courses, projects and can be filtered by tech stack (Python, SQL, Spark and etc), skill level (Beginner, Intermediate and so on) provider name or free/paid. So if you know other people who like staying up to date on all things data, please share Data Tinkerer with them!

Refer a friend

With that out of the way, let’s get to the today’s topic of decision making!

Read more

Today we will look at how Airtable cut archive storage costs by 100x and saved millions.

But before that, I wanted to share with you what you could unlock if you share Data Tinkerer with just 1 more person.

There are 100+ resources to learn all things data (science, engineering, analysis). It includes videos, courses, projects and can be filtered by tech stack (Python, SQL, Spark and etc), skill level (Beginner, Intermediate and so on) provider name or free/paid. So if you know other people who like staying up to date on all things data, please share Data Tinkerer with them!

Refer a friend

Now, with that out of the way, let’s get into how Airtable pulled it off.

TL;DR

Read more

Today we will look at how Pinterest used multimodal AI to organise products into relevant shopping collections

But before that, I wanted to share with you what you could unlock if you share Data Tinkerer with just 1 more person.

There are 100+ resources to learn all things data (science, engineering, analysis). It includes videos, courses, projects and can be filtered by tech stack (Python, SQL, Spark and etc), skill level (Beginner, Intermediate and so on) provider name or free/paid. So if you know other people who like staying up to date on all things data, please share Data Tinkerer with them!

Refer a friend

Now, with that out of the way, let’s get to PinLanding: Pinterest’s AI system for shopping collections

Read more

It’s time for another round-up on all things data and AI!

But before that, I wanted to share with you what you could unlock if you share Data Tinkerer with just 1 more person.

There are 100+ resources to learn all things data (science, engineering, analysis). It includes videos, courses, projects and can be filtered by tech stack (Python, SQL, Spark and etc), skill level (Beginner, Intermediate and so on) provider name or free/paid. So if you know other people who like staying up to date on all things data, please share Data Tinkerer with them!

Refer a friend

Without further ado, let’s get to the round up for March!

Read more

Today we will look at how Notion scaled its AI Q&A to millions of users.

But before that, I wanted to share with you what you could unlock if you share Data Tinkerer with just 1 more person.

There are 100+ resources to learn all things data (science, engineering, analysis). It includes videos, courses, projects and can be filtered by tech stack (Python, SQL, Spark and etc), skill level (Beginner, Intermediate and so on) provider name or free/paid. So if you know other people who like staying up to date on all things data, please share Data Tinkerer with them!

Refer a friend

Read more

It’s time for another round-up on all things data and AI!

But before that, I wanted to share with you what you could unlock if you share Data Tinkerer with just 1 more person.

There are 100+ resources to learn all things data (science, engineering, analysis). It includes videos, courses, projects and can be filtered by tech stack (Python, SQL, Spark and etc), skill level (Beginner, Intermediate and so on) provider name or free/paid. So if you know other people who like staying up to date on all things data, please share Data Tinkerer with them!

Refer a friend

Without further ado, let’s get to the round up for February!

Read more

Today we will look at how Shopify scales its product categorisation using agentic AI

But before that, I wanted to share with you what you could unlock if you share Data Tinkerer with just 1 more person.

There are 100+ resources to learn all things data (science, engineering, analysis). It includes videos, courses, projects and can be filtered by tech stack (Python, SQL, Spark and etc), skill level (Beginner, Intermediate and so on) provider name or free/paid. So if you know other people who like staying up to date on all things data, please share Data Tinkerer with them!

Refer a friend

Now, with that out of the way, let’s get to Shopify’s multi-agent taxonomy

Read more

Today we will look at how LinkedIn ingests data at scale.

But before that, I wanted to share with you what you could unlock if you share Data Tinkerer with just 1 more person.

There are 100+ resources to learn all things data (science, engineering, analysis). It includes videos, courses, projects and can be filtered by tech stack (Python, SQL, Spark and etc), skill level (Beginner, Intermediate and so on) provider name or free/paid. So if you know other people who like staying up to date on all things data, please share Data Tinkerer with them!

Refer a friend

Now, with that out of the way, let’s get to Venice: LinkedIn’s data storage platform

Read more

It’s time for another round-up on all things data and AI!

But before that, I wanted to share with you what you could unlock if you share Data Tinkerer with just 1 more person.

There are 100+ resources to learn all things data (science, engineering, analysis). It includes videos, courses, projects and can be filtered by tech stack (Python, SQL, Spark and etc), skill level (Beginner, Intermediate and so on) provider name or free/paid. So if you know other people who like staying up to date on all things data, please share Data Tinkerer with them!

Refer a friend

Without further ado, let’s get to the round up for January!

Read more

Following on from previous posts talking to people in the field, today I will be talking with Ahsaas Bajaj who is a Senior Machine Learning Engineer at Instacart. He works on large-scale recommendation systems that serves millions of customers.

We talked about his rise from software engineering to machine learning at Instacart, how does he decide between rules based vs ML approaches and how he approaches the work now as a more senior stakeholder.

So without further ado, let’s get into it!

Read more

Today we will look at how DoorDash uses anomaly detection to save millions of dollars by flagging fraud trends early.

But before that, I wanted to share with you what you could unlock if you share Data Tinkerer with just 1 more person.

There are 100+ resources to learn all things data (science, engineering, analysis). It includes videos, courses, projects and can be filtered by tech stack (Python, SQL, Spark and etc), skill level (Beginner, Intermediate and so on) provider name or free/paid. So if you know other people who like staying up to date on all things data, please share Data Tinkerer with them!

Refer a friend

Now, with that out of the way, let’s get to DoorDash’s fraud detection!

Read more

Today we will look at how Grab detects data issues in real-time.

But before that, I wanted to share with you what you could unlock if you share Data Tinkerer with just 1 more person.

There are 100+ resources to learn all things data (science, engineering, analysis). It includes videos, courses, projects and can be filtered by tech stack (Python, SQL, Spark and etc), skill level (Beginner, Intermediate and so on) provider name or free/paid. So if you know other people who like staying up to date on all things data, please share Data Tinkerer with them!

Refer a friend

Now, with that out of the way, let’s get to Grab’s real-time work!

Read more

It’s time for another round-up on all things data and AI!

But before that, I wanted to share with you what you could unlock if you share Data Tinkerer with just 1 more person.

There are 100+ resources to learn all things data (science, engineering, analysis). It includes videos, courses, projects and can be filtered by tech stack (Python, SQL, Spark and etc), skill level (Beginner, Intermediate and so on) provider name or free/paid. So if you know other people who like staying up to date on all things data, please share Data Tinkerer with them!

Refer a friend

Without further ado, let’s get to the round up for December!

Read more

Today we will look at how Uber moved from batch to streaming in their data lake.

But before that, I wanted to share with you what you could unlock if you share Data Tinkerer with just 1 more person.

There are 100+ resources to learn all things data (science, engineering, analysis). It includes videos, courses, projects and can be filtered by tech stack (Python, SQL, Spark and etc), skill level (Beginner, Intermediate and so on) provider name or free/paid. So if you know other people who like staying up to date on all things data, please share Data Tinkerer with them!

Refer a friend

Now, with that out of the way, let’s get to Uber’s streaming solution

Read more