It’s time for another round-up on all things data and AI!

But before that, I wanted to share with you what you could unlock if you share Data Tinkerer with just 1 more person.

There are 100+ resources to learn all things data (science, engineering, analysis). It includes videos, courses, projects and can be filtered by tech stack (Python, SQL, Spark and etc), skill level (Beginner, Intermediate and so on) provider name or free/paid. So if you know other people who like staying up to date on all things data, please share Data Tinkerer with them!

Refer a friend

Without further ado, let’s get to the round up for March!

Data science & AI

• State of Context Engineering in 2026 (12 minute read)
  argues that context engineering is evolving from prompt tinkering into a structured discipline where managing memory, retrieval and state becomes the core challenge of building reliable AI systems.

  

• Bayesian statistics for confused data scientists (15 minute read)
  Nicolas Chagnet explains Bayesian statistics in plain terms by showing that its real strength is not mathematical elegance but giving data scientists a cleaner way to reason about uncertainty and sparse real-world data.

• Agentic AI Engineering Guide (10 minute read)
  argues that most agentic AI systems fail not because the model is weak, but because teams make avoidable engineering mistakes around context, architecture, planning and evaluation.

  

• The Context Problem (29 minute read)
  argues that the AI industry has turned context into a token-priced billing unit even though context should really mean the relational structure that makes information coherent and useful.

• A Visual Guide to Attention Variants in Modern LLMs (26 minute read)
  shows how modern LLM attention has evolved from standard multi-head attention into a growing set of variants each designed to balance quality, memory use and inference efficiency.

  

• The Boon of Dimensionality (6 minute read)
  Amandeep Singh shows that high-dimensional space creates the geometric conditions that make embeddings, random projections and feature separation work in modern machine learning.

• GPT 5.4 is a big step for Codex (9 minute read)
  writes that GPT 5.4 feels like a real step forward for Codex, with gains in usability, speed, context handling and agent reliability that matter more in practice than benchmark scores alone.

• How Shopify Scales Taxonomy Evolution Across 10,000+ Categories With Multi-Agent AI (14 minute read)
  This piece breaks down how Shopify moved from reactive manual updates to a multi-agent system that scans taxonomy branches in parallel, proposes new categories/attributes from merchant data, detects duplicates and runs automated QA through domain-specific judges.

Data engineering

• Layer by Layer, We Built Data Systems No One Understands (9 minute read)
  writes that modern data stacks keep piling on layers in the name of simplicity but the result is often more sprawl, more cost and systems that are harder to understand or tie back to business outcomes.

  

• ETL is Dead (12 minute read)
  argues that ETL is not disappearing in volume but it is fading as the core identity of data engineering as AI shifts the real work toward context and semantics.

• The Data Engineer’s GitHub Portfolio (2026 Edition) (10 minute read)
  writes that a strong data engineering GitHub portfolio should prove technical taste, system thinking and real-world problem solving, not just show a pile of tutorial projects.

  

• The Data Engineering Mindset Every AI Builder Needs (14 minute read)
  writes that most AI products do not break because of the model but because builders ignore the data foundations early on, especially data flow design, data quality and monitoring.

• The absolute beginners guide to databasemaxxing (18 minute read)
  This article walks through database internals from a beginner’s perspective, showing how concepts like parsing, binding, scans and index seeks fit together under the hood.

• Your Data Model Isn’t Broken, Part I: Why Refactoring Beats Rebuilding (12 minute read)
  Chris Hillman makes the case that most broken data models are really bundles of hard-won business knowledge and that careful refactoring is usually smarter than blowing everything up and starting again.

• ClickHouse -> Real-time insight in 15 minutes (19 minute read)
  shows that ClickHouse is built for real-time analytics but getting that performance in production usually means handling enough operational complexity that makes platforms like Tinybird appealing.

  

• How Notion Scaled AI Q&A to Millions of Workspaces (14 minute read)
  This article walks through how Notion scaled AI Q&A to millions of workspaces while increasing onboarding throughput 600x and cutting costs by up to 90%.

Other interesting reads

• Relevance of Business Models for Data (10 minute read)
  makes the case that data teams should start with the business model first, because strategy, architecture, governance and analytics all work better when they are tied to how the company actually creates and captures value.

  

• It’s about the strategy, stupid (14 minute read)
  Timo Dechau makes the case that data work becomes far more useful when it starts with business strategy, not with dashboards, tracking audits or whatever tactic happens to be fashionable.

• What do frontier AI companies’ job postings reveal about their plans? (9 minute read)
  Interesting article suggesting that frontier labs’ job postings reveal where the market is heading, with hiring patterns pointing to heavier go-to-market pushes, new product bets and different strategies for securing compute and data.

Quick favor - need your take

Was there any standout article or topic from March I missed? Feel free to drop a comment or hit reply, even a quick line helps.

If you are already subscribed and enjoyed the article, please give it a like and/or share it others, really appreciate it 🙏

Share

Keep learning

It’s time for another round-up on all things data and AI!

But before that, I wanted to share with you what you could unlock if you share Data Tinkerer with just 1 more person.

There are 100+ resources to learn all things data (science, engineering, analysis). It includes videos, courses, projects and can be filtered by tech stack (Python, SQL, Spark and etc), skill level (Beginner, Intermediate and so on) provider name or free/paid. So if you know other people who like staying up to date on all things data, please share Data Tinkerer with them!

Refer a friend

Without further ado, let’s get to the round up for February!

Data science & AI

• Inside OpenAI’s in-house data agent (14 minute read)
  OpenAI explains how its in-house data agent combines rich internal context, live querying and self-learning memory to help employees go from vague business questions to trustworthy analysis in minutes.

• A Guide to Which AI to Use in the Agentic Era (17 minute read)
  breaks down the current AI tool landscape into a simple question: which model is best for this specific task, not which one wins the internet on any given day.

• Judgment isn’t uniquely human (19 minute read)
  argues judgment and taste are not uniquely human, and that treating them as off-limits to AI is another case of people underestimating how quickly models can learn high-level cognitive tasks.

  

• I spent $10,000 to automate my research at OpenAI with Codex (6 minute read)
  A researcher from OpenAI argues that people still underestimate what Codex can do in real workflows, sharing a high-usage setup and the practical lessons from using it at serious scale.

• Semantic Linking: the Aboutness of Data (12 minute read)
  expalins that semantic linking is the missing connection between data and meaning, where the real job is not adding labels to tables but explicitly mapping data objects to shared business concepts.

  

• AI Is Finally Eating Software’s Total Market: Here’s What’s Next (11 minute read)
  argues that AI won’t just disrupt software products, it will collapse whole layers of software value unless companies control the workflow, the customer relationship or the data moat.

• World Models and the Data Problem in Robotics (13 minute read)
  Nvidia researcher makes the case that robotics hits a data wall long before an algorithm wall and that world models learned from human first-person video are the most plausible route to scalable robot intelligence.

• Lessons learned from scaling data scientists with AI (10 minute read)
  Whatnot’s lesson from deploying AI for data science is that LLMs don’t remove the need for data scientists, they force teams to get serious about semantic layers and production-grade context management.

  

• How Shopify Scales Taxonomy Evolution Across 10,000+ Categories With Multi-Agent AI (14 minute read)
  This piece breaks down how Shopify moved from reactive manual updates to a multi-agent system that scans taxonomy branches in parallel, proposes new categories/attributes from merchant data, detects duplicates and runs automated QA through domain-specific judges.

Data engineering

• A Portable Analytics Stack (13 minute read)
  shows how a portable analytics stack built on DuckDB, DuckLake, dlt and SQLMesh can replace warehouse-heavy setups with lightweight, version-controlled pipelines that run locally or on cheap scheduled compute.

  

• Healing Tables: When Day-by-Day Backfills Become a Slow-Motion Disaster (24 minute read)
  Chris Hillman shows why incremental historical backfills corrupt dimensions and proposes a healing table pattern that separates change detection from period building so history can be rebuilt cleanly.

• 2028 - THE GREAT DATA RECKONING (16 minute read)
  A speculative but funny take by where he imagines a 2028 data industry shakeout where AI wipes out much of the tooling and data theater while the people who survive are the ones with real business context and architecture skills.

• Data Engineers Are Becoming MetadataOps Engineers (10 minute read)
  An interesting take by that the next layer of data engineering is MetadataOps: building AI-ready metadata, semantic structure and agent-facing governance so LLMs stop guessing and start using data reliably.

  

• How Data Engineers Can Leverage AI Tools Without Losing Fundamentals (13 minute read)
  makes the case that data engineers should use AI to accelerate the boilerplate, not outsource the fundamentals because the real leverage still comes from owning modeling, architecture and performance

• Backfills - The Necessary Evil of Data Engineering (12 minute read)
  A practical look by at why backfills happen, why engineers hate them, and how better parameterization, rerunnability, and storage-aware design can make them less painful.

  

• Why Your 5-Second BigQuery Query Isn’t Cheap (13 minute read)
  A practical breakdown of BigQuery pricing by that shows why short query runtimes are a misleading proxy for cost, and why slots are the compute metric that actually matters.

• How LinkedIn Built a Pipeline That Scales to 230M Records/sec Without Breaking SLAs (12 minute read)
  LinkedIn pushed Venice to handle 175M+ lookups per second while ingesting 230M writes per second. This piece breaks down how they balanced compaction, CPU bottlenecks and adaptive throttling to scale ingestion under eventual consistency.

Data analysis and visualisation

• The State of Machine Learning Competitions (34 minute read)
  This report maps the 2025 competition landscape and finds that winning solutions are getting more compute-hungry, transformer-heavy and increasingly shaped by Qwen in NLP while classic tabular methods still hold their ground.

  

Other interesting reads

• AI Doesn’t Reduce Work - It Intensifies It (9 minute read)
  Interesting findings in HBR that AI doesn’t really remove work so much as intensify it, speeding up expectations and raising the risk of burnout instead of delivering the productivity gains companies hoped for.

• The Anthropic Hive Mind (21 minute read)
  Steve Yegge’s take is that Anthropic’s real edge is not just Claude but a hive-mind way of working where humans and AI operate in a shared, high-speed loop that most companies aren’t built for yet.

• The least understood driver of AI progress (36 minute read)
  Anson Ho highlights that software progress, not just bigger chips or more spending, is a major and underappreciated reason AI keeps getting better faster than many people expect.

Quick favor - need your take

Was there any standout article or topic from February I missed? Feel free to drop a comment or hit reply, even a quick line helps.

If you are already subscribed and enjoyed the article, please give it a like and/or share it others, really appreciate it 🙏

Share

Keep learning

It’s time for another round-up on all things data and AI!

But before that, I wanted to share with you what you could unlock if you share Data Tinkerer with just 1 more person.

There are 100+ resources to learn all things data (science, engineering, analysis). It includes videos, courses, projects and can be filtered by tech stack (Python, SQL, Spark and etc), skill level (Beginner, Intermediate and so on) provider name or free/paid. So if you know other people who like staying up to date on all things data, please share Data Tinkerer with them!

Refer a friend

Without further ado, let’s get to the round up for January!

Data science & AI

• Agent design patterns (8 minute read)
  Anthropic engineer provides a grounded guide to designing AI agents that separates real, reliable architectures from overcomplicated agent hype that doesn’t survive production.

  

• Use Agents or Be Left Behind? A Personal Guide to Automating Your Own Work (31 minute read)
  Tim Dettmers cuts through the agent hype, arguing the real value isn’t autonomous magic but practical agents that reliably coordinate tools, memory and execution.

• Piecewise Regression for Time Series Forecasting (7 minute read)
  shares a practical walkthrough of using piecewise regression on time series to detect structural breaks, regime changes and trend shifts that single global models tend to smooth over.

  

• AI is Hitting a Measurement Wall (27 minute read)
  Makes the case that today’s AI benchmarks are saturated and misleading, masking the growing gap between model performance on tests and value in real applications.

• Drift Detection in Robust Machine Learning Systems (18 minute read)
  The article shows how unnoticed drift can quietly degrade model performance and outlines practical techniques to detect it early in production.

• 8 plots that explain the state of open models (7 minute read)
  Eight charts by cut through the noise to show that Chinese open models, led by Qwen, dominate real-world adoption and benchmarks, while Western challengers only compete at the very top end.

  

• LLMs as Judges: Measuring Bias, Hinting Effects, and Tier Preferences (10 minute read)
  and examine when LLMs can act as evaluators, showing how bias, prompt framing and hinting can distort model-as-judge benchmarks.

• How to Build a Recommendation System at Scale: Insights from Instacart (10 minute read)
  A practical walk-through of how large-scale recommendation systems are actually built in production by , covering modeling choices and the tradeoffs that matter once you move past toy examples.

Data engineering

• I spent 5 hours learning Unity Catalog. Here’s everything you need to know (10 minute read)
  provides a breakdown of how Databricks’ open-sourced Unity Catalog works under the hood.

  

• Databricks Lakeflow vs Apache Airflow (13 minute read)
  A candid comparison by showing how Databricks Lakeflow trades Airflow’s flexibility and openness for tighter platform integration, simpler ops and better defaults if you’re already all-in on Databricks.

• The Certifications Scam (7 minute read)
  A blunt takedown of data certifications by , arguing they mostly signal marketing and gatekeeping rather than real skills, experience or on-the-job impact.

• End To End Agentic Data Modeling: Using AI and OpenMetadata MCP for Impact Analysis (8 minute read)
  A hands-on look at building end-to-end agentic data modeling by combining OpenMetadata with MCP-style agents to automate lineage, context sharing and model evolution across the data stack by with

  

• Autofilling the Boring Semantic Layer: From Sakila to Chat-BI with dltHub (9 minute read)
  Adrian Brudaru explores how LLMs can help generate and maintain semantic models on top of data pipelines, reducing manual modeling effort while keeping analytics definitions consistent.

• A Diary of a Data Engineer (13 minute read)
  A candid, day-in-the-life reflection on what data engineering actually looks like in practice by , highlighting the unglamorous but essential work that keeps data systems running day to day.

• Database Development with AI in 2026 (11 minute read)
  Brent Ozar argues that in 2026 AI will meaningfully speed up database development tasks like query writing and troubleshooting but real impact still depends on human judgment and understanding production constraints.

• How Uber Cut Data Lake Freshness From Hours to Minutes With Flink (11 minute read)
  Uber rebuilt its data lake ingestion to move freshness from hours to minutes. This piece breaks down how they replaced batch Spark jobs with Flink streaming, cut compute by 25% and dealt with the very real problems that show up at petabyte scale.

Data analysis and visualisation

• Best Data Visualization Projects of 2025 (3 minute read)
  FlowingData shares the best data visualisations of 2025

  

• The book that finally taught me how to tell stories with data (12 minute read)
  reviews Storytelling with Data, highlighting that impact comes from framing the message and audience first, not from visualisation tricks.

• How to create a more accessible line chart (10 minute read)
  Nicola Rennie shows how small design choices in line charts (color, contrast, labeling and annotations) dramatically improve accessibility without sacrificing clarity or insight.

• 5 Rules for Dashboard Filter Placement (6 minute read)
  breaks down five practical rules for placing dashboard filters so users understand what they’re controlling without adding cognitive load or breaking trust.

Other interesting reads

• ONTOLOGIES - SOME PERSPECTIVES (20 minute read)
  A great intro and explanation of ontologies by (Bill Inmon) and . Really worth a read if you have heard the term a lot but are not sure what it means and how it can be applied

  

• Lessons from Building AI Agents for Financial Services (23 minute read)
  breaks down what building AI agents actually looks like in production, separating real engineering constraints from agent hype.

• Introducing the AI Chip Sales Data Explorer (3 minute read)
  Epoch AI introduces an interactive dataset tracking global AI chip sales, shedding light on who’s actually buying compute and how hardware demand is shaping the AI race.

  

Quick favor - need your take

Was there any standout article or topic from January I missed? Feel free to drop a comment or hit reply, even a quick line helps.

If you are already subscribed and enjoyed the article, please give it a like and/or share it others, really appreciate it 🙏

Share

Keep learning

It’s time for another round-up on all things data and AI!

But before that, I wanted to share with you what you could unlock if you share Data Tinkerer with just 1 more person.

There are 100+ resources to learn all things data (science, engineering, analysis). It includes videos, courses, projects and can be filtered by tech stack (Python, SQL, Spark and etc), skill level (Beginner, Intermediate and so on) provider name or free/paid. So if you know other people who like staying up to date on all things data, please share Data Tinkerer with them!

Refer a friend

Without further ado, let’s get to the round up for December!

Data science & AI

• The State Of LLMs 2025: Progress, Problems, and Predictions (34 minute read)
  provides a great recap of main developments in 2025 and a couple of predictions for 2026 (like classical RAG slowly fading away)

  

• Building a Data Cleaning Agent with LangGraph (7 minute read)
  shows how to build a LangGraph-based data cleaning agent that auto-generates, executes, and fixes Python cleaning code to cut down manual data prep.

• Making Sense of Memory in AI Agents (10 minute read)
  This post breaks down how different memory types (short-term, long-term, and structured) let AI agents retain context across steps so they can act coherently instead of responding statelessly.

  

• Exploring TabPFN: A Foundation Model Built for Tabular Data (12 minute read)
  The article explores TabPFN, a foundation model pretrained on synthetic tasks that delivers strong tabular ML performance with near-zero tuning by reframing tabular prediction as conditional inference.

• How to Use Simple Data Contracts in Python for Data Scientists (8 minute read)
  Eirik Berge walks through a lightweight data contract implementation in Python to catch schema breakages early

• How AI is transforming work at Anthropic (35 minute read)
  An interesting look at how engineers and researchers at Anthropic actually use AI day to day and which parts of their work it genuinely helps with.

  

• We removed 80% of our agent’s tools (4 minute read)
  Vercel rebuilt their text-to-SQL agent by stripping away complex tooling and giving Claude direct file-system access, discovering that fewer tools, better documentation and ‘doing less’ made the agent faster, cheaper and more reliable.

  

Data engineering

• Opinionated Data Platforms vs. Open-Source (18 minute read)
  Good article by breaking down the tradeoffs between open-source and ‘opinionated’ and when it makes sense to go for the latter.

  

• LLMs for {PDF} Data Pipelines (8 minute read)
  experiments with using LLMs as part of a data pipeline, showing that agent-style PDF-to-JSON extraction can work in practice despite slowness and may be good enough for real-world automation.

• Snowflake vs Databricks Is the Wrong Debate (9 minute read)
  argues that the Snowflake vs Databricks debate is a distraction, with Databricks deliberately expanding role by role to own the full data stack and compete with cloud and enterprise platforms.

• Data Quality Design Patterns (11 minute read)
  breaks down practical data quality design patterns like WAP, AWAP, TAP and signal tables, showing how teams balance safety, cost and speed to keep bad data out of production pipelines.

  

• DuckDB: The Swiss Army Knife For Data Engineers (8 minute read)
  argues that DuckDB can replace most pandas, Spark, and Airflow workflows by letting data engineers run fast, scalable analytics and ETL directly with SQL, zero infrastructure and minimal complexity.

• How Snap Rebuilt Its ML Platform to Handle 10,000+ Daily Spark Jobs (14 minute read)
  This post breaks down how Snap rebuilt its ML platform with a unified Spark layer to tame spiky workloads, standardise pipelines, and reliably run 10,000+ production jobs a day without blowing up clusters.

Data analysis and visualisation

• Saloni’s guide to data visualization (41 minute read)
  Great and comprehensive post by where she distils data visualization down to first principles, showing how to choose charts, reduce clutter and design visuals that communicate insight instead of just decorating dashboards.

• Broken Chart: discover 9 visualization alternatives (10 minute read)
  Dominic Royé breaks down how common chart design mistakes distort interpretation, showing why many broken charts mislead viewers and how to fix them with clearer scales, context, and visual discipline.

Other interesting reads

• The Most Useful, Timeless Skill to Learn as a Data Professional (11 minute read)
  makes the case that real impact in data comes from using leverage and not just more lines of code

• 2026 - General Thoughts on What’s Ahead (6 minute read)
  thinks that 2026 will be a deliberately ‘boring’ year where AI hype cools off and teams are forced to focus on fundamentals that actually make AI work.

• The next data bottleneck (11 minute read)
  argues that as tools and models get better, the real constraint shifts to human bottlenecks like decision-making, ownership and organisational ability to turn data into action.

Quick favor - need your take

Was there any standout article or topic from November I missed? Feel free to drop a comment or hit reply, even a quick line helps.

If you are already subscribed and enjoyed the article, please give it a like and/or share it others, really appreciate it 🙏

Share

Keep learning

It’s time for another round-up on all things data!

But before that, I wanted to share with you what you could unlock if you share Data Tinkerer with just 1 more person.

There are 100+ resources to learn all things data (science, engineering, analysis). It includes videos, courses, projects and can be filtered by tech stack (Python, SQL, Spark and etc), skill level (Beginner, Intermediate and so on) provider name or free/paid. So if you know other people who like staying up to date on all things data, please share Data Tinkerer with them!

Refer a friend

Without further ado, let’s get to the round up for November.

Data science & AI

• Effective context engineering for AI agents (16 minute read)
  Anthropic’s Applied AI team explains that context engineering is now the real bottleneck in agent design - the craft of curating a small set of high-signal tokens so agents stay focused, retrieve what they need just-in-time, and remain coherent across long, complex tasks.

  

• Simple, Battle-Tested Algorithms Still Outperform AI (11 minute read)
  Jose Crespo argues that companies are wasting hundreds of billions on overhyped AI that delivers negative ROI, while a handful of simple, century-old algorithms plus competent programmers crush it on real business problems.

• How Can You Identify an Agentic AI Use Case? (13 minute read)
  Pierre Petrella explains that an AI agent is an LLM with tools, a clear task and reasoning and gives a practical framework to spot high-value agent use cases in your org by mapping inputs, outputs, tools, scope and playbooks.

• Gemini 3 Prompting: Best Practices for General Usage (6 minute read)
  Philipp Schmid from Google Deepmind lays out practical prompting patterns for Gemini 3: be direct, highly structured (XML/Markdown), explicitly plan and self-check and use agent-style tool and domain-specific workflows to get more reliable and high-quality outputs.

• Beyond Standard LLMs (36 minute read)
  reviews four non-standard LLM directions - linear-attention hybrids, text diffusion, code world models and tiny recursive transformers - and concludes they’re promising niche upgrades but still complements, not replacements for classic autoregressive transformers.

  

• RL is even more information inefficient than you thought (11 minute read)
  argues that RL learns far fewer bits per sample than supervised learning so it only becomes efficient once models are already strong

• The 1 Billion Token Challenge: Finding the Perfect Pre-training Mix (10 minute read)
  Asankhaya Sharma finds that a simple 50-30-20 data mix (finePDFs, DCLM-baseline, FineWeb-Edu) lets a 1B-token GPT-2 run reach >90% of GPT-2 performance using a tenth of the data, outperforming all curriculum strategies.

• A Visual Introduction to Dimensionality Reduction with Isomap (9 minute read)
  Alec Helbling provides a good introduction to dimensionality reduction with visuals (and math)

  

• Semantic Layers and the Future of Agentic Analytics (9 minute read)
  says Agentic Analytics is the next big shift, powered by semantic layers and OSI standards that let AI agents finally understand business context instead of guessing.

• From Data Analyst to Senior DS Manager at Skyscanner (16 minute read)
  talks about his rise from data analyst to Senior DS Manager at Skyscanner, what “production-ready” really means and why the real intelligence in data science lives before and after the model.

Data engineering

• Is DuckLake a Step Backward? (17 minute read)
  argues that DuckLake revives relational metadata for lakehouses to simplify operations and improve transactional safety but may struggle to scale beyond mid-sized workloads unless the ecosystem and tooling mature.

  

• Event Streaming is Topping Out (16 minute read)
  argues that the real-time streaming market is saturated and shrinking, with Kafka vendors facing price wars, weak growth and looming consolidation even as Kafka itself remains entrenched.

• Learnings after 4 years working with +50 companies on data engineering projects (5 minute read)
  ’s takeaway after working with 50+ companies on data: forget the hype, fix your ingestion and schemas, drop unused data and keep your stack simple if you want real speed and savings.

• The Dark Data Tax: How Hoarding is Poisoning Your AI (10 minute read)
  warns that hoarding data has become a metabolic failure where 90% unused tables bloat costs, degrade AI signal and choke analytics.

• What It Really Takes to Move From Senior to Staff Data Engineer (10 minute read)
  A good discussion by and a Staff Data Engineer at Apple about what it takes to crack the move to more senior positions.

• Apache Spark Fundamentals for Data Engineers (11 minute read)
  explains how Apache Spark works from the ground up, using a simple CSV example to walk through its history, architecture, ecosystem and step-by-step execution.

• Iceberg CDC: Stream a Little Dream of Me (15 minute read)

  Anton Borisov breaks down how Iceberg’s v3 identity tracking and v4 compact metadata turn messy equality-delete upserts into predictable, low-overhead CDC, especially when paired with a catalog that hands out clean global deltas.

• Data Warehouse, Data Lake, Data Lakehouse, Data Mesh: What They Are and How They Differ (14 minute read)
  breaks down warehouse, lake, lakehouse and mesh and talks about picking the ‘right’ one depending on your biggest bottleneck

• 650GB of Data (Delta Lake on S3). Polars vs DuckDB vs Daft vs Spark (9 minute read)
  shows that DuckDB, Polars and Daft can chew through a 650GB Delta Lake on a single 32GB node, proving most teams don’t need clusters nearly as often as they think.

Data analysis and visualisation

• The Complete Guide to Dashboard Testing: Ensuring Quality and Reliability (7 minute read)
  explains how to bulletproof dashboards end-to-end by validating design, interaction, data pipelines, filters and load before they ever reach stakeholders.

• The Data Analyst’s Dilemma: Accuracy vs Speed (7 minute read)
  The analyst’s dilemma: knowing when to aim for perfect accuracy and when “good enough” is all the business really needs.

Other interesting reads

• Eroding the Edges: (AI-Generated) Build vs. Buy and the Future of Software (12 minute read)
  Interesting take by about “good enough” AI-coded apps blowing up the old buy vs. build calculus and pushing engineers to act more like orchestrators rather than builders.

• The Low-Tech Revolution: Why AI Will Transform the Industries Tech Forgot (32 minute read)
  makes the case that cheap models, edge hardware and agentic software flip the math for thin-margin sectors, making AI adoption inevitable even as platform consolidation threatens to capture most of the value.

  

• Benchmark Scores = General Capability + Claudiness (8 minute read)
  Greg Burnham shows that most benchmark scores collapse into one ‘general capability’ factor with a smaller ‘Claudiness’ axis on top and argues this hints we’re in a world where broad skills partly generalize but still require targeted, paid-for improvements on specific abilities.

  

Quick favor - need your take

Was there any standout article or topic from November I missed? Feel free to drop a comment or hit reply, even a quick line helps.

If you are already subscribed and enjoyed the article, please give it a like and/or share it others, really appreciate it 🙏

Share

Keep learning

It’s time for another round-up on all things data!

But before that, I wanted to share with you what you could unlock if you share Data Tinkerer with just 1 more person.

There are 100+ resources to learn all things data (science, engineering, analysis). It includes videos, courses, projects and can be filtered by tech stack (Python, SQL, Spark and etc), skill level (Beginner, Intermediate and so on) provider name or free/paid. So if you know other people who like staying up to date on all things data, please share Data Tinkerer with them!

Refer a friend

Without further ado, let’s get to the round up for October.

Read more

It’s time for another round-up on all things data!

But before that, I wanted to share with you what you could unlock if you share Data Tinkerer with just 1 more person.

There are 100+ resources to learn all things data (science, engineering, analysis). It includes videos, courses, projects and can be filtered by tech stack (Python, SQL, Spark and etc), skill level (Beginner, Intermediate and so on) provider name or free/paid. So if you know other people who like staying up to date on all things data, please share Data Tinkerer with them!

Refer a friend

Without further ado, let’s get to the round up for September.

Read more

It’s time for another round-up on all things data!

But before that, I wanted to share with you what you could unlock if you share Data Tinkerer with just 1 more person.

There are 100+ resources to learn all things data (science, engineering, analysis). It includes videos, courses, projects and can be filtered by tech stack (Python, SQL, Spark and etc), skill level (Beginner, Intermediate and so on) provider name or free/paid. So if you know other people who like staying up to date on all things data, please share Data Tinkerer with them!

Refer a friend

Without further ado, let’s get to the round up for August.

Read more

As I mentioned last week, I’m making a few changes to the newsletter based on your feedback. Here’s what’s new:

1- Data roundup: I’m trialing a new format that focuses more on deep dives, practical how-tos, and longer-form content to help you sharpen your craft. Less hype about the latest model, more substance. Each month, I’ll share a curated roundup of the most useful pieces I’ve come across in AI and data across data science, data engineering and data analysis.
So: less news, more depth.

2- Referral prizes: I’m changing the referral prizes to:

1 share - 100+ resources to learn all things data (science, engineering, analysis). It includes videos, courses, projects and can be filtered by skill, level or free/paid

2 shares - 100+ cheatsheets for all things data (science, engineering, analysis)

Read more

It’s time for another weekly round-up on all things Data and AI. But before that, I wanted to mention If keeping up with the latest news is not your jam, you can click on settings and turn it off (Data Bytes) and keep receiving emails for the other topics you are interested 👇

I’d rather you receive fewer emails that you are interested in than getting stuff you are not crazy about :)

Now, with that out of the way, let’s get to this week’s round-up!

Read more

It’s time for another weekly round-up on all things Data and AI. But before that, I wanted to mention If keeping up with the latest news is not your jam, you can click on settings and turn it off (Data Bytes) and keep receiving emails for the other topics you are interested 👇

I’d rather you receive fewer emails that you are interested in than getting stuff you are not crazy about :)

Now, with that out of the way, let’s get to this week’s round-up!

Read more

It’s time for another weekly round-up on all things AI and data. But before that, I want to mention again, if you are a data person in the trenches and interested in sharing your experience and learnings with other people, reply to the email or message me and we can work it out. We get almost 10k views every month and your experience would be valuable to others

Now, with that out of the way, let’s get to this week’s round-up!

Read more

Today I’m announcing a brand new addition to my Substack publication: Data Tinkerer subscriber chat. This is a conversation space for subscribers. kind of like a group chat or live hangout. I’ll post questions and updates that come my way and you can jump into the discussions and or make suggestions.

Join chat

How to get started

1. Get the Substack app by clicking this link or the button below. New chat threads won’t be sent sent via email, so turn on push notifications so you don’t miss conversation as it happens. You can also access chat on the web.

Get app

2. Open the app and tap the Chat icon. It looks like two bubbles in the bottom bar, and you’ll see a row for my chat inside.

3. That’s it! Jump into my thread to say hi, and if you have any issues, check out Substack’s FAQ.

Feel free to share your opinion or suggestions there, I’d really appreciate it.

Now let’s get to this week’s round-up!

Read more

It’s time for this week’s round-up on all things data and AI. Before that, just sharing that you can have access to 100+ cheat sheets like below, if you share Data Tinkerer with just 2 other people

So if you enjoy reading Data Tinkerer, share with friends and earn referral rewards!

Refer a friend

Now, with that out of the way, let’s get to this week’s news round up on all things data and AI

Read more

It’s time for this week’s round-up on all things data and AI. Before that, just sharing that you can have access to 100+ cheat sheets like below, if you share Data Tinkerer with just 2 other people

So if you enjoy reading Data Tinkerer, share with friends and earn referral rewards!

Refer a friend

Now, with that out of the way, let’s get to this week’s news round up on all things data and AI

Read more

Before we get to this week’s round up of all things data and AI, just wanted to remind you that you can unlock the following prizes by referring Data Tinkerer to others:

1 share - a PDF of top 15 articles on Data Tinkerer

2 shares - 100+ cheat sheets for all things data. From Python, SQL, Power BI to Tableau, Spark, Git and many more

4 shares - 100+ resources to learn all things data (science, engineering, analysis). It includes videos, courses, projects and can be filtered by skill or whether it’s free or paid

Read more

Before we get to this week’s round up of all things data, just wanted to mention the referral program with the new subscribers. Based on the number of people you share Data Tinkerer with, you can unlock the following rewards:

1 share - a PDF of top 15 articles on Data Tinkerer

2 shares - 100+ cheat sheets for all things data. From Python, SQL, Power BI to Tableau, Spark, Git and many more

4 shares - 100+ resources to learn all things data (science, engineering, analysis). It includes videos, courses, projects and can be filtered by skill or whether it’s free or paid

Read more

Before we get to this week’s round up of all things data, I’m excited to announce that I’m making the referral gifts even easier to grab. So based on the number of people you share Data Tinkerer with, you can unlock the following rewards:

1 share - a PDF of top 15 articles on Data Tinkerer

2 shares - 100+ cheat sheets for all things data (science, engineering, analysis)

4 shares - 100+ resources to learn all things data (science, engineering, analysis). It includes videos, courses, projects and can be filtered by skill or whether it’s free or paid

Read more

Thank you to those who provided feedback last week and to those who shared the publication with others. I really appreciate it ❤️

I wanted to share an example of what you could unlock if you share Data Tinkerer with just 3 other people.

There are 100+ more cheat sheets covering everything from Python, R, SQL, Spark to Power BI, Tableau, Git and many more. So if you know other people who like staying up to date on all things data, please share Data Tinkerer with them!

Refer a friend

Now, with that out of the way, let’s get to this week’s news round up on all things data and AI

Read more

Thank you to those who provided feedback last week and to the 3 people who shared the publication with others. I really appreciate it ❤️

I wanted to share an example of what you could unlock if you share Data Tinkerer with just 3 other people.

There are 100+ more cheat sheets covering everything from Python, R, SQL, Spark to Power BI, Tableau, Git and many more. So if you know other people who like staying up to date on all things data, please share Data Tinkerer with them!

Refer a friend

Now, with that out of the way, let’s get to this week’s news round up on all things data and AI

Read more