Technical Talks

Anant Agarwal
Anant Agarwal
Staff Software Engineer & Engineering Lead | Instacart

Go Back

Orchestrating at Scale: How Instacart Manages 20M+ Daily Workflows

video
Missing value detected...
Video will be populated after the conference

ABOUT THE TALK
  • Data Eng & Infrastructure

Building High-Throughput Data Orchestration: Instacart's Journey to 20M Daily Workflows | Explore how Instacart built an enterprise-grade orchestration system handling 20 million daily workflows across diverse technical domains. Learn implementation details of their cloud-native platform combining Apache Airflow and Temporal for robust scheduling and execution. Deep dive into YAML-based workflow definitions, GitOps deployment patterns, and observability solutions that enable reliable scaling. Practical insights from years of production experience, applicable to both startups and enterprises building scalable data infrastructure.

Anant Agarwal

Staff Software Engineer & Engineering Lead

Anant Agarwal

Instacart

Anant Agarwal is a Staff Software Engineer and Technical Lead in Instacart's Data Infrastructure group, leading initiatives for developer and data productivity at scale. During his 4+ years at Instacart, he has developed mission-critical distributed platforms and orchestration systems that power the company's data pipelines, workflow engines, and services ecosystem, processing millions of workflows daily. Previously at VMware, as Staff Software Engineer and one of the early engineers on VMware Cloud on AWS, he built high availability and distributed resource scheduling systems for hybrid cloud environments.

Discover the data foundations powering today's AI breakthroughs. Join leading minds as we explore both cutting-edge AI and the infrastructure behind it. Reserve your spot at before tickets sell out!

Buy Tickets