Software Engineer, Data Foundations

Whatnot is the largest live shopping platform in North America and Europe, seeking a Software Engineer for their Data Foundations team. This role involves leading the development of critical data platform systems and ensuring the scalability and reliability of data infrastructure to support real-time analytics and machine learning.

Responsibilities:

• Define technical direction and lead development of critical data platform systems that power next-generation capabilities at Whatnot
• Lead the buildout and evolution of our data orchestration and streaming platforms, helping drive them from early platform investment to production-ready usage across the company
• Design scalable, reliable, multi-tenant data infrastructure rather than simply building pipelines on top of an existing platform
• Own core platform concerns including scalability, performance, reliability, SLOs, and operational excellence for data processing systems
• Build high-quality systems that support real-time analytics, event-driven applications, and machine learning infrastructure needs across Whatnot
• Make key decisions about how data systems should be integrated across Whatnot, and then partner cross-functionally to bring that architecture to life
• Collaborate closely with engineering partners across product, platform, machine learning, and data to enable fast, safe, and scalable development
• Help shape broader company architecture across orchestration, stream processing, real-time systems, analytics, and content understanding
• Apply DevOps/SRE practices, including CI/CD, infrastructure automation, monitoring, and incident response, to keep platform systems reliable at scale
• Help define and enforce data governance and compliance standards across our data lake and downstream systems

Requirements:

• 8+ years of experience
• Ability to write well-tested, API-driven applications and services that run at scale in production
• Track record of building scalable, high-performance distributed systems
• Strong experience with streaming data systems (e.g., Kafka, Flink, or Pulsar) and/or orchestration platforms (e.g., Airflow, Dagster, or Temporal), with particular strength in building or evolving the platform layer itself, not just downstream pipelines
• Hands-on experience designing and operating stateful, real-time data pipelines at scale, including concepts like checkpointing, state management, and watermarking
• Experience evaluating architectural tradeoffs in data processing and orchestration systems, including when a given tool is the right solution and when it is not
• Experience building multi-tenant or end-to-end data infrastructure that supports multiple teams or production use cases
• Expertise in managing cloud data stores (Snowflake, BigQuery, Redshift, etc.) and data lake architectures (e.g., AWS S3) for processing and serving data products
• Strong experience supporting operational systems to ensure uptime, reliability, and maintainability, including monitoring, alerting, and incident management
• Familiarity with DevOps/SRE practices and CI/CD tooling, including GitHub and GitHub Actions
• Understanding of data governance and compliance principles (e.g., data access controls, retention policies, auditability)
• Experience managing cloud technologies (AWS or Google Cloud) and comfort with infrastructure-as-code approaches (for example, Terraform)
• Proficiency in at least one server-side programming language (preferably Python), common algorithms and data structures, and software design principles
• Self-starter ethic, with the ability to thrive under a high level of autonomy
• Exceptional interpersonal and communication skills