Data Engineer

Stack Overflow is hiring a Data Engineer to build and operate data capabilities that support application features and shared data foundations. The role designs data models, integrations, pipelines, and reliable data flows in collaboration with engineering and product teams.

Responsibilities:

• Embed with engineering teams to turn application requirements into data models, pipeline designs, and clear interfaces between systems
• Design, build, and operate production data operations using SQL, Python, PySpark and cloud data technologies such as Databricks, integrating APIs, databases, and structured, semi-structured, and unstructured content
• Build ingestion and transformation workflows that deliver usable data to application services and downstream capabilities
• Design customer-aware data flows that preserve tenant boundaries, source permissions, and data lifecycle requirements, including updates, deletions, and regional constraints
• Make pipelines resilient through incremental processing, checkpointing, safe retries, backfills, replay, and data-quality checks
• Build observability that helps the team understand data freshness, completeness, processing progress, and failures across the pipeline
• Evaluate architecture and tooling choices against product needs, balancing reliability, performance, maintainability, and operating cost
• Maintain and improve shared datasets and pipelines that support both application features and internal analytics, avoiding unnecessary duplication of data and processing
• Contribute to design reviews, testing, documentation, and deployment practices. Share what you learn and mentor less experienced engineers

Requirements:

• 5+ years of relevant engineering experience, including building and operating production data pipelines and contributing to their architecture
• Strong SQL and Python skills, with experience in relational databases, data modeling, and query optimization
• Experience integrating external APIs and working with changing schemas, imperfect source data, and incremental updates
• Experience applying software engineering practices to data systems, including version control, automated testing, code review, CI/CD tooling, observability logging and monitoring
• You love solving hard problems with  the ability to investigate issues across ingestion, transformation, and downstream delivery
• The ability to explain technical tradeoffs and collaborate with technical and non-technical stakeholders to deliver customer value through reliable, understandable, and maintainable solutions
• A track record of sharing knowledge and helping other engineers grow, along with the communication and self-direction needed to work effectively on a distributed team
• Experience with Azure/GCP and cloud data platforms such as Databricks, Apache Spark, Delta Lake, or comparable technologies
• Experience building data systems for customer-facing applications, particularly multi-tenant products or integrations with third-party services
• Experience with document processing, knowledge systems, or pipelines that support AI-powered features
• Experience with access controls, data governance, regional data requirements, or customer-data retention and deletion
• Experience with event-driven processing, infrastructure as code, and monitoring or optimizing cloud workload costs
• Examples of work you're proud of, whether from a previous role, a personal project, open source, or contributions to the Stack Overflow community