Cloudera is a leading company in data services, empowering organizations to transform complex data into actionable insights. They are seeking a Staff Software Engineer with expertise in data infrastructure and a strong focus on Apache Airflow to lead the orchestration and workflow engineering experience within Cloudera Data Engineering.
Responsibilities:
- Drive the multi-year technical roadmap and architectural vision for enterprise-grade Apache Airflow within Cloudera Data Engineering
- Design and optimize highly available, secure, and multi-tenant Airflow environments capable of managing complex dependencies and scheduling workflows across thousands of distributed cluster nodes
- Gain and apply deep technical knowledge across the CDE stack, with a primary focus on extending Airflow, integrating it with Spark and Iceberg, and contributing back to the open-source community where applicable
- Foster a culture of technical excellence through hands-on technical mentorship, rigorous design reviews, and robust architectural guidance for distributed workflow execution
- Collaborate closely with product management, engineering leaders, and cross-functional partners to deliver critical features that make workflow automation seamless and intuitive for CDE customers
Requirements:
- BS or MS in Computer Science or a related technical field
- 10+ years of professional software engineering experience, with a heavy emphasis on data infrastructure, workflow orchestration, and distributed systems
- Deep, production-proven experience architecting, scaling, and extending Apache Airflow (e.g., custom operators, hooks, providers, Celery/Kubernetes executors, and security models)
- Strong understanding of Python (essential for Airflow development) and at least one compiled language such as Java, Scala, C++, or Go
- Deep expertise in cloud-native architectures, specifically running stateful and stateless workloads on Kubernetes (EKS, AKS, GKE) and/or private cloud environments (OpenShift, Rancher)
- Strong understanding of large-scale systems design, scheduling algorithms, resource allocation, and performance tuning
- Exceptional communication skills, an open-minded attitude, and a passion for clean code, technical quality, and mentorship