KDR Talent Solutions is a modern cloud and data engineering consultancy that helps enterprise organizations build scalable, cloud-native platforms and production-grade data solutions. They are seeking a Lead Data Engineer with deep Databricks expertise to lead complex, client-facing data engineering engagements, architecting and building modern data platforms while mentoring engineers and managing project delivery.
Responsibilities:
- Designing and building scalable data pipelines and lakehouse architectures using Databricks
- Leading end-to-end delivery across complex client-facing data engineering engagements
- Architecting production-grade solutions using Apache Spark, Delta Lake, and cloud-native services
- Writing production-quality code and setting engineering standards through hands-on leadership
- Driving DataOps best practices including CI/CD, automated testing, observability, and infrastructure as code
- Translating complex business requirements into scalable technical architectures and data models
- Mentoring engineers through code reviews, technical guidance, and hands-on collaboration
- Partnering closely with client stakeholders to manage technical scope, delivery timelines, and solution quality
Requirements:
- 8+ years of hands-on data engineering experience building production-grade platforms
- Expert-level Apache Spark experience, including performance tuning and runtime optimization
- Strong programming skills in Python, with Scala experience highly desirable
- Deep experience with Databricks, including Delta Lake, Unity Catalog, and Databricks Workflows
- Experience working across at least two major cloud platforms such as AWS, Azure, or GCP
- Strong understanding of ETL/ELT pipelines, lakehouse architectures, data warehousing, and data modelling
- Experience implementing DataOps practices including CI/CD, testing, observability, and infrastructure as code
- Proven experience leading technical delivery in consulting or client-facing environments
- Strong communication skills with the ability to engage both technical and non-technical stakeholders
- Databricks certifications are a strong plus
- The client is unable to offer visa sponsorship. Applicants must have US Citizenship or a Green Card