Lead Data Engineer

KDR Talent Solutions is a modern cloud and data engineering consultancy that helps enterprise organizations build scalable, cloud-native platforms and production-grade data solutions. They are seeking a Lead Data Engineer with deep Databricks expertise to lead complex, client-facing data engineering engagements, architecting and building modern data platforms while mentoring engineers and managing project delivery.

Responsibilities:

• Designing and building scalable data pipelines and lakehouse architectures using Databricks
• Leading end-to-end delivery across complex client-facing data engineering engagements
• Architecting production-grade solutions using Apache Spark, Delta Lake, and cloud-native services
• Writing production-quality code and setting engineering standards through hands-on leadership
• Driving DataOps best practices including CI/CD, automated testing, observability, and infrastructure as code
• Translating complex business requirements into scalable technical architectures and data models
• Mentoring engineers through code reviews, technical guidance, and hands-on collaboration
• Partnering closely with client stakeholders to manage technical scope, delivery timelines, and solution quality

Requirements:

• 8+ years of hands-on data engineering experience building production-grade platforms
• Expert-level Apache Spark experience, including performance tuning and runtime optimization
• Strong programming skills in Python, with Scala experience highly desirable
• Deep experience with Databricks, including Delta Lake, Unity Catalog, and Databricks Workflows
• Experience working across at least two major cloud platforms such as AWS, Azure, or GCP
• Strong understanding of ETL/ELT pipelines, lakehouse architectures, data warehousing, and data modelling
• Experience implementing DataOps practices including CI/CD, testing, observability, and infrastructure as code
• Proven experience leading technical delivery in consulting or client-facing environments
• Strong communication skills with the ability to engage both technical and non-technical stakeholders
• Databricks certifications are a strong plus
• The client is unable to offer visa sponsorship. Applicants must have US Citizenship or a Green Card