Lead Data Engineer

EPAM Systems is seeking a Lead Data Engineer to modernize data environments with a focus on cloud and lakehouse solutions using Databricks. The role involves leading scalable pipeline delivery, data migration, and governance while collaborating with technical and business teams.

Responsibilities:

• Design scalable batch and streaming data pipelines
• Deliver data migration initiatives from on-premises to cloud or between platforms
• Develop and optimize data solutions using Databricks and Apache Spark
• Build ETL/ELT processes aligned with data engineering best practices
• Elicit requirements with technical and business teams in a consulting context
• Ensure data quality, governance, and security across platforms
• Monitor and optimize cloud performance and cost
• Troubleshoot and resolve production pipeline issues
• Document architectures, technical processes, and migration strategies

Requirements:

• 5+ years of experience as a Data Engineer
• Proven consulting experience working with clients and stakeholders
• Hands-on experience with data migration projects, from legacy systems to cloud or platform-to-platform
• Expertise in Databricks for development and optimization
• Proficiency in advanced SQL and data modeling
• Skills in Python or Scala for data engineering workloads
• Upper-Intermediate English proficiency (B2, upper-intermediate)
• Strong analytical and solution-oriented mindset
• Clear communication skills and teamwork focus
• Proactive attitude with strong ownership mindset
• Databricks certification
• Experience with CI/CD for data pipelines
• Skills in Infrastructure as Code with Terraform or ARM
• Experience in Spark performance tuning
• Familiarity with Power BI, Kafka, or Unity Catalog