Databricks Data Engineer

Dice is seeking an experienced Databricks Data Engineer to design, develop, and optimize enterprise-scale data pipelines on the Databricks platform. The role involves building scalable data solutions for Sales Incentive Compensation and collaborating with various stakeholders to translate business requirements into data solutions.

Responsibilities:

• Design, develop, and maintain scalable data pipelines using Databricks, Delta Lake, and Delta Live Tables
• Build and optimize data models, notebooks, workflows, and enterprise data pipelines
• Develop data solutions supporting Sales Incentive Compensation (SIC) including quota management, commission calculations, attainment tracking, and payout reporting
• Collaborate with Sales Operations, Finance, BI, and business stakeholders to translate business requirements into scalable data solutions
• Integrate Databricks with cloud services (Azure, AWS, or Google Cloud Platform) and data sources such as Azure Data Factory, Kafka, Event Hubs, or similar

Requirements:

• 7+ years of Data Engineering experience
• 3+ years of hands-on Databricks development
• Strong expertise in Databricks
• Strong expertise in Delta Lake
• Strong expertise in Delta Live Tables (DLT)
• Strong expertise in Unity Catalog
• Strong expertise in Databricks Workflows
• Strong expertise in Python and/or Scala
• Strong expertise in Advanced SQL
• Strong expertise in ETL/ELT development and data modeling
• Experience with Sales Incentive Compensation (SIC) processes including commissions, quotas, payouts, and attainment calculations
• Experience integrating with Anaplan, Oracle ICM, Varicent
• Experience with Azure, AWS, or Google Cloud Platform cloud platforms
• Excellent analytical, communication, and stakeholder management skills
• Databricks Data Engineer Certification is a plus