Key skills
PythonSparkSQLUnityData EngineeringBISnowflakeDatabricksVersion ControlCI/CD
About this role
Role Overview
- Lead the technical evolution of our data platform.
- Design and implement the Unity Catalog structure.
- Lead migration of complex business logic from legacy systems.
- Architect our internal transformation framework using open-source tooling.
- Serve as the resident query performance expert.
- Govern our Databricks compute footprint.
- Build and maintain CI/CD pipelines to automate testing, validation, and deployment.
- Architect the semantic layer in Omni.
- Occasionally take on the BI Developer role.
Requirements
- 8+ years in Data Engineering or Data Architecture, with deep, hands-on experience in the Databricks ecosystem (Unity Catalog, Delta Lake, SQL Warehouses).
- Expert-level SQL and distributed computing skills.
- Demonstrated experience in query optimization and data platform migration.
- Proven experience building data transformation workflows using Delta Live Tables or custom Python/SQL Spark pipelines.
- Strong command of dimensional modeling (Star and Snowflake schemas).
- Hands-on Unity Catalog experience.
- Fluency with Delta Lake internals.
- Hands-on experience with Omni or a comparable modern BI semantic layer.
- Experience with CI/CD pipelines and version control workflows.
Tech Stack
- Python
- Spark
- SQL
- Unity
Benefits
- Employees can work remotely