Job Title: Sr. Databricks Data Engineer

Location: Austin, TX, (Hybrid)-3 days remote with 2 days (Mondays and Thursdays) required to be onsite

Position Type: Contract

Interview Mode: Both MS Teams & In-person

Key Responsibilities:

Design, develop, and optimize scalable data solutions on the Databricks platform.
Develop large-scale data processing solutions using PySpark or Scala.
Build and maintain data ingestion pipelines and Delta Live Tables (DLT).
Implement and support Medallion Architecture (Bronze, Silver, Gold).
Develop data models and implement data quality, validation, and governance processes.
Create dashboards, Databricks Apps, and analytical solutions to support business reporting.
Optimize workloads, monitoring, and operational processes for scalability, security, and cost efficiency.
Analyze business requirements and identify effective technical solutions.
Review existing system capabilities and recommend enhancements where appropriate.
Prepare functional and technical documentation to support application development and system modifications.
Collaborate with business users and technical teams to gather requirements and support solution delivery.
Independently perform complex tasks while leveraging strong analytical, problem-solving, and decision-making skills.
Utilize 8+ years of experience to plan, prioritize, and deliver data engineering solutions effectively.

II. CANDIDATE SKILLS AND QUALIFICATIONS

Minimum Requirements:

Candidates that do not meet or exceed the minimum stated requirements (skills/experience) will be displayed to customers but may not be chosen for this opportunity.

Years

Required/Preferred

Experience

Required

Experience in IT, supporting the design, development, deployment, or delivery of technology solutions.

Required

Experience with Databricks, including building and optimizing ETL/ELT data pipelines using Apache Spark.

Required

Experience in data warehousing and dimensional data modeling (star/snowflake schemas).

Required

Proficiency in SQL and Python (or Scala) for large-scale data processing.

Required

Experience designing and developing dashboards and applications natively within Databricks (e.g., Databricks SQL dashboards, Databricks Apps).

Required

Experience implementing data governance, data quality, and data security practices.

Required

Experience implementing Lakeflow Declarative Pipelines (formerly Delta Live Tables/DLT) for building and managing production data pipelines.

Required

Experience with Delta Lake, medallion architecture (bronze/silver/gold layers), data lakehouse design, and creating and scheduling offline jobs using Lakeflow Jobs (formerly Databricks Workflows) or similar orchestration tools (e.g., Airflow).

Required

Excellent communication skills, both verbal and written, including presenting insights to technical and business stakeholders.

Preferred

Experience working in public sector or state government environments.

Preferred

Databricks certification (e.g., Databricks Certified Data Engineer Associate/Professional).

Preferred

Experience with CI/CD practices for data pipelines (DevOps, Git-based workflows).

Sr. Databricks Data Engineer

Key skills

About this role