Sr. Databricks Architect

Location: Remote
Work Hours: Must be able to work in Pacific Time (PST)
Duration: 12 Months
Employment Type: Contract
Industry Experience: Public Sector experience required

Position Overview

We are seeking an experienced Databricks Architect / Resident Solution Architect to design, develop, and deliver modern, scalable data platforms and solutions. The ideal candidate will have extensive experience in data engineering, cloud data platforms, Databricks implementations, and client-facing technical leadership.

This role requires strong hands-on expertise with Databricks and Apache Spark, along with the ability to translate complex business and technical requirements into scalable, secure, and cost-effective data architecture.

Primary Skills

• 12–15+ years of experience in Data Engineering, Data Platforms, Data Analytics, and Modern Data Warehouse solutions
• 10+ years of consulting and client-facing technical delivery experience
• Proven experience delivering 6–8+ end-to-end Databricks implementations
• Strong hands-on experience as a Databricks Developer, Technical Lead, or Solution Architect
• Advanced expertise in Databricks Lakehouse Platform
• Strong knowledge of Apache Spark, including:
  • Performance optimization
  • Partitioning strategies
  • Execution plans
  • Memory management
  • Spark runtime internals
• Extensive experience building scalable ETL/ELT pipelines
• Hands-on experience with:
  • Databricks
  • Delta Lake
  • Structured Streaming
  • Modern data engineering frameworks
• Strong experience designing and implementing cloud-native data platforms across AWS, Azure, and/or Google Cloud Platform
• Deep hands-on proficiency with at least one major cloud platform
• Strong experience tuning large-scale distributed data workloads
• Ability to design highly performant, scalable, and cost-efficient data processing solutions
• Experience troubleshooting complex data platform challenges and recommending appropriate architecture patterns

Secondary Skills

• Databricks Data Engineering Professional Certification or equivalent advanced Databricks certification
• Completion of relevant Databricks learning paths and coursework
• Databricks Solutions Architect Champion experience/program participation
• DevOps and CI/CD experience for production-grade data solutions
• Experience with:
  • Azure DevOps
  • GitHub Actions
  • GitLab CI/CD
  • Jenkins
• Working knowledge of MLOps
• Experience with machine learning lifecycle management, model deployment, and monitoring
• Experience with:
  • Unity Catalog
  • Databricks Workflows
  • MLflow
  • Delta Live Tables
  • Other Databricks platform capabilities
• Strong understanding of enterprise data architecture, governance, scalability, security, and reliability

Required Qualifications

• 12–15+ years of relevant data engineering/data platform experience
• Strong Databricks implementation background
• Extensive Apache Spark experience
• Public sector experience
• Strong client-facing consulting experience
• Excellent communication and stakeholder management skills
• Ability to work remotely while supporting Pacific Time (PST) business hours
• English language proficiency

Key Responsibilities

• Lead the architecture and implementation of enterprise Databricks solutions
• Design scalable Lakehouse and modern data platform architectures
• Provide technical leadership across data engineering and analytics initiatives
• Develop and optimize ETL/ELT pipelines using Databricks, Spark, Delta Lake, and related technologies
• Establish performance, scalability, reliability, and cost-optimization strategies
• Guide CI/CD and DevOps practices for data platform deployments
• Troubleshoot complex data engineering and platform issues
• Collaborate with business, engineering, data science, and technology stakeholders
• Provide architecture recommendations aligned with business objectives and technical requirements
• Support enterprise adoption of modern Databricks capabilities and best practices