Solution Architect data engineering- Azure Databricks and cloud platform

Yoh, A Day & Zimmermann Company is seeking a Solution Architect specializing in data engineering on Azure Databricks and cloud platforms. The role involves designing enterprise-scale data solutions and ensuring the architecture supports analytics, AI/ML, and operational use cases.

Responsibilities:

• Define and implement end-to-end data architecture (ingestion ? transformation ? consumption)
• Design scalable Lakehouse architectures (Databricks, Delta Lake)
• Develop reference architectures, reusable frameworks, and design patterns
• Align architecture with performance, scalability, security, and cost optimization goals
• Define canonical data models, dimensional schemas, and semantic layers
• Architect enterprise data across domains: Claims, Member, Provider, Eligibility
• Ensure data consistency, interoperability, and reuse across teams
• Architect solutions using: Azure Databricks (Mandatory)
• Guide engineering teams on: Spark optimization, Delta Lake design, and data pipeline best practices
• Enable multi-cloud integrations when required
• Define and enforce: Data governance frameworks (lineage, quality, metadata)
• Access control, RBAC, and data security policies
• Ensure compliance with: HIPAA, HITRUST, and enterprise regulatory standards
• Establish data quality SLAs and auditability mechanisms
• Work closely with: Data Engineering teams, Platform Engineering teams, AI/ML Engineering teams
• Provide architectural oversight for pipelines, workflows, and platform capabilities
• Drive standardization across CI/CD, DataOps, and MLOps practices
• Collaborate with business, product, and analytics teams to: Translate requirements into scalable technical solutions
• Lead architecture reviews, design approvals, and governance forums (ARB)
• Communicate architecture decisions and trade-offs to leadership

Requirements:

• Experience: 10+ Years
• Education: Bachelor's Degree (4-year) – Computer Science / Engineering / Related Field
• Strong experience in Azure Databricks (Mandatory) – PySpark, Delta Lake
• Cloud Architecture (Azure preferred)
• Deep expertise in Data Architecture & Data Modeling (OLTP, OLAP, Dimensional)
• Lakehouse / Data Warehouse design patterns
• Hands-on experience with ETL/ELT pipelines and orchestration frameworks
• CI/CD and DevOps practices (Azure DevOps, Git)
• Metadata, lineage, and governance tools (Unity Catalog, Purview)
• Define and implement end-to-end data architecture (ingestion ? transformation ? consumption)
• Design scalable Lakehouse architectures (Databricks, Delta Lake)
• Develop reference architectures, reusable frameworks, and design patterns
• Align architecture with performance, scalability, security, and cost optimization goals
• Define canonical data models, dimensional schemas, and semantic layers
• Architect enterprise data across domains: Claims, Member, Provider, Eligibility
• Ensure data consistency, interoperability, and reuse across teams
• Architect solutions using Azure Databricks (Mandatory)
• Azure Data Services (ADF, ADLS, Synapse)
• Guide engineering teams on Spark optimization, Delta Lake design, and data pipeline best practices
• Enable multi-cloud integrations when required
• Define and enforce data governance frameworks (lineage, quality, metadata)
• Access control, RBAC, and data security policies
• Ensure compliance with HIPAA, HITRUST, and enterprise regulatory standards
• Establish data quality SLAs and auditability mechanisms
• Work closely with Data Engineering teams, Platform Engineering teams, AI/ML Engineering teams
• Provide architectural oversight for pipelines, workflows, and platform capabilities
• Drive standardization across CI/CD, DataOps, and MLOps practices
• Collaborate with business, product, and analytics teams to translate requirements into scalable technical solutions
• Lead architecture reviews, design approvals, and governance forums (ARB)
• Communicate architecture decisions and trade-offs to leadership
• Experience with AI/ML data architectures (feature stores, model pipelines)
• Multi-cloud environments (Azure + GCP/Snowflake)
• Strong understanding of Medallion architecture (Bronze–Silver–Gold)
• Domain Experience (Good to Have) US Healthcare ecosystem: Claims, Member, Provider datasets
• Familiarity with FHIR / HL7 / EMR data models
• Payment Integrity / Risk Adjustment use cases