Data Engineer

Commercial Bank of California is hiring a Data Engineer to build, maintain, and support secure data pipelines and analytical data stores for enterprise analytics, reporting, and AI use cases. The role contributes to Azure and Microsoft Fabric lakehouse solutions, supports Databricks-to-Fabric migrations, and partners with business intelligence, analytics, AI, and business stakeholders.

Responsibilities:

• Develop, test, and maintain batch and streaming data pipelines using Databricks, Fabric Data Pipelines, and Azure-native services
• Implement ETL/ELT processes that ensure data quality, accuracy, and consistency across systems
• Build and maintain lakehouse and medallion (Bronze/Silver/Gold) data layers following team standards
• Support Databricks-to-Fabric migration work when and where applicable, including converting notebooks and pipelines, and validating results
• Monitor pipelines, troubleshoot failures, and help tune for performance, scalability, and cost efficiency
• Integrate and model data from FIS core banking and H360 systems into the lakehouse for analytics and reporting
• Support pipelines for payments data (e.g., ACH, wire, card, and real-time payments), including ingestion, reconciliation, and reporting
• Design, build, and maintain enterprise-grade data platforms using Microsoft Fabric, Azure Data Lake, Azure Synapse, and Azure Databricks
• Architect and implement lakehouse and medallion architectures (Bronze/Silver/Gold) to support analytics and BI workloads
• Define standards for data ingestion, transformation, storage, and access across structured and semi-structured data sources
• Microsoft expertise across the data and analytics stack (Fabric, Azure Data Lake, Synapse, Power BI, Microsoft Purview)
• Plan and execute Databricks-to-Fabric migrations, including workload assessment, Delta/OneLake data movement, notebook and pipeline conversion, semantic model re-pointing, validation, and cutover where applicable
• Design and maintain logical and physical data models optimized for analytics, reporting, and downstream consumption
• Build semantic models and curated datasets to support Power BI and enterprise Business Intelligence use cases
• Partner with BI developers, analysts, and business stakeholders to translate reporting requirements into scalable data solutions
• Implement and manage CI/CD pipelines using Azure DevOps, Git for data engineering source control and automated deployments of data engineering workloads
• Enforce best practices for source control, automated testing, deployment, and environment management
• Ensure high availability, fault tolerance, and operational excellence for production data systems
• Implement data governance, security, and access controls, including role-based access and data classification
• Ensure compliance with enterprise data policies and regulatory requirements
• Collaborate with security and infrastructure teams to enforce secure data handling and platform standards
• Validate daily, weekly, and monthly data loads
• Investigate data mismatches
• Reconcile source-to-target records
• Validate business-critical reporting datasets
• Verify completeness, accuracy, and timeliness of data
• Participate in code reviews, design discussions, and team knowledge sharing
• Communicate clearly with business, analytics, and AI teams to enable data-driven decision making

Requirements:

• 3-6 years of professional experience in data engineering, data platform development, or analytics engineering
• Bachelor's degree in Computer Science, Engineering, Information Systems, or related field (or equivalent experience)
• Advanced working knowledge of SQL and hands-on experience with Python or Spark-based development
• Hands-on experience with data pipelines and ETL/ELT concepts, including data quality checks
• Microsoft Fabric (Lakehouse, Data Pipelines, OneLake, Semantic Models) or Azure Databricks (Spark, Delta Lake)
• Data modeling for analytics and BI
• Git and Azure DevOps (CI/CD, automated deployments)
• Business Intelligence platforms, especially Power BI
• Strong problem-solving skills and willingness to learn
• Applicants must be legally authorized to work in the United States at the time of hire
• This position is not eligible for employment sponsorship
• FIS core banking and H360 experience, including extracting, modeling, and reconciling data from FIS core systems and H360 for analytics, reporting, and payments use cases
• Payments experience in financial services, such as ACH, wire, card, or real-time payments data, Payment processors like TSYS, REPAY, FISERV etc; payment reconciliation and reporting; ISO 20022 and NACHA data formats
• Databricks expertise (Unity Catalog, Delta Live Tables / Lakeflow, Databricks Workflows)
• Databricks-to-Fabric migration experience or exposure, including notebook and pipeline conversion and Power BI semantic model cutover
• Experience in financial services, regulated industries, or enterprise environments
• Exposure to streaming data and real-time analytics
• Familiarity with data governance frameworks, metadata management, and data quality tooling
• Azure Data Engineer Associate (DP-203)
• Microsoft Fabric Analytics Engineer
• Databricks Data Engineer Associate