Senior Data Engineer

Role Overview

• The Senior Data Engineer designs, builds and maintains the data ingestion and transformation pipelines that populate Inchcape’s central data platforms.
• This role ingests raw data from diverse systems, curates the bronze and silver layers of the medallion architecture, ensures data quality and performance, and delivers reliable, reusable datasets for modelling, analysis, reporting and BI.
• Develop and maintain ETL/ELT pipelines to ingest data from ERP, CRM, operational, financial, and third‑party sources into the Microsoft Fabric Platform
• Lakehouse (OneLake) using Delta Lake/Parquet.
• Implement data quality standards and controls, incremental loading, CDC and error handling to ensure reliability and correctness.
• Create bronze tables, curated Silver and gold tables by transforming, cleansing and conforming raw data, document lineage and transformations.
• Optimise performance through partitioning, clustering and tuning of MS Fabric, SQL and Spark workloads, monitor pipeline health and costs.
• Collaborate with Data Architects, Modellers and Data BAs to design schemas, naming conventions and integration patterns to ensure business self-service enablement through Gold data.
• Build reusable components, templates and automation scripts using Fabric Dataflows Gen2, Spark notebooks or equivalent tools.
• Maintain metadata and sensitivity labels via Purview, ensure proper access controls and compliance.
• Research and evaluate new features and patterns in Microsoft Fabric and make recommendations for adoption, enabling an AI-driven data strategy.
• Ensure NFR requirements like resilience, availability and security standards are applied as per agreed NFR metrics.

Requirements

• 8+ years in data engineering or data integration roles, working with SQL, Python and or Scala.
• Experience of ETL/ELT patterns, CDC, incremental loading and data warehousing principles using Microsoft Fabric.
• Practical experience with Delta Lake, Lakehouse/Medallion architecture and cloud data platforms (Azure and AWS).
• Proficient in building pipelines with tools like MS Fabric Dataflows Gen2, Spark/SQL notebooks or equivalent.
• Experienced in data quality frameworks and monitoring/observability tools.
• Ability to collaborate across teams and communicate technical concepts to non‑engineers.
• Experience with streaming data (Kafka, Event Hubs) or real-time ingestion.
• Knowledge of master data management and data governance practices.
• Familiarity with version control and CI/CD for data pipelines.
• Experience in, and assisted in, the implementation of governance, controls, and standards.
• Any Shipping or Maritime experience Big Data Ecosystem prior experience is a plus.
• Bachelor’s degree in computer science, engineering or related discipline.
• Certification in Azure Data Engineering is a MUST.
• Certifications in AWS cloud data platform or other data platform.

Tech Stack

• AWS
• Azure
• Cloud
• ERP
• ETL
• Kafka
• Python
• Scala
• Spark
• SQL

Benefits

• Working in a diverse society, we recognize that our customers, colleagues, and contractors are central to our success.