Prinicpal Data Engineer

WSS Associates is seeking a Principal Data Engineer to provide technical leadership across their global data engineering function. This role focuses on designing and evolving enterprise-scale data platforms that support analytics, AI, and machine learning, while mentoring engineers and driving best practices across multiple teams.

Responsibilities:

• Define and evolve data engineering standards, frameworks, and architectural patterns across the organisation
• Lead the design and implementation of enterprise-scale Lakehouse architectures and cloud-native data platforms
• Provide technical oversight for complex data engineering initiatives and large-scale transformation programmes
• Drive platform scalability, performance, security, governance, and operational excellence
• Evaluate emerging technologies and identify opportunities to improve platform capabilities and engineering productivity
• Establish best practices for metadata-driven, configuration-driven, and reusable engineering frameworks
• Design and build complex data pipelines, workflows, integrations, and data products
• Develop solutions using Databricks, Spark, SQL, Delta Lake, Unity Catalog, Lakeflow, and related technologies
• Lead troubleshooting and resolution of critical production issues
• Contribute to architecture reviews, code reviews, and technical design sessions
• Drive automation across CI/CD, testing, infrastructure provisioning, and deployment processes
• Build reusable data products that support AI, machine learning, and advanced analytics initiatives
• Partner with Data Science and AI teams to enable scalable model development and operationalisation
• Support implementation of end-to-end ML pipelines and modern MLOps practices
• Act as a senior technical mentor for engineers across the organisation
• Promote engineering excellence through coaching, pair programming, technical reviews, and knowledge sharing
• Support capability development and help establish career growth pathways for engineers
• Foster a culture of innovation, continuous improvement, and technical curiosity
• Partner with business, product, architecture, and technology teams to align engineering delivery with business objectives
• Translate complex business requirements into scalable technical solutions
• Communicate technical concepts, risks, trade-offs, and recommendations to both technical and non-technical audiences
• Influence decision-making across teams through expertise, credibility, and collaboration

Requirements:

• Extensive experience designing and delivering large-scale cloud-native data platforms
• Deep expertise with Databricks, including Spark, Delta Lake, Unity Catalog, Lakehouse architectures, Lakeflow, and modern data engineering tooling
• Strong experience within Microsoft Azure data environments
• Proven track record building scalable, secure, and highly available data pipelines and data products
• Strong understanding of modern data architecture principles including Lakehouse, DataOps, metadata-driven frameworks, and data governance
• Significant experience with CI/CD, Infrastructure as Code, automation, and engineering best practices
• Experience supporting AI, machine learning, and advanced analytics workloads
• Excellent stakeholder management and communication skills
• Proven ability to influence technical direction across multiple teams and programmes
• Experience supporting enterprise AI and Machine Learning initiatives
• Exposure to MLOps frameworks and operationalised machine learning environments
• Experience working across global teams and multiple geographies
• Experience delivering data solutions within commercial real estate, property technology, investment management, asset management, or similarly complex enterprise environments
• Experience leading technology transformation and cloud modernisation initiatives