Senior Data Engineer

i2e Consulting is seeking a Senior Data Engineer to join their team. The role involves designing and maintaining data pipelines, implementing architecture patterns, and optimizing data workflows for analytics-ready products.

Responsibilities:

• Design, develop, and maintain scalable data pipelines using Databricks, PySpark, Spark SQL, and Delta Lake
• Implement Lakehouse and Medallion architecture patterns for reliable, analytics-ready data products
• Build ETL/ELT pipelines integrating data from multiple sources such as APIs, databases, files, and streaming platforms
• Optimize Spark jobs, Delta tables, clusters, and workflows for performance, reliability, and cost efficiency
• Implement data quality, governance, access control, and lineage practices using Unity Catalog or equivalent tools
• Collaborate with architects, analysts, data scientists, and business teams; mentor junior engineers and promote engineering best practices

Requirements:

• 6+ years of data engineering experience, including strong hands-on experience with Databricks
• Strong proficiency in PySpark, Spark SQL, Python, SQL, and distributed data processing concepts
• Experience with Delta Lake, Databricks Workflows, Delta Live Tables, Unity Catalog, and Databricks SQL
• Good understanding of Lakehouse architecture, data warehousing, data modelling, and ETL/ELT design patterns
• Experience with cloud platforms such as Azure or AWS and orchestration tools such as ADF, Airflow, or dbt
• Familiarity with Git, CI/CD, testing, release management, and production support practices
• Strong communication, problem-solving, stakeholder management, and mentoring skills
• Experience in pharma, life sciences, healthcare, or regulated environments such as GxP, GDPR, or HIPAA
• Databricks Certified Data Engineer Associate or Professional certification
• Experience supporting analytics, AI/ML, or data product use cases on the Databricks platform