Data Engineer

Anteriad is a B2B solution provider focused on data-driven marketing solutions. They are seeking a Data Engineer to play a vital role in managing and optimizing data pipelines, ensuring data integration solutions meet the needs of their clients and the organization.

Responsibilities:

• Partner with technical and non-technical stakeholders to translate business requirements into scalable, reliable, and maintainable data engineering solutions
• Design, develop, and maintain data integration solutions using Microsoft SQL Server, Azure Data Factory, Azure Databricks, Microsoft Fabric, and related Azure services
• Build, manage, and optimize ETL/ELT pipelines using Azure Data Factory and Microsoft Fabric Pipelines, including pipeline orchestration, parameterization, scheduling, monitoring, and error handling
• Develop and maintain data transformation logic using Azure Databricks notebooks and Microsoft Fabric notebooks, with a focus on performance, scalability, reusability, and operational reliability
• Design and implement data ingestion, transformation, validation, and delivery processes across structured, semi-structured, and external data sources
• Develop and optimize data integration processes utilizing Azure Data Factory, Azure Databricks, Azure Storage Accounts, Azure Data Lake Storage, Azure Key Vault, Microsoft Fabric Lakehouses, Warehouses, Pipelines, and Notebooks
• Build and maintain SQL-based data models, stored procedures, views, automation scripts, and database objects to support operational and analytical workloads
• Support the modernization and migration of legacy data integration processes from SSIS and on-premises platforms to cloud-based Azure and Microsoft Fabric architectures
• Design and implement reusable pipeline patterns, data quality checks, validation controls, logging, alerting, and performance monitoring processes
• Support new client onboarding efforts by designing and implementing reliable data ingestion, transformation, reconciliation, and validation workflows
• Troubleshoot and resolve data pipeline, data quality, performance, and integration issues in collaboration with teams across Intelligence & Analytics
• Design, develop, and maintain datasets, semantic models, and data structures that support reporting, analytics, and visualization solutions across Power BI, Excel, Microsoft Fabric, and other business intelligence platforms
• Contribute to data engineering standards, documentation, best practices, and reusable frameworks that improve scalability, maintainability, and operational efficiency
• Research, evaluate, and recommend emerging technologies and platform capabilities that improve data architecture, automation, performance, and reliability

Requirements:

• 3+ years of experience in data engineering, data integration, business intelligence engineering, or a related technical role
• 3+ years of experience writing, optimizing, and troubleshooting SQL, including T-SQL, stored procedures, views, indexing strategies, and query performance tuning
• Hands-on experience designing, building, and supporting ETL/ELT pipelines using Azure Data Factory and/or Microsoft Fabric Pipelines
• Hands-on experience developing data processing and transformation logic using Azure Databricks notebooks, Microsoft Fabric notebooks, PySpark, Spark SQL, Python, or similar technologies
• Strong Microsoft SQL Server and relational database design principles skills
• Experience with Azure data platform services such as Azure Data Factory, Azure Databricks, Azure Storage Accounts, Azure Data Lake Storage, Azure Key Vault, and related cloud data services
• Strong knowledge and in depth work with Microsoft Fabric components such as Pipelines, Notebooks, Lakehouses, Warehouses, Dataflows, and semantic models
• Experience building reliable data ingestion and transformation processes from APIs, flat files, databases, web services, cloud storage, and other external data sources
• Hands on experience implementing data validation, reconciliation, monitoring, logging, and alerting processes for production data pipelines
• Experience supporting production data workloads, including troubleshooting failures, optimizing performance, and improving pipeline reliability
• Bachelor's degree in Computer Science, Information Technology, Data Analytics, a related field, or equivalent professional experience
• Strong analytical, problem-solving, and critical thinking skills
• Excellent verbal and written communication skills, with the ability to explain technical concepts to both technical and non-technical audiences
• Ability to work independently on moderately complex data engineering tasks while collaborating effectively in a team-oriented environment
• Prior experience modernizing or migrating legacy ETL processes from SSIS, SQL Server, or on-premises environments to Azure or Microsoft Fabric is strongly preferred
• Power BI, semantic models, or other data visualization and analytics tools experience is a plus
• Experience with Customer Data Platforms, CDPs, database marketing, marketing analytics, or client data onboarding environments is a plus
• Working knowledge with AWS or Google Cloud Platform is a plus