Key Responsibilities
Design and implement data pipelines using Azure Data Factory for ETL processes.
Develop and optimize data flows for batch and real-time data integration.
Integrate ADF with Azure services such as Azure Synapse, Data Lake, and Blob Storage.
Monitor, troubleshoot, and enhance ADF pipelines for performance and reliability.
Ensure data security and compliance with organizational standards.
Collaborate with data engineers, architects, and business stakeholders to deliver solutions.
Document processes and provide knowledge transfer to team members.
Required Skills
Expertise in Azure Data Factory (ADF) pipeline and data flow development.
Strong understanding of ETL concepts and data integration patterns.
Experience with Azure services: Data Lake, Blob Storage, Synapse Analytics.
Proficiency in SQL and data transformation techniques.
Knowledge of Azure security and role-based access control.
Hands-on experience with monitoring and troubleshooting ADF pipelines.
Good to Have
Experience with Azure DevOps for CI/CD of ADF pipelines.
Knowledge of Power BI or other visualization tools.
Familiarity with Python or .NET for custom activities in ADF.
Understanding of big data technologies and data warehousing concepts.
