Azure Data Engineer(Data Lake + Databricks )

Scalence L.L.C. is seeking a Senior Azure Data Engineer to support the evolution and optimization of an enterprise Marketing Data Lake environment. The role involves data engineering, data quality, metadata management, and collaborating with stakeholders to meet data requirements.

Responsibilities:

• Assess existing data lake tables and create a structured data inventory
• Review and optimize data feeds, schemas, and storage structures
• Improve data performance, scalability, and documentation standards
• Collaborate with business and technical stakeholders to understand data requirements
• Identify, onboard, and integrate new marketing and non-marketing data sources
• Design, develop, and test new data ingestion pipelines
• Ensure data quality, consistency, and reliability across feeds
• Support stakeholder enablement and data adoption initiatives
• Replace manual data upload processes with automated pipelines
• Design and implement data quality frameworks, validations, and monitoring
• Perform detailed quality reviews and remediation activities
• Develop Proof of Concepts (PoCs) for automated quality controls
• Document data lineage, transformations, metadata, and business rules
• Develop data dictionaries, taxonomy standards, and PII handling frameworks
• Support ER model development and data governance initiatives
• Contribute to metadata management and artifact standardization
• Partner with cross-functional teams within the data community
• Support access management, platform onboarding, and user enablement
• Maintain data repositories and knowledge-sharing documentation

Requirements:

• 8+ years of Data Engineering experience
• Strong expertise in Azure Data Lake Storage (ADLS)
• Hands-on experience with Databricks and Azure Data Factory (ADF)
• Advanced SQL development and optimization skills
• Experience building modern ETL/ELT data pipelines
• Knowledge of data quality frameworks and monitoring solutions
• Experience with data lineage, metadata management, and governance
• Strong documentation and stakeholder communication skills
• Experience with SAP Data Lake, SAP Datasphere, BDC, or Collibra
• Exposure to marketing data platforms and large-scale data lake environments
• Knowledge of PII governance, taxonomy management, and data cataloging
• Experience with Azure cloud ecosystem and enterprise data architecture