Data Engineer

Role Overview

Data Pipeline Development:

• Design and build ETL pipelines using Microsoft Fabric (Dataflow Gen2, Notebooks, or equivalent tools)
• Write optimized SQL queries and transformations for data ingestion from designated source systems
• Apply data quality rules and validation logic at each pipeline stage
• Implement incremental loads and manage refresh schedules for performance
• Escalate to Lead for architectural decisions or complex transformation patterns Data Quality & Validation:
• Define and implement data quality checks at ingestion, transformation, and output stages
• Perform ongoing data validation to ensure pipeline outputs align with business logic and source system expectations
• Identify, document, and escalate data quality issues with root cause analysis
• Maintain data quality dashboards and SLA monitoring
• Support UAT for new data sources or transformation logic Transformation & Modeling:
• Build and maintain data transformations using Power Query, SQL, or Python as appropriate
• Develop dimensional models and define aggregation logic aligned with analytics requirements
• Optimize data structures for performance and maintainability
• Document transformation logic, lineage, and assumptions per team standards
• Collaborate with Lead to define semantic Operational Support:
• Troubleshoot pipeline failures and performance issues; coordinate resolution with IT/Engineering
• Respond to data discrepancy reports from business users and analysts
• Maintain documentation of data sources, data dictionaries, and transformation specifications
• Support capacity planning and optimization of Fabric environments and pipelines models and calculated metrics

Requirements

Technical

• Advanced SQL
• query optimization, window functions, performance tuning, debugging complex transformations
• Proficient with Microsoft Fabric
• (Dataflow Gen2, Notebooks, Lakehouse) OR equivalent ETL tools (Python, dbt, Talend, Informatica)
• Strong understanding of relational database design and dimensional modeling
• Power Query / M
• complex data shaping, merging, error handling, and transformation logic
• Python or similar scripting language
• data manipulation, pipeline automation
• Git/version control basics
• able to collaborate on code and track changes
• Data quality and testing frameworks
• unit tests, assertions, validation rules Non-Technical
• Ability to interpret business requirements and design efficient data solutions
• Data governance mindset
• understands data lineage, documentation, and quality standards
• Proactive about identifying edge cases and potential data issues
• Mortgage/lending domain familiarity preferred; willingness to learn domain required
• Works effectively within defined standards and escalates architectural questions to Lead
• Able to balance speed with quality; advocates for technical excellence

Tech Stack

• ETL
• Informatica
• Python
• SQL