Data Engineer

IDR, Inc. is seeking a Data Engineer to join a remote team. This role offers the opportunity to design and maintain scalable data pipelines that support analytics and reporting, helping organizations make data-driven decisions.

Responsibilities:

• Develop and optimize data pipelines to ensure reliable data flow across platforms
• Build and maintain scalable data storage solutions using modern formats like Delta Lake
• Implement data transformation processes to prepare data for analytics and reporting
• Support data governance, quality, and lineage tracking to ensure data integrity
• Collaborate with cross-functional teams to support reporting tools such as Power BI
• Maintain security protocols and access controls to safeguard sensitive data
• Stay current with emerging data engineering technologies and best practices

Requirements:

• Bachelor's degree in Computer Science or Engineering, or an associate's degree with two years of relevant experience, plus 3–5+ years working in data engineering
• Proven experience designing and managing large-scale data pipelines and platforms
• Strong skills in SQL and Python for data processing and automation
• Hands-on experience with ETL/ELT development and data ingestion frameworks
• Knowledge of modern data storage formats like Delta Lake and data architecture models such as Medallion (Bronze/Silver/Gold layers)
• Understanding of data transformation techniques, data lineage, and data quality principles
• Experience supporting reporting and analytics tools, preferably Power BI
• Familiarity with data security practices, including role-based access controls (RBAC)
• Experience with Microsoft Fabric
• Knowledge of data catalog and lineage tools such as Purview
• Experience with CI/CD tools like Azure DevOps or Git
• Exposure to real-time data streaming and ingestion