Senior Data Engineer

IDR, Inc. is seeking a Senior Data Engineer to join one of their top clients for a remote contract opportunity. This role involves designing and managing critical data pipelines on cloud platforms to support healthcare data analytics and ensure reliable, scalable data operations.

Responsibilities:

• Develop and maintain end-to-end ETL/ELT pipelines on cloud platforms like GCP and Azure to support healthcare data needs
• Collaborate with stakeholders to gather requirements and translate them into efficient data solutions
• Model data structures for analytical purposes, including dimensional modeling and handling slowly-changing dimensions
• Ensure pipelines are optimized for performance, reliability, and security, adhering to healthcare compliance standards
• Lead technical direction for data projects, especially when requirements are unclear or evolving
• Use orchestration tools such as Airflow, Cloud Composer, or Prefect to automate workflows
• Drive continuous improvement by exploring new tools and techniques to enhance data processes

Requirements:

• Bachelor's degree in Computer Science, Information Technology, or a related field, or equivalent experience with at least 5 years in data engineering
• Over 5 years of practical experience working with SQL in a production environment
• At least 3 years of building and maintaining automated data pipelines using batch or streaming methods
• Proficiency in programming with SQL, Java, or Python for pipeline development
• Hands-on experience working with cloud platforms such as GCP, Azure, BigQuery, and SQL Server
• Deep understanding of designing scalable, high-performance ETL/ELT pipelines
• Familiarity with orchestration tools like Airflow, Cloud Composer, or Prefect
• Strong knowledge of data modeling techniques, including dimensional modeling and handling slowly-changing dimensions
• Working knowledge of HIPAA regulations and PHI data security practices
• Experience with version control systems like Git and CI/CD pipelines
• Ability to independently manage projects from requirements to deployment, prioritizing operational stability
• Skilled in translating business needs into technical solutions and communicating effectively with stakeholders
• Proactive in exploring new data tools and trends, with a focus on continuous learning