Senior Data Pipeline Engineer/Developer

Gene by Gene is dedicated to advancing the field of genomics through innovation and research. They are seeking a Senior Data Pipeline Engineer/Developer to design, build, and own data pipelines that facilitate genomic and operational data flow, ensuring data quality and compliance in a regulated environment.

Responsibilities:

• Builds and operates production ETL and ELT for high-volume genomic data as well as operational and business data
• Integrates data across LIMS, lab instruments, internal applications, and third-party sources through robust, well-tested interfaces
• Develops across the stack in Python (data services, internal APIs, and supporting application code) and provides architectural and code-level guidance to engineering teams on data-layer integration
• Sets technical direction for batch and streaming data pipelines by evaluating frameworks, orchestration, storage, and processing patterns, making recommendations, and leading adoption
• Models and tunes data stores (Microsoft SQL Server and PostgreSQL, plus a cloud warehouse or lake) for performance and scale
• Defines and enforces engineering standards for testing, CI/CD, infrastructure as code, code review, and architecture decision records
• Owns data quality, lineage, and observability, including freshness, completeness, schema-drift detection, cost-per-job, and SLAs
• Builds pipelines for reproducibility and audit-readiness, incorporating versioned data and code, lineage, decision logging, access controls, and evidence collection
• Partners with security and compliance on data privacy, PII and PHI handling, and regulatory requirements across the data lifecycle
• Apply deep understanding of CAP/CLIA, HIPAA, GDPR, and GxP regulations specifically to data pipeline architecture
• Oversee protected health information (PHI) handling, data lineage, retention, and comprehensive audit logging
• Produce and maintain the critical operational evidence required to carry the data platform successfully through compliance audits
• Adhere to strict data-governance controls necessary for securely handling sensitive genomic data across global regions
• Enforce least-privilege access and ensure zero data egress to personal or unapproved infrastructure
• Utilize exclusively de-identified or synthetic data within development environments
• Maintain strict compliance with international data-residency and localization requirements

Requirements:

• Strong SQL proficiency on Microsoft SQL Server and PostgreSQL, including schema design, query tuning, and performance troubleshooting
• Strong Python proficiency across the stack (data pipelines, backend services, and APIs). This is a Python-first role
• Production experience with a workflow orchestrator (Argo Workflows, Step Functions, Prefect, Dagster, Airflow, or similar)
• Production experience with containers (Docker) and Kubernetes, which the lead orchestrator (Argo Workflows) runs on
• Strong testing discipline, including unit, integration, and data-quality or contract tests
• Excellent written and verbal communication, including the ability to explain data tradeoffs to technical and compliance stakeholders
• Genomics or NGS data formats and handling (FASTQ, BAM/CRAM, VCF)
• Bioinformatics workflow engines (Nextflow, WDL/Cromwell, or Snakemake)
• AWS data stack proficiency (S3, Glue, EMR, Batch, Lambda, Redshift) and/or AWS HealthOmics
• Distributed processing and streaming frameworks (Spark, Kafka, Kinesis)
• Data warehouse and transformation tooling (Snowflake, dbt)
• Infrastructure as code practices (Terraform, CDK, CloudFormation, or Pulumi)
• Familiarity with .NET (C#) and/or C++ as supporting languages for integrating with existing services
• Knowledge of LIMS integration and on-premises plus hybrid data architectures
• 8+ years of professional software or data engineering experience
• 4+ years building and operating production data pipelines at scale
• 3+ years of production cloud experience (AWS preferred)
• Demonstrable experience working in regulated environments. Compliance is a hard requirement for this role
• Experience supporting HIPAA or GxP audits
• Bachelor's degree in Computer Science, Bioinformatics, or a related field, or equivalent professional experience
• An advanced degree in a quantitative field is preferred