Staff Software Engineer - Semantic Foundation

WEX is seeking a Staff Software Engineer to join its Data Platform Engineering Team. The role is responsible for designing, deploying, scaling, and maintaining multi-cloud data platforms, Apache Airflow orchestration, Kubernetes environments, and big data infrastructure. The engineer will also lead CI/CD automation, observability, data governance, cloud cost optimization, technical architecture, and mentorship.

Responsibilities:

• Design, deploy, scale, and maintain highly available Apache Airflow clusters (using Helm, Kubernetes, and Terraform) to support critical enterprise ETL/ELT workflows
• Drive DevOps practices using Terraform, Helm Charts, and ArgoCD to automate platform deployments, manage self-hosted GitHub runners, and enforce GitOps workflows
• Architect and manage robust CI/CD pipelines utilizing GitHub Actions for seamless deployment of data pipelines, infrastructure components, and DAGs
• Provision and manage scalable Kubernetes (EKS/AKS) clusters, Docker containers, and underlying cloud infrastructure across AWS (EC2, EMR, S3, VPC) and Azure (Synapse, ADLS)
• Build and maintain end-to-end monitoring, logging, and alerting systems using Grafana, Prometheus, Loki, and centralized log management solutions to ensure high platform uptime and reliability
• Build scalable data infrastructure supporting big data processing engines and data warehouses, including Apache Spark, AWS EMR, Snowflake, Azure Synapse, Apache Kafka, and dbt
• Deploy and maintain central data discovery and metadata tooling (e.g., DataHub) to facilitate data governance, schema management, and cataloging
• Actively monitor, audit, and optimize data infrastructure compute and storage costs across AWS and Azure (EC2, EMR, Snowflake queries, Kubernetes nodes)
• Build internal tools, CLI utilities, and dynamic workflow templates to improve developer productivity for data engineers, analytics engineers, and data scientists
• Take full technical ownership of data platform modules from architectural design through deployment, production operations, and incident management
• Define and enforce high engineering standards for code quality, design patterns, testing, data lineage, and security (access controls, IAM, dynamic schema management)
• Partner with data leads, product managers, and business stakeholders to identify infrastructure gaps, define a 1–2 year data platform roadmap, and prioritize platform initiatives
• Serve as a subject matter expert (SME) on data infrastructure, guiding and mentoring junior and mid-level data platform engineers

Requirements:

• This is a remote position; however, the candidate must reside within 30 miles of one of the following locations: Portland, ME; Boston, MA; Chicago, IL; Dallas, TX; San Francisco Bay Area, CA; and Seattle/WA
• • 6+ years of hands-on professional experience in Data Platform Engineering, DevOps, or Site Reliability Engineering (SRE) supporting big data environments
• • Airflow Subject Matter Expertise: Deep production experience managing, tuning, dynamic scaling, and troubleshooting Apache Airflow infrastructure (Celery/Kubernetes Executors, DAG parsing performance, dynamic configurations)
• • Container & Infrastructure Automation: Expert-level skills in Kubernetes, Helm, Terraform, Docker, ArgoCD, and GitHub Actions (including custom runner configurations)
• • Observability & Monitoring: Proven track record of configuring production alerting, metrics collection, and log aggregation using Grafana, Prometheus, and Loki
• • Big Data & Analytics Tech Stack: Deep operational and configuration experience with Spark, AWS EMR, Snowflake, Azure Synapse, dbt, and real-time streaming via Apache Kafka
• • Cloud & FinOps: Solid hands-on experience with AWS (EC2, S3, VPC, IAM, EKS) and/or Azure, with a demonstrated history of driving cloud cost optimization
• • Governance Tooling: Experience deploying or managing data cataloging tools like DataHub, Amundsen, or similar metadata management platforms
• • Programming & Scripting: Strong programming skills in Python, Bash, Go, or SQL
• • High Ownership: Comfortable taking complex architectural requirements from concept to production-grade deployment in a high-paced environment
• • Automation-First Philosophy: Driven to replace manual operational tasks with code, automated tests, automated CI/CD checks, and resilient self-healing infrastructure