Senior Data Engineer

Alpaca is a global provider of agent-first brokerage infrastructure and financial services APIs. The company is seeking a Senior Data Engineer to design, build, and scale its data platform infrastructure, including ingestion, lakehouse, BI, monitoring, and data governance capabilities.

Responsibilities:

• Design, build, and evolve the core data platform infrastructure e.g., distributed query engines, orchestration, warehousing, cataloging, and more
• Own our lakehouse infrastructure as code, managing deployments through Terraform and Ansible on Kubernetes
• Build and maintain low-latency streaming and CDC ingestion pipelines, as well as batch ingestion paths landing in Iceberg
• Develop and scale our BI landscape so downstream teams and agents get performant, self-serve access to lakehouse data
• Enforce platform reliability best practices, including monitoring and alerting, on-call rotations, incident response, maintenance windows, runbooks, and SLAs
• Partner with DevOps, Analytics Engineering, and other stakeholders to close infrastructure gaps and support new data requirements

Requirements:

• 5+ years of experience in Data Engineering, including 2+ years building and operating scalable, low-latency data platforms handling > 100M events/day
• Strong hands-on experience running data infrastructure on Kubernetes, with cloud-native tooling like Docker and Helm
• Production experience with IaC: Terraform, Ansible, and ArgoCD (or equivalents)
• Deep knowledge of distributed systems (storage, transactions, and query processing) with hands-on experience operating open-source query engines like Trino or Presto
• Strong experience with object storage and open table formats, specifically Apache Iceberg
• Experience with streaming and CDC systems: Kafka, Redpanda, and Debezium
• Hands-on experience with orchestration frameworks (Airflow) and ELT tools (Airbyte)
• Strong working knowledge of Python and SQL for building pipelines and platform tooling
• Experience with Google Cloud Platform and its data services (GCS, Cloud Build, Cloud SQL, Dataproc, etc); or related experience with other cloud services
• Ability to thrive in a fast-paced startup environment and adapt infrastructure to rapidly changing needs
• Experience with semantic/metrics layers (Cube, dbt, Looker)
• Familiarity with transformation frameworks (dbt)
• Familiarity with reverse ETL tooling (Hightouch)
• Familiarity with data catalog and lineage tooling (OpenMetadata, Datahub)
• Experience with data access control and governance frameworks (Apache Ranger)