Key skills
AirflowApachePySparkPythonScalaSparkSQLTerraformData EngineeringData LakeApache Spark
About this role
Role Overview
- Design, implement, and sustain modern Data Lake and Lakehouse architectures
- Build and optimize both real-time and batch data ingestion pipelines
- Establish Data Quality standards and security/privacy policies
- Serve as a technical reference for the team and promote development best practices
- Identify bottlenecks in large-scale data processing
Requirements
- Minimum of 7 years of hands-on experience in Data Engineering
- Advanced experience with Apache Spark (PySpark/Scala) and workflow orchestrators (such as Airflow, Prefect, or Dagster)
- Proficiency in Python or Scala, plus strong SQL skills
- Knowledge of dimensional modeling techniques
- Experience with Terraform or similar infrastructure-as-code tools
Tech Stack
- Airflow
- Apache
- PySpark
- Python
- Scala
- Spark
- SQL
- Terraform
Benefits
- Health insurance
- DEI initiatives
- Work/life balance
- Professional development
- Wellness programs