Key skills
AWS S3AWS GlueAWS RedshiftAWS LambdaAWS EMRPythonSQLData Pipeline DevelopmentDistributed Data SystemsData ModelingData WarehousingLakehouse ArchitectureKafkaAWS KinesisScalaApache SparkAWS SageMakerStep FunctionsApache AirflowAIMLETLELTData EngineeringData LakeRedshiftSparkAirflowAWSLambdaS3IAMGlueAthenaSageMakerBedrockKinesis
About this role
Harnham is a high-growth technology organization building a cutting-edge AI-driven platform. They are seeking a Staff Data Engineer to architect and scale their data infrastructure, focusing on designing modern data pipelines and enabling AI/ML capabilities in a cloud-first environment.
Responsibilities:
- Design, build, and scale robust data pipelines and data platforms in AWS
- Develop ETL/ELT workflows using tools such as Glue, Lambda, and EMR/Spark
- Architect and optimise data lake and warehouse solutions leveraging S3, Redshift, and Athena
- Implement real-time and batch data processing using Kafka, Kinesis, or MSK
- Collaborate with data science teams to support AI/ML model development and deployment (SageMaker)
- Build and manage orchestration workflows using Step Functions or Airflow (MWAA)
- Ensure data quality, governance, and security across systems (IAM, Lake Formation, KMS)
- Mentor engineers and drive best practices in scalable data architecture
Requirements:
- 7 - 10 years of experience in Data Engineering
- Strong AWS expertise including S3, Glue, Redshift, Lambda, and EMR (Spark)
- Advanced proficiency in Python and SQL
- Experience building scalable data pipelines and distributed data systems
- Strong understanding of data modelling, warehousing, and lakehouse architecture
- Experience with Kafka or AWS streaming tools (Kinesis, MSK)
- Exposure to Scala and/or Spark-based processing
- Experience with AI/ML platforms and tools such as SageMaker or Bedrock
- Familiarity with workflow orchestration tools (Step Functions, Airflow)