Kellton is looking for a hands-on Senior Data Engineer with expertise in developing data ingestion pipelines. This role is crucial in designing, building, and maintaining data infrastructure, focusing on scalable pipelines and ensuring data integrity.
Responsibilities:
- Design, build, test, and implement scalable data pipelines using Python and SQL
- Maintain and optimize our Snowflake data warehouse’s performance, including data ingestion and query optimization
- Extract data from APIs using Python and AWS Lambda and automate workflows with AWS Airflow
- Perform analysis and critical thinking to troubleshoot data-related issues and implement checks/scripts to enhance data quality
- Collaborate with other data engineers and architect to develop new pipelines and/or optimize existing ones
- Maintain code via CI/CD processes as defined in our Azure DevOps platform
Requirements:
- 7+ years of experience in Data Engineering roles, with a focus on building and implementing scalable data pipelines for data ingestion
- Expertise in Snowflake, including data ingestion and performance optimization
- Strong SQL skills for writing efficient queries and optimizing existing ones
- Proficiency in Python for data extraction from APIs using AWS Lambda, Glue, etc
- Experience with AWS services such as Lambda, Airflow, Glue, S3, SNS, etc
- Highly self-motivated and detail-oriented with strong communication skills
- Familiarity with ETL/ELT processes
- Experience with Fivetran and DBT is a plus