Senior Data Engineer, AWS, Databricks

Role Overview

• Design and build production data pipelines across AWS, Snowflake, Databricks supporting both batch and near real-time analytics workloads.
• Establish reusable engineering patterns and frameworks
• Parameterized, modular, idempotent pipeline templates that other engineers adopt, reducing duplicated effort and inconsistent implementations.
• Drive down lead time from commit to production by removing manual steps, leveraging AI, building self-service tooling, and standardizing the path to deployment; treat cycle time as a metric you actively own and improve.
• Champion SDLC discipline covering version control, peer code review, automated testing, environment promotion, change management, and documentation.
• Integrate AI coding tools into daily workflow to accelerate scaffolding, refactoring, test generation, code optimization, and documentation, with measurable and demonstrable impact on throughput and quality.
• Measure and demonstrate impact , tying AI-tool adoption to concrete outcomes such as reduced lead time, faster test coverage, and improved consistency, and sharing those results to drive broader adoption.
• Evaluate emerging tooling and make pragmatic recommendations on what engineers should adopt, standardize on, or avoid.
• DataOps
• Blur the lines between data and software engineering practices.
• Employ CI/CD, automated testing, and apply trunk-based or short-lived branch development to data the same way it is to software.
• Modernize legacy workloads
• Help manage and optimize Ab>Initio pipelines and when applicable, help migrate or re-platform Ab>Initio pipelines toward cloud-native, declarative, ELT-based patterns on Snowflake and Databricks where it delivers value.
• Embed data quality, observability, and lineage into pipelines as a default, not an afterthought, automated data tests, freshness/quality SLAs, and traceable lineage.
• Optimize for cost and performance across Snowflake compute, Databricks clusters, and storage, applying FinOps-aware engineering practices.
• Mentor and upskill engineers through code review, pairing, design guidance, and documented standards, acting as a technical multiplier for the team.

Requirements

• Bachelor’s Degree in STEM related field or equivalent
• Ten years of related experience building, designing and operating production data pipelines at scale.
• Demonstrable experience architecting, designing and building scalable, secure data solutions using AWS, Databricks, Snowflake and Ab>Initio or similar platforms.
• A track record of leveraging AI assistants, create skills/tools to augment data engineering practices throughout the development lifecycle.
• The ability to lead technical direction for data engineering initiatives across cloud and on-premises infrastructure.
• Willingness to run mentoring sessions and offer technical guidance to the 20-person admin team.
• Ability to manage infrastructure deployment and optimize cloud resources.
• Drive to learn, identify and set technical standards and influence engineering practices and data governance policies.
• Ability to lead and take action even when there is no clear owner, inspire and motivate others, and be effective at influencing team members.

Tech Stack

• AWS
• Cloud
• SDLC

Benefits

• Health Insurance: Employees and their eligible family members – including spouses, domestic partners, and children – are eligible for coverage from the first day of employment.
• Retirement: Travelers matches your 401(k) contributions dollar-for-dollar up to your first 5% of eligible pay, subject to an annual maximum. If you have student loan debt, you can enroll in the Paying it Forward Savings Program. When you make a payment toward your student loan, Travelers will make an annual contribution into your 401(k) account. You are also eligible for a Pension Plan that is 100% funded by Travelers.
• Paid Time Off: Start your career at Travelers with a minimum of 20 days Paid Time Off annually, plus nine paid company Holidays.
• Wellness Program: The Travelers wellness program is comprised of tools, discounts and resources that empower you to achieve your wellness goals and caregiving needs. In addition, our mental health program provides access to free professional counseling services, health coaching and other resources to support your daily life needs.
• Volunteer Encouragement: We have a deep commitment to the communities we serve and encourage our employees to get involved. Travelers has a Matching Gift and Volunteer Rewards program that enables you to give back to the charity of your choice.