Senior Data Engineer

Thoughtworks is a technology consultancy that helps clients solve complex business problems through technology. The Senior Data Engineer builds, maintains, tests, and operates modern data architecture, processing pipelines, data models, and data-centric applications while collaborating with data scientists and stakeholders.

Responsibilities:

• You will develop and operate modern data architecture approaches to meet key business objectives and provide end-to-end data solutions
• You will develop intricate data processing pipelines, addressing clients' most challenging problems
• You will collaborate with data scientists to design scalable implementations of their models
• You will write clean, iterative code using TDD and leverage various continuous delivery practices to deploy, support and operate data pipelines
• You will use different distributed storage and computing technologies from the plethora of options available
• You will develop data models by selecting from a variety of modeling techniques and implementing the chosen data model using the appropriate technology stack
• You will collaborate with the team on the areas of data governance, data security and data privacy
• You will incorporate data quality into your day-to-day work

Requirements:

• Working with data excites you: you can build and operate data pipelines, and maintain data storage, all within distributed systems
• You have hands-on experience of data modeling and modern data engineering tools and platforms
• You have experience in writing clean, high-quality code using the preferred programming language
• You have built and deployed large-scale data pipelines and data-centric applications using any of the distributed storage platforms and distributed processing platforms in a production setting
• You have experience with data visualization techniques and can communicate the insights as per the audience
• You have experience with data-driven approaches and can apply data security and privacy strategy to solve business problems
• You have experience with different types of databases (i.e.: SQL, NoSQL, data lake, data schemas, etc.)
• Experience with Python, SQL, ETL/Data Pipelines, Databricks, modern Data Warehousing/Data lakes/Delta lakes (Snowflake, Redshift, BigQuery)
• Experience with one or more major cloud platforms (AWS, Azure or GCP) and Machine Learning/AI/Gen AI/LLM
• You understand the importance of stakeholder management and can easily liaise between clients and other key stakeholders throughout projects, ensuring buy-in and gaining trust along the way
• You are resilient in ambiguous situations and can adapt your role to approach challenges from multiple perspectives
• You don't shy away from risks or conflicts, instead you take them on and skillfully manage them
• You are eager to coach, mentor and motivate others and you aspire to influence teammates to take positive action and accountability for their work
• You enjoy influencing others and always advocate for technical excellence while being open to change when needed
• Applicants for employment in the US must have work authorization that does not now or in the future require sponsorship of a visa for employment in the United States