Senior Data Engineer

Tredence Inc. is a data analytics company, and they are seeking a Senior Data Engineer to manage the end-to-end delivery of data solutions. The role involves creating and maintaining data pipelines, optimizing data processes, and ensuring data quality while collaborating with cross-functional teams.

Responsibilities:

• Manage end to end delivery by Investigating problem areas, working cross-functionally with product manager & other stakeholders
• Follow the Agile development methodology; think strategically and execute methodically
• Develop and manage capacity and growth projection forecasts of the environment within budgets
• Create and maintain optimal data pipeline architecture, assemble large, complex data sets that meet functional / non-functional business requirements
• Drive optimization, testing and tooling to improve quality of solutions
• Manage teams that build and operate high volume distributed systems in a SaaS environment
• Great at devising efficient processes that increase velocity and quality
• Understanding business use cases and deriving codes as per requirements
• Promote the data discovery, availability and quality capabilities and expertise to business unit leaders and educate them in leveraging these capabilities in achieving their business goals
• Accountable for operational effectiveness in performance, uptime, release management of the enterprise data platform
• Responsible for creating and validating end-to-end scalable data pipeline templates
• Responsible on the migrating the streaming data from the S3 to Databricks layers (raw, processed, consumption) and applying required data quality checks and transformations
• Handle slowly changing dimensions, change data capture, dimensions, facts and manage upsets
• Responsible for handling adhoc data analysis requests
• Responsible for handling client communication
• Responsible for the data accuracy, data governance, file handling, error handling, data validation

Requirements:

• Master's degree (or foreign equivalent) in Business Analytics, Information Systems, Data Science, or a related field
• One (1) year of experience in the job offered or a related position
• Demonstrable knowledge of Data Engineering Pipelines
• Demonstrable knowledge of Data Warehousing
• Demonstrable knowledge of Data Modeling
• Demonstrable knowledge of SQL
• Demonstrable knowledge of Cloud Technology (Google Cloud)
• Demonstrable knowledge of Data Wrangling
• Demonstrable knowledge of Data Analysis
• Ability to travel to unanticipated client locations throughout the U.S., approximately 30%, as required