Ab Initio Data Engineer Lead

Through 1/5/2027

$69.50/hr w2

About this role:

In this role, candidates will:

Lead moderately complex initiatives within Technology and contribute to large scale data processing framework initiatives related to enterprise strategy deliverables
Build and maintain optimized and highly available data pipelines that facilitate deeper analysis and reporting
Review and analyze moderately complex business, operational or technical challenges that require an in-depth evaluation of variable factors
Oversee the data integration work, including developing a data model, maintaining a data warehouse and analytics environment, and writing scripts for data integration and analysis
Resolve moderately complex issues and lead teams to meet data engineering deliverables while leveraging solid understanding of data information policies, procedures and compliance requirements
Collaborate and consult with colleagues and managers to resolve data engineering issues and achieve strategic goals.

Required Qualifications:

6 plus years of Data Engineering experience, or equivalent demonstrated through one or a combination of the following: work experience, training, military experience, education
6 plus years of Strong Experience of Ab Initio, Spark Core, Spark SQL, Data Frames, Datasets
3 plus Experience on Python or other scripting languages
Hands-on experience with Google Cloud Platform or other cloud platforms (AWS/Azure)
3 plus years of data warehouse experience, SQL experience (Teradata, SQL, PL/SQL). Understand and write the complex SQL queries and have in-depth understanding of oracle database
Strong experience with Teradata, Hadoop, Hive, Autosys, Airflow, BigQuery SparkFlow
Good understanding of distributed system and parallel processing, spark architecture in depth
Ability to conduct code reviews with focus on testability and code coverage.
Should have executed end to end projects in data engineering
Should have experience working in an Agile environment using Scrum.
Fluent in communication

Desired Qualifications:

Prior experience of leading the teams and running the major programs is needed.
Good knowledge of non-relational DBMS platforms, (MongoDBj), real-time data streaming via Kafka
Working knowledge in developing end to end ML/AI pipeline in Apache Spark, Sparkling Waters, H2O, experience in working on LLMs. Exposer to Agentic AI frameworks.
Ability to work with Data Engineers in discovering and optimizing bottlenecks in the AI/ML pipeline for real-time or near-real-time applications that consumes large throughput of data
Perform various complex activities related to statistical/machine learning. Provide analytical support for developing, evaluating, implementing, monitoring and executing models across business verticals using emerging technologies including but not limited to Python, Agentic AI frameworks like ADK, Google Cloud Platform vertex AI and NLP.
Proficient in large scala data processing, Hands-on experience with different file formats, very good understanding of Unix command & Shell scripting.
Familiarity with CI/CD pipelines, infrastructure as code and cloud deployment best practices

Top Skills:

Role and Team:

Role will be working with the enterprise information warehouse.
They are looking for a specialized ETL backend developer to help with additional work they've taken on related to the migration to Google Cloud Platform
Part of the Data and Insights team, they build data platforms to deliver strategic information and insights to the business
Role will be analyzing new development requests, designing and developing new ETL processes, testing, and putting them into production

Ab Initio Data Engineer Lead

Key skills