Mid-level Data Engineer

Role Overview

• Participate in migrating processes and analytical solutions from SAS to Databricks.
• Develop and maintain data pipelines (ETL/ELT) using Python, PySpark, and Databricks.
• Analyze existing SAS code and processes, proposing alternatives and improvements for the new platform.
• Automate data ingestion, transformation, and delivery processes.
• Support the design of data solutions and data architecture.
• Collaborate with business stakeholders to understand requirements and validate results.
• Prepare and maintain technical and functional documentation for migrated solutions.
• Ensure data quality, performance, and reliability during and after the migration process.

Requirements

• Strong experience with Azure Data Services and Databricks.
• Knowledge of or relevant experience with SAS.
• Experience with Databricks and PySpark.
• Productionizing Machine Learning models on Databricks (MLOps).
• Proficiency in Python and SQL for data manipulation and transformation.
• Experience with CI/CD and version control using GitHub.
• Experience in data modeling (conceptual, logical, and physical).
• Experience in cloud migration projects.
• Experience in ETL/ELT and integrating large volumes of data.
• Experience deploying AI models and using tools such as Azure ML.
• Experience with AI projects and pipeline automation.

Tech Stack

• Azure
• Cloud
• ETL
• PySpark
• Python
• SQL

Benefits

• Health and dental insurance;
• Meal and food allowance;
• Childcare assistance;
• Extended parental leave;
• Partnerships with gyms and health and wellness professionals via Wellhub (Gympass) and TotalPass;
• Profit-sharing (PLR);
• Life insurance;
• Continuous learning platform (CI&T University);
• Employee discount club;
• Free online platform dedicated to promoting physical and mental health and wellbeing;
• Pregnancy and responsible parenting course;
• Partnerships with online course platforms;
• Language learning platform;
• And many more