Own the full life cycle of understanding the business requirements data and designing the migration of our data from several systems to a cutting-edge Databricks environment.
Architect, implement, and optimize ETL/ELT processes to ensure robust, secure, and analytics-ready data flows for our business operations.
Collaborate with stakeholders, including data analysts, data scientists, architects, and business teams, to ensure data solutions meet organizational requirements and compliance standards.
Design, develop, and maintain data pipelines to transfer data from various databases to Databricks, leveraging both batch and real-time methodologies as appropriate.
Lead the overall ETL/ELT architecture, including extraction, transformation, and data loading strategies, ensuring scalability, performance, and data integrity.
Optimize and tune ETL/ELT processes for large-scale and real-time data movement, ensuring resource efficiency.
Ensure data security, quality, and compliance across the whole process.
Model, integrate, and optimize data structures in the Databricks platform, taking into account data lakes, data warehousing needs, and analytics workloads.
Automate and monitor data workflows for reliability, transparency, and fault tolerance throughout the migration and beyond.
Document data flows, transformation logic, and technical specifications for all data integration processes, enabling maintainability and smooth knowledge transfer.
Requirements
Bachelor's Degree in Computer Science, Data Analytics, Engineering, Operations Management or related field
5+ years of experience in data engineering or related roles, with demonstrated expertise in database migrations and ETL/ELT process design.
Deep hands-on experience with databases Oracle, SQL Server, MySQL, Teradata, Snowflake
In-depth knowledge of Databricks (including PySpark, Databricks SQL, and Lakehouse architectures).
Expertise in ETL/ELT tools and frameworks (e.g., Informatica, Talend, Databricks Workflows, or custom pipelines).
Proven ability to design and implement scalable, efficient data pipelines and to optimize complex transformations in modern cloud environments.
Demonstrated experience in data modeling and integration for analytics and BI use cases.
Excellent communication and documentation skills for cross-functional collaboration and process transparency.
Tech Stack
Cloud
ETL
Informatica
MySQL
Oracle
PySpark
SQL
Benefits
From day one, we're looking out for your well-being–at work and at home–so you can focus on realizing your ambitions.