Data Engineer

ProMach is a packaging performance and automation company. The Data Engineer will design, build, and maintain data flows between corporate IT systems and the company’s data lake using Databricks, Apache Spark, and Delta Lake. The role focuses on data ingestion, transformation, cloud integration, data quality, CI/CD automation, and collaboration with analysts to deliver data-driven insights.

Responsibilities:

• Data Pipeline Development: Design, build, and maintain scalable data pipelines using Databricks, Apache Spark, and Delta Lake
• ETL/ELT Processes: Develop and manage robust ETL/ELT workflows for data extraction, transformation, and loading from diverse source systems
• Monitor and resolve any issues with scheduled jobs that move data
• Data Architecture and Modeling: Contribute to designing and implementing data models and architecture for data lakes and data warehouses
• Provide guidance on architecture and platforms that facilitate reporting and analytics needs
• Cloud Integration: Integrate data solutions with cloud platforms (Azure, AWS, or GCP) and related services
• Data Quality and Governance: Implement data quality checks, monitoring, and data governance best practices to ensure consistency, accuracy, and reliability
• Automation: Implement and maintain CI/CD pipelines to automate deployment and streamline workflows
• Collaboration: Work closely with business analysts and stakeholders to define data requirements and deliver actionable insights

Requirements:

• Proven experience in data engineering and cloud-based development
• 3-5 Years' experience in a similar role
• Expertise in Databricks and the broader Databricks ecosystem (Databricks SQL, Delta Lake, Databricks Runtime)
• Proficiency in SQL, Python, and Apache Spark
• Experience with cloud platforms such as Azure, AWS, or GCP
• Strong understanding of data modeling, ETL/ELT, and data warehousing concepts
• Experience with CI/CD tools and practices
• Excellent problem-solving, analytical, and communication skills
• Eager to learn new skills and step in to deliver where needed
• Bachelor's Degree in Information Systems, Computer Science, or related field, or equivalent work experience
• Experience with ERP systems (SAP, Infor) and Salesforce CRM
• Familiarity with BI tools such as Tableau
• Manufacturing industry experience
• Motivated, committed and energetic self-starter dedicated to providing high quality and responsive IT service