Data Engineer

Quantenex is seeking a Data Engineer to design, develop, and maintain scalable data platforms and pipelines. The role focuses on ETL/ELT workflows, data quality, storage and query optimization, cloud deployment, data modeling, production monitoring, and collaboration with analytics and machine learning teams.

Responsibilities:

• Design, develop, and maintain scalable data pipelines for batch and near-real-time processing
• Build and optimize ETL/ELT workflows to ingest data from multiple sources (databases, APIs, files, streaming systems)
• Work with structured and semi-structured data (relational data, JSON, logs, events)
• Ensure data quality, integrity, and reliability through validation, testing, and monitoring
• Optimize data storage and query performance in data warehouses and data lakes
• Collaborate with data scientists and analysts to support analytics and ML requirements
• Implement and maintain data models to support reporting and downstream applications
• Support deployment and operation of data pipelines on cloud platforms such as Azure, AWS, or GCP
• Apply best practices for data security, governance, and access control
• Use version control and CI/CD pipelines for data engineering workflows
• Monitor pipeline performance and troubleshoot data issues in production environments
• Document data pipelines, architectures, schemas, and operational procedures
• Participate in Agile ceremonies and collaborate effectively with cross-functional teams
• Contribute to continuous improvement of data platforms, tools, and processes
• Stay updated with emerging data engineering technologies and best practices

Requirements:

• 6+ Years
• Strong Python programming skills for data processing and pipeline development
• Proficiency in SQL and working with relational databases
• Experience with data integration and ETL/ELT tools or frameworks
• Familiarity with data warehouses and data lakes (e.g., Snowflake, BigQuery, Redshift, Delta Lake)
• Experience working with cloud platforms (Azure, AWS, or GCP)
• Understanding of data modeling concepts (dimensional modeling, star/snowflake schemas)
• Exposure to big data technologies such as Spark or similar frameworks
• Experience with workflow orchestration tools (e.g., Airflow, Prefect, Azure Data Factory)
• Knowledge of version control systems such as Git
• Understanding of basic DevOps and CI/CD concepts
• Strong problem-solving and analytical skills
• Good communication and documentation skills
• Ability to work independently and collaboratively in an Agile environment