Data Engineer

Arkhya Tech. Inc. is seeking a Data Engineer to design, build, and operationalize scalable data pipelines and products using a Python-based engineering approach. The role involves developing data ingestion and transformation pipelines, integrating data from various systems, and ensuring production reliability.

Responsibilities:

• Develop pipelines and transformations using Python (PySpark / notebooks / scripts)
• Work within VS Code + GitHub development workflows
• Build ingestion, transformation, and curated data layers aligned to medallion architecture
• Integrate data from ERP, MES, OT, historian, and operational systems
• Convert notebooks into production-grade, reusable components
• Implement unit testing, logging, monitoring, and observability frameworks
• Follow branching strategies, pull request processes, and CI/CD pipelines
• Package reusable logic into shared modules and libraries
• Enable creation of certified, semantic-ready data products
• Optimize performance, troubleshoot failures, and ensure production reliability
• Maintain technical documentation and operational runbooks

Requirements:

• Strong proficiency in Python and PySpark-based data engineering
• Experience with VS Code, GitHub, and code-based pipeline development
• Strong experience with Azure Data Factory, Synapse, Microsoft Fabric, SQL
• Understanding of Notebook vs. production pipeline design
• Understanding of code modularization and reuse
• Strong debugging, optimization, and problem-solving capabilities
• Manufacturing data experience
• Exposure to AI-ready datasets, feature engineering, and data observability