Data Engineer

SoftStandard Solutions is seeking an experienced Data Engineer to design, develop, and maintain scalable data pipelines and data platforms. The role focuses on ETL/ELT workflows, data warehousing, cloud-based data processing, data quality, governance, and collaboration with data and business teams.

Responsibilities:

• Design, develop, and maintain scalable ETL/ELT data pipelines
• Develop data ingestion and transformation workflows using Python, SQL, Spark, and cloud-native services
• Build and optimize data warehouses, data lakes, and data lakehouse solutions
• Perform data cleansing, transformation, validation, and quality checks
• Work with structured and semi-structured data from multiple sources
• Develop batch and near-real-time data processing pipelines
• Optimize queries, pipelines, and data processing jobs for performance and cost
• Implement data quality, governance, security, and monitoring processes
• Work closely with Data Scientists, Analysts, BI Developers, and business teams
• Integrate data from APIs, databases, files, and enterprise applications
• Implement CI/CD and version-control practices for data engineering workflows
• Troubleshoot pipeline failures and perform root-cause analysis
• Document data pipelines, architecture, data models, and technical processes

Requirements:

• 4–5+ years of professional experience in Data Engineering
• Strong hands-on experience with Python and advanced SQL
• Experience developing production-grade ETL/ELT pipelines
• Strong understanding of data warehousing, dimensional modeling, and data lakes
• Experience with at least one major cloud platform: AWS, Azure, or GCP
• Hands-on experience with Spark/PySpark and data processing
• Experience with Airflow, ADF, or another orchestration platform
• Understanding of CI/CD, Git, testing, monitoring, and data quality
• Bachelor's or Master's degree in Computer Science, Information Technology, Engineering, Data Science, or related field
• Apache Spark, Kafka, Hadoop — preferred
• dbt — preferred
• CloudWatch, Azure Monitor, Grafana — preferred
• Candidates with any of the mentioned certifications are preferred
• AWS Certified Data Engineer – Associate
• Microsoft Certified: Azure Data Engineer Associate (DP-203)
• Google Cloud Professional Data Engineer