Key skills
PythonScalaSQLMachine LearningMLETLData EngineeringSnowflakeDatabricksSparkApache SparkHadoopAWSAzureGCP
About this role
ImpetusIT is seeking a highly skilled Data Engineer for one of their staffing partners who is hiring for a financial client. The role involves designing, developing, and optimizing scalable data pipelines and infrastructure for ML-driven applications.
Responsibilities:
- Design and build scalable data pipelines to support machine learning workflows
- Develop, optimize, and maintain ETL processes for structured and unstructured data
- Work with big data frameworks (Apache Spark, Hadoop, Databricks) to process large datasets
- Optimize data storage, retrieval, and performance for machine learning applications
- Ensure data integrity, security, and compliance with industry standards
- Automate data processing workflows using Python, SQL, or Scala
Requirements:
- Experience of 2-3 years
- Design and build scalable data pipelines to support machine learning workflows
- Develop, optimize, and maintain ETL processes for structured and unstructured data
- Work with big data frameworks (Apache Spark, Hadoop, Databricks) to process large datasets
- Optimize data storage, retrieval, and performance for machine learning applications
- Ensure data integrity, security, and compliance with industry standards
- Automate data processing workflows using Python, SQL, or Scala
- Strong experience in data engineering
- Hands-on experience with big data technologies (Apache Spark, Hadoop, Snowflake)
- Knowledge of cloud platforms (AWS, Azure, GCP) for ML model deployment