Data Engineer II

NPS Prism is a market-leading, cloud-based CX benchmarking and operational improvement platform owned by Bain & Company. We are seeking a highly skilled and experienced Data Engineer to design and implement robust data solutions for scalable business needs, utilizing cloud platforms and big data technologies.

Responsibilities:

• Design, build, and optimize ETL/ELT workflows using tools like Databricks, SQL, Python/pyspark & Alteryx (Good to have)
• Develop and maintain robust, scalable, and efficient data pipelines for processing large datasets. from source to emerging data
• Work on cloud platforms (Azure, AWS) to build and manage data lakes, data warehouses, and scalable data architectures
• Utilize cloud services like Azure Data Factory, AWS Glue, or for data processing and orchestration
• Use Databricks for big data processing, analytics, and real-time data processing
• Leverage Apache Spark for distributed computing and handling complex data transformations
• Create and manage SQL-based data solutions, ensuring high availability, scalability, and performance
• Develop and enforce data quality checks and validation mechanisms
• Collaborate with cross-functional teams, including data scientists, analysts, and business stakeholders, to deliver impactful data solutions
• Understand business requirements and translate them into technical solutions
• Leverage CI/CD pipelines to streamline development, testing, and deployment of data engineering workflows
• Work with DevOps tools like Git, Jenkins, or Azure DevOps for version control and automation
• Maintain clear documentation for data workflows, pipelines, and processes
• Optimize data systems for performance, scalability, and cost-efficiency

Requirements:

• Bachelor's or Master's degree in Computer Science, Information Technology, Engineering, or a related field
• 3–6 years of experience in Data Engineering or related roles
• Hands-on experience with big data processing frameworks, data lakes, and cloud-native services
• Proficiency in Python, SQL, and PySpark for data processing and manipulation
• Proven experience in Databricks and Apache Spark
• Expertise in working with cloud platforms like Azure, AWS
• Sound knowledge of ETL processes and tools like Alteryx
• Leveraging data lakes, data warehouses, and data pipelines
• Build a Data Pipeline from scratch
• Strong understanding of distributed systems and big data technologies
• Basic understanding of DevOps principles and familiarity with CI/CD pipelines
• Hands-on experience with tools like Git, Jenkins, or Azure DevOps
• Strong problem-solving skills and a knack for optimizing data solutions
• Excellent communication (oral and written) skills
• Familiarity with data visualization tools like Power BI, Tableau, or similar
• Knowledge of streaming technologies such as Kafka or Event Hubs