Data Engineer - AI (Spark, Databricks and GCP)

Cotiviti is a data management company that is seeking a Data Engineer specialized in AI technologies. The role involves managing data implementations, integrations, and ensuring data quality and security while collaborating with various teams to troubleshoot issues.

Responsibilities:

• Create, maintain and execute intermediate to advanced Spark scripts for data management and data validation, and data integration
• Create, maintain and execute basic to intermediate SQL scripts for data management and data validation
• Optimize the queries to improve the efficiency of daily tasks
• Perform data analysis and identify any issues
• Work with other groups such as Engineering team, DBA, Cloud ops, etc. to troubleshoot and resolve any environmental or network issues that impact your work. Extend your support to after – hours or weekends as needed
• Create and maintain data pipelines as needed
• Validates the tasks results to ensure that all the requirements are met
• Adhere to all the industry level and organization level compliance rules and regulations to maintain data integrity
• Complete individual productivity tracking
• Complete task assignments using department ticketing system within assigned deadline
• Achieve organizational and individual goals as identified in performance reviews and goal setting exercises
• Complete all special projects and other duties as assigned
• Must be able to perform duties with or without reasonable accommodation

Requirements:

• Bachelor's degree in Computer Science, Information Technology or equivalent work experience
• 3+ years of working knowledge of big data technologies (Spark, S3, Kafka, Ray, Hadoop, etc.)
• 2+ years of working knowledge of big data / cloud technologies (Databricks, AWS, Azure, Hadoop, Spark, Snowflake etc.)
• 3+ years of working knowledge of cloud (AWS, Azure, GCP, OCI etc.)
• 3+ years of working knowledge of RDBMS (Oracle, MS SQL, Vertica, etc.) and experience using SQL, PL/SQL or other data integration/ETL tools
• 3+ years of data analysis
• Proficient in Microsoft Office Suite applications PowerPoint, Word, Excel and Outlook
• Strong analytical skills
• Excellent verbal, listening and written communication skills
• Ability to multitask and prioritize projects to meet scheduled deadlines and tight turnaround times
• Ability to work well independently or in a team environment
• Any Databricks / AWS certifications is a big plus
• Familiarity with data pipeline orchestration tools (e.g., Airflow, Databricks Workflows)
• Preferably in the Healthcare industry of enrollment, medical claims and/or pharmacy claims
• Experience with project management tools like JIRA
• Databricks and/or Snowflake environment familiarity a plus