DATABRICKS DATA ENGINEER

National Data Solutions is supporting a data warehouse and RCM modernization program for a large national healthcare client. They are seeking a Databricks Data Engineer with strong healthcare claims data experience to build data pipelines and validate claims data integrity during the program's initial phase.

Responsibilities:

• Build and configure nightly automated data extraction pipelines from multiple source systems including practice management, clearinghouse, and EHR platforms
• Normalize and canonicalize claims data across heterogeneous sources into a unified enterprise data model
• Validate claims data integrity, completeness, and accuracy — including charge, payment, adjustment, denial, and ERA/EOB data — against a 24-month historical baseline
• Identify and document data quality gaps, feed failures, and reconciliation discrepancies
• Implement data lineage, refresh cadence, and governance controls within the Databricks environment
• Support Power BI connectivity from the Databricks semantic layer
• Coordinate directly with NDS domain leads and client stakeholders on data dictionary alignment, KPI logic, and validation findings
• Participate in UAT and resolve data quality issues surfaced during dashboard validation

Requirements:

• 3+ years of hands-on Databricks engineering experience including Delta Lake, Unity Catalog, and Databricks Workflows
• Direct experience with healthcare claims data — charge entry, claim submission, remittance, denial management, or AR — in a data engineering or analytics context
• Demonstrated ability to validate and reconcile complex multi-source claims datasets
• Proficiency in PySpark and SQL
• Experience connecting Databricks to Power BI via DirectQuery or semantic layer
• Working knowledge of HIPAA data handling requirements and PHI masking in non-production environments
• Strong communication and client-facing skills — you will interact regularly with NDS leads and client stakeholders and are expected to present findings, ask the right questions, and manage expectations professionally
• Ability to work independently against a defined scope and surface blockers clearly and promptly
• Familiarity with clearinghouse data structures, ERA/EOB formats, or CARC/RARC code sets
• Prior work in RCM, home health, or managed care analytics environments