Senior Databricks Platform Engineer - W2 position

Tharu Technologies LLC is seeking a Senior Databricks Platform Engineer to support their enterprise modernization initiative focused on Azure-native infrastructure and data operations. The role involves architecture, operational engineering, and platform enablement across Databricks environments, requiring collaboration with cloud platform teams and DevOps organizations.

Responsibilities:

• Databricks Platform Engineering & Operations
• Support architecture and operational engineering across enterprise Databricks environments
• Design and maintain scalable Lakehouse, Delta Lake, and Unity Catalog solutions
• Support administration of:
• Databricks Workspaces
• Jobs
• Notebooks
• Clusters
• Optimize Spark workloads, cluster configurations, and platform performance
• Support batch and streaming data processing workloads
• Assist with multi-environment platform management across Dev, QA, and Production
• Platform Integration & DevOps Alignment
• Support integration of Databricks platforms with Azure-native ecosystems and enterprise applications
• Work across integrations involving:
• APIs
• Azure Storage Accounts
• Data Lakes
• Reporting and analytics platforms
• Collaborate with platform and DevOps teams to align Databricks activities with existing CI/CD and automation standards
• Support deployment workflows utilizing:
• GitHub
• Azure DevOps
• Infrastructure as Code frameworks
• Help ensure modernization activities do not disrupt existing automated deployment pipelines and operational processes
• Governance, Reliability & Platform Support
• Support governance, access management, monitoring, logging, and operational security practices
• Assist with implementation of:
• RBAC
• Lineage
• Encryption
• Platform governance controls
• Participate in troubleshooting, operational support, and root cause analysis activities
• Contribute to documentation, operational standards, and platform best practices

Requirements:

• 6+ years of experience within cloud, platform engineering, data engineering, or enterprise analytics ecosystems
• 3+ years of hands-on experience supporting enterprise Databricks environments
• Strong experience with Delta Lake
• Strong experience with Unity Catalog
• Strong experience with Spark workloads
• Strong experience with cluster operations
• Strong experience with Databricks Workspaces
• Experience with Azure cloud services and cloud-native platform concepts
• Familiarity with Azure DevOps
• Familiarity with GitHub workflows
• Familiarity with CI/CD pipeline concepts
• Familiarity with deployment automation
• Experience supporting APIs, distributed application integrations, and enterprise data platforms
• Strong troubleshooting and operational support capabilities
• Strong communication and stakeholder collaboration skills
• Experience with PySpark
• Experience with Python
• Experience with SQL
• Experience with Spark SQL
• Experience with Scala
• Familiarity with AKS/Kubernetes
• Familiarity with Terraform
• Familiarity with Bicep
• Familiarity with Infrastructure as Code practices
• Experience with Azure Data Lake
• Experience with Event Hub
• Experience with streaming architectures
• Experience with enterprise analytics environments
• Familiarity with monitoring and observability platforms such as Azure Monitor
• Familiarity with Datadog
• Exposure to enterprise modernization and cloud migration initiatives