Tharu Technologies LLC is seeking a Senior Databricks Platform Engineer to support their enterprise modernization initiative focused on Azure-native infrastructure and data operations. The role involves architecture, operational engineering, and platform enablement across Databricks environments, requiring collaboration with cloud platform teams and DevOps organizations.
Responsibilities:
- Databricks Platform Engineering & Operations
- Support architecture and operational engineering across enterprise Databricks environments
- Design and maintain scalable Lakehouse, Delta Lake, and Unity Catalog solutions
- Support administration of:
- Databricks Workspaces
- Jobs
- Notebooks
- Clusters
- Optimize Spark workloads, cluster configurations, and platform performance
- Support batch and streaming data processing workloads
- Assist with multi-environment platform management across Dev, QA, and Production
- Platform Integration & DevOps Alignment
- Support integration of Databricks platforms with Azure-native ecosystems and enterprise applications
- Work across integrations involving:
- APIs
- Azure Storage Accounts
- Data Lakes
- Reporting and analytics platforms
- Collaborate with platform and DevOps teams to align Databricks activities with existing CI/CD and automation standards
- Support deployment workflows utilizing:
- GitHub
- Azure DevOps
- Infrastructure as Code frameworks
- Help ensure modernization activities do not disrupt existing automated deployment pipelines and operational processes
- Governance, Reliability & Platform Support
- Support governance, access management, monitoring, logging, and operational security practices
- Assist with implementation of:
- RBAC
- Lineage
- Encryption
- Platform governance controls
- Participate in troubleshooting, operational support, and root cause analysis activities
- Contribute to documentation, operational standards, and platform best practices
Requirements:
- 6+ years of experience within cloud, platform engineering, data engineering, or enterprise analytics ecosystems
- 3+ years of hands-on experience supporting enterprise Databricks environments
- Strong experience with Delta Lake
- Strong experience with Unity Catalog
- Strong experience with Spark workloads
- Strong experience with cluster operations
- Strong experience with Databricks Workspaces
- Experience with Azure cloud services and cloud-native platform concepts
- Familiarity with Azure DevOps
- Familiarity with GitHub workflows
- Familiarity with CI/CD pipeline concepts
- Familiarity with deployment automation
- Experience supporting APIs, distributed application integrations, and enterprise data platforms
- Strong troubleshooting and operational support capabilities
- Strong communication and stakeholder collaboration skills
- Experience with PySpark
- Experience with Python
- Experience with SQL
- Experience with Spark SQL
- Experience with Scala
- Familiarity with AKS/Kubernetes
- Familiarity with Terraform
- Familiarity with Bicep
- Familiarity with Infrastructure as Code practices
- Experience with Azure Data Lake
- Experience with Event Hub
- Experience with streaming architectures
- Experience with enterprise analytics environments
- Familiarity with monitoring and observability platforms such as Azure Monitor
- Familiarity with Datadog
- Exposure to enterprise modernization and cloud migration initiatives