CloudTech InnovationsRemote
Databricks Forward Deployed Engineer
United States of America
Contract
2 weeks ago
No H1B
Key skills
ScalaSQLETLELTData EngineeringDatabricksSparkKafkaAirflowdbtNode.jsFastAPIFlaskAWSAzureGCPTerraformDockerJenkinsGitHub ActionsKinesisAzure DevOpsPub/SubGitHubVersion ControlPerformance OptimizationCRMERPCI/CD
About this role
CloudTech Innovations is seeking an experienced Databricks Forward Deployed Engineer to act as the primary technical bridge between their data platforms and client success. The role involves managing multiple client projects, implementing Databricks-centric Lakehouse platforms, and ensuring rapid delivery while maintaining technical rigor.
Responsibilities:
- Manage technical priorities across multiple client engagements simultaneously, pivoting between distinct architectural needs in a fast-paced, high-demand environment
- Architect and maintain automated CI/CD pipelines (GitHub Actions, Azure DevOps, Jenkins) for data engineering workflows, ensuring robust version control, automated testing, and seamless deployment of Databricks assets
- Act as a core member of client delivery teams, translating business challenges into functional, high-impact data solutions
- Architect, build, and launch end-to-end Databricks Lakehouse platforms using the Medallion Architecture (Bronze/Silver/Gold)
- Own the full development lifecycle, from technical design and ETL/ELT pipeline construction (Databricks Workflows, SQL Warehouses, Spark) to deployment, monitoring, and production support
- Implement governed Lakehouse patterns using Unity Catalog, RBAC/ABAC, lineage, and compliance controls while balancing security with development agility
- Design distributed integration patterns across cloud-native services (AWS / Azure / GCP) and enterprise systems (CRM, ERP, Kafka/Kinesis streams)
- Lead architecture reviews, mentor client engineering teams, and drive cloud cost optimization across storage and compute
Requirements:
- Strong hands-on experience with Unity Catalog, Delta Live Tables (DLT), Photon Engine, and Lakehouse Federation
- Demonstrated experience implementing CI/CD pipelines for data platforms
- Proficiency with Terraform and version control workflows
- Proficiency in Spark (PySpark & Scala), complex query tuning, performance optimization, and pipeline automation (Airflow, dbt)
- Hands-on experience with cloud infrastructure (AWS / Azure / GCP), containerization (Docker), and building/debugging data-connected applications (FastAPI, Flask, Node.js)
- Experience with real-time streaming technologies (Kafka, Kinesis, Pub/Sub, Auto Loader)