Administer and manage the Databricks platform, including workspace configuration, cluster optimization, and performance tuning
Implement and maintain Unity Catalog for enterprise-grade data governance and security
Design, develop, and optimize ETL/ELT pipelines supporting large-scale data integration and analytics workflows
Architect and maintain Delta Lake and lakehouse environments to ensure high performance, scalability, and data reliability
Support AI/ML workflows and integration within Databricks environments
Apply SysEngOps and DevSecOps practices, including CI/CD pipelines, automation, and release management
Utilize version control systems (Git) to manage code repositories and track the development lifecycle
Implement identity and access management (IAM) and role-based access control (RBAC) policies across enterprise systems
Ensure compliance with DoD security standards and data governance policies
Collaborate with cross-functional teams to design and deploy cloud-based data solutions across AWS, Azure, and GCP
Leverage technologies including Python, SQL/NoSQL, Apache Spark, Databricks SQL, Terraform, and cloud-native services for large-scale analytics.
Requirements
Master’s degree or higher (e.g., Ph.D.) in Computer Science, Information Technology, Systems Engineering, Data Science, Business Administration, Engineering Management, or related field OR Minimum of 11 years of experience managing complex technical projects in enterprise data architecture, Databricks administration, and cloud data platforms
Must have strong background in: Databricks platform administration, including workspace configuration, Unity Catalog, cluster performance tuning, and cloud integration
Data pipeline development (ETL/ELT), Delta Lake optimization, and AI/ML workflow integration
SysEngOps, DevSecOps, version control systems (Git), and CI/CD pipelines for automated deployment and governance
Cloud security, role-based access control (RBAC), and compliance with DoD data policies across AWS, Azure, and GCP
Experience supporting DoD or federal government programs
Familiarity with data governance and compliance frameworks specific to defense environments