Senior Data Engineer

IntelliTech is seeking a Senior Data Engineer to support a large-scale Department of War data platform and analytics environment. The role focuses on designing, building, maintaining, and optimizing scalable data pipelines and data products for mission-critical analytics, reporting, and downstream use cases.

Responsibilities:

• Design, develop, and maintain scalable batch and streaming data pipelines supporting enterprise analytics and data operations
• Build and optimize Databricks and PySpark workflows for ingestion, transformation, validation, and processing of large datasets
• Develop robust ETL/ELT solutions across structured and unstructured data sources, including APIs, relational databases, NoSQL databases, and cloud storage
• Support data modeling, pipeline orchestration, data quality, lineage, and performance tuning across the data lifecycle
• Implement and maintain cloud-native data engineering solutions in environments such as AWS, Azure, or GCP
• Collaborate with engineers, analysts, data scientists, and other stakeholders to support reporting, analytics, and AI/ML data preparation workflows
• Integrate data pipelines into DevSecOps and CI/CD workflows and support secure, repeatable deployment practices
• Troubleshoot pipeline failures, improve observability, and optimize the performance and reliability of enterprise data services
• Contribute to technical documentation, engineering standards, and best practices for enterprise-scale data development
• Operate effectively within Agile or SAFe delivery environments supporting mission systems

Requirements:

• Active DoD Secret Clearance
• U.S. Citizenship required
• Bachelor's degree in Computer Science, Data Science, Engineering, Information Systems, or a related technical discipline and 8+ years of relevant experience; or Master's degree in a related field and 6+ years of relevant experience
• Strong hands-on experience designing, building, and maintaining production-grade data pipelines and integration solutions
• Strong experience with Databricks, Apache Spark, and PySpark in enterprise data environments
• Strong programming skills in Python and SQL
• Experience implementing ETL/ELT pipelines and data integration processes across multiple sources and platforms
• Experience with relational and non-relational databases, data modeling, and query optimization
• Experience working in cloud-based data environments such as AWS, Azure, or GCP
• Experience with source control, CI/CD pipelines, and modern software engineering practices
• Ability to identify and resolve issues related to data quality, pipeline performance, and operational reliability
• Strong communication and collaboration skills, with the ability to work effectively across technical and non-technical teams
• Experience with Java or similar languages is a plus
• Experience supporting enterprise data platforms in DoD or other federal environments
• Experience with data governance, metadata management, lineage, and access control concepts
• Experience supporting AI/ML data preparation and feature engineering workflows
• Experience with Airflow, Kafka, Docker, Kubernetes, or related orchestration and containerization technologies
• Experience supporting data engineering solutions in secure or multi-environment enterprise settings
• Relevant cloud or data engineering certifications