Data Engineer

540 is hiring a Data Engineer to support a U.S. Army financial-system modernization initiative. The role builds and operates scalable data pipelines and cloud-native AWS solutions, with a focus on reliable, secure, governed, and high-performing data workflows.

Responsibilities:

• Design, build, maintain, and troubleshoot reliable data pipelines throughout the software development lifecycle, supporting the modernization of legacy U.S. Army financial systems and migration to AWS cloud environments
• Develop Python-based workflows to ingest, transform, validate, and deliver structured and semi-structured data
• Build and support cloud-native data solutions using AWS services such as Glue, Lambda, S3, and CloudWatch
• Integrate data from multiple systems and sources, including APIs and various file formats, while supporting data accuracy, consistency, and reliability
• Work with large datasets, identify performance bottlenecks, and help improve pipeline efficiency and scalability
• Participate in technical design discussions and contribute to decisions around data integration, storage, performance, and scalability
• Implement automated testing, logging, monitoring, and error handling to support data quality and reliable system performance
• Write clean, reusable, and maintainable code while following established software engineering and DataOps practices
• Collaborate with engineers, architects, and stakeholders to translate business and technical requirements into working software
• Support secure data management practices, including access controls, data governance, and compliance with DoW requirements
• Participate in code reviews and Agile development activities, sharing ideas and contributing to engineering improvements
• Document data transformations, pipeline processes, and technical decisions to support maintainability and downstream users
• Identify opportunities to improve existing workflows, reduce manual processes, and strengthen the reliability of the Army's data ecosystem

Requirements:

• per client requirements, must be a U.S. Citizen with the ability to obtain a DoW Secret clearance
• 2+ years of related experience in data engineering, software development, data integration, or a closely related field
• Experience developing Python-based data transformations, scripts, or workflow automation
• Hands-on experience building, maintaining, or troubleshooting data pipelines and ETL/ELT processes
• Experience consuming data from APIs and working with file formats such as X12, JSON, flat files, or fixed-width files
• Experience working with AWS cloud services, particularly AWS Glue, AWS Lambda, Amazon S3, and CloudWatch Logs
• Familiarity with working with large datasets, including identifying and troubleshooting data processing or performance issues
• Understanding of data structures, data modeling concepts, and common approaches to organizing and transforming data
• Experience writing and maintaining automated tests using Pytest or similar testing frameworks
• Familiarity with data validation, error handling, logging, and debugging practices
• Understanding of data security, access controls, and data quality principles
• Experience using GitLab for version control and collaborative development
• Comfortable working through the terminal or command-line interface
• Experience using Jira and Confluence in an Agile development environment
• Ability to interpret ambiguous requirements, file specifications, and JSON structures to develop functional data solutions
• Strong problem-solving skills and ability to collaborate effectively with technical and non-technical stakeholders
• Must currently possess or have the ability to obtain an AWS Cloud Practitioner certification (or higher) within 30 days of hire
• Must currently possess or have the ability to obtain a CompTIA Security+ certification (or higher) within 30 days of hire
• Bachelor's Degree in Computer Science or related engineering field (preferred)
• Experience working with DoW systems, data, or other regulated environments
• Familiarity with X12/MILS processing
• Experience with Apache Spark, Kafka, or other distributed data processing frameworks
• Knowledge of data orchestration tools such as Apache Airflow or AWS Step Functions
• Exposure to Infrastructure-as-Code tools, including Terraform or CloudFormation
• Understanding of DevOps and DataOps practices, including CI/CD pipelines
• AWS certifications such as Solutions Architect, Data Analytics, or similar
• Experience developing reusable pipeline components or improving established engineering patterns
• Experience contributing to data architecture decisions, schema design, or pipeline performance optimization
• Familiarity with database performance optimization, indexing, or partitioning strategies
• Experience with data lineage, metadata management, or data cataloging tools
• Experience providing technical guidance, mentoring teammates, or contributing to engineering best practices
• Familiarity with AI-assisted development tools for coding, testing, documentation, or workflow automation