Assist in the development of a secure batch/real-time data platform for a government client leveraging the latest commercial and open-source technologies
Code, test, and document new or modified data pipelines that meet functional / non-functional business requirements
Conduct logical and physical database design
Expand and grows data platform capabilities to solve new data and analytics problems
Conduct data analysis, identifying feasible solutions and enhancements to data processing challenges
Ensure that data models are consistent with the data architecture (e.g. entity names, relationships and definitions)
Perform root cause analysis on internal and external data and processes to answer specific business questions and identify opportunities for improvement

Bachelor's degree
Data pipeline development using data processing technologies and frameworks
Agile or other rapid application development methods
Data modelling and understanding of different data structures and their benefits and limitations under particular use cases
Experience in Public Cloud services, such as AWS. Practical experience with core services such as EC2, RDS, Lambda, Athena & Glue would be even better!
Configuring and tuning Relational and NoSQL databases, including both query processing and query planning, or other data processing infrastructure
Programming or scripting languages, such as Python
Test Driven Development with appropriate tools and frameworks

Senior Data Engineer – eDV

Key skills