Key skills
HadoopHDFSJavaLinuxMapReducePythonSparkSQLCommunicationCollaboration
About this role
Role Overview
- Support pipelines end to end
- Build enhancements and new developments
- Build and deploy new data pipelines
- Identify optimisation opportunities
- Improvement on recovery time in case of prod failures
- Test prototypes and oversee handover to the Data Operations teams
- Attend and contribute to regular team and User meetings
- Responsible for the actual coding or programming of Hadoop applications
Requirements
- 2+ years’ experience working in Big data environment
- Familiarity with Hadoop ecosystem and its components
- Good knowledge of the concepts of Hadoop
- Solid experience in a working environment in Big Data development utilising SQL or Python
- Experience in Big Data development using Spark
- Experience in Hadoop, HDFS and MapReduce
- Experience in database design, development and data modelling
- Good knowledge in back-end programming, specifically java
- Experience with development in a Linux environment and its basic commands
- Ability to write reliable, manageable, and high-performance code
- Should have basic knowledge of SQL, database structures, principles, and theories
- Knowledge of workflow/schedulers
- Strong collaboration and communication skills
- Strong analytical and problem-solving skills
Tech Stack
- Hadoop
- HDFS
- Java
- Linux
- MapReduce
- Python
- Spark
- SQL
Benefits
- Health insurance
- Professional development opportunities
- Flexible working arrangements