MSR Technology GroupRemote
Big Data Engineer (W2 Position)
United States of America
Full Time
2 hours ago
H1B Sponsor Likely
Key skills
Big Data applicationsAWS EMRAWS EC2AWS ECSAWS S3AWS Step FunctionsApache AirflowAWS API GatewayScalaPySparkJavaREST APIsSOAP web servicesHadoopHDFSSparkMapReduceCassandraBatch processingPerformance tuningLinuxShell scriptingPythonSnowflakeData pipelinesShellAirflowAWSECSEC2S3API Gateway
About this role
MSR Technology Group is seeking a Big Data Engineer to join their team. The role involves building and optimizing Big Data applications and data pipelines using various technologies and platforms.
Requirements:
- 7+ years of experience building Big Data applications (batch & API)
- Strong AWS experience (EMR, EC2, ECS, S3, Step Functions, Airflow, API Gateway)
- Proficiency in Scala, PySpark, and Java (REST APIs & SOAP web services)
- Experience with Big Data technologies: Hadoop, HDFS, Spark, MapReduce
- Hands-on experience with Cassandra
- Strong batch processing & performance tuning (Spark, Hadoop, EMR)
- Solid Linux experience (shell scripting and/or Python)
- Experience building data pipelines in Snowflake (ingestion & transformation using Spark)