Production Engineer - AI Storage & HPC Infrastructure

Role: Production Engineer - AI Storage & HPC Infrastructure

Location: United States

Duration: Fulltime (No Contractors)

Job id: TechM245010

Client: This requirement is with Meta through Amazon.

Work Type: Remote-first, with occasional on-site visits to customer offices. Candidates located near any Meta office location will be preferred.

Where strong candidates come from: HPC storage, cloud filesystem services, storage vendors, or storage-focused SRE/production-engineering teams. Prior model-development experience is not required and should not be a filter.

Sourcing keywords: distributed filesystems, NFS, POSIX, Lustre, ZFS/OpenZFS, Hammerspace, VAST, HPC storage, AI training I/O, checkpointing, S3, Linux performance, Kubernetes operators, Terraform, Helm.

We are seeking a highly skilled Production Engineer - AI Storage & HPC Infrastructure to support and optimize next-generation AI and High-Performance Computing (HPC) environments.

In this role, you will help build, operate, and scale high-performance storage platforms that support large-scale machine learning, AI training, advanced analytics, and distributed computing workloads. The ideal candidate will combine expertise in storage systems, Linux, automation, performance engineering, and infrastructure operations to ensure reliable and efficient access to critical datasets and model artifacts across enterprise-scale environments.

This is a hands-on engineering role involving storage architecture, performance optimization, automation, observability, and production support.

Key Responsibilities

• Design, deploy, and support distributed file systems and object storage platforms used by large-scale AI, analytics, and HPC workloads.
• Manage storage environments supporting shared research, analytics, model training datasets, and checkpoint storage requirements.
• Troubleshoot and optimize storage performance across metadata services, throughput, latency, caching, networking, and data-access patterns.
• Build and maintain automation for provisioning, capacity expansion, upgrades, recovery processes, and lifecycle management.
• Manage data movement, replication, migration, and synchronization across multiple clusters, regions, and cloud environments while maintaining security and data integrity.
• Develop tooling and automation to improve storage operations, scalability, monitoring, and reliability.
• Partner with platform engineering, infrastructure, security, and development teams to ensure highly available and resilient storage services.
• Participate in on-call support, incident response, root cause analysis, and continuous service improvement initiatives.
• Implement observability, alerting, health monitoring, testing strategies, and controlled production rollouts.

Required Skills & Experience

• Strong experience designing, operating, or supporting large-scale distributed storage platforms, high-performance file systems, or enterprise data infrastructure.
• Deep hands-on Linux administration and troubleshooting experience.
• Strong understanding of file systems, storage protocols, networking, concurrency, and performance optimization.
• Experience diagnosing production issues related to storage performance, data access, throughput bottlenecks, and system reliability.
• Hands-on programming or automation experience using one or more of the following:
  • Python
  • Go
  • C++
  • Rust
• Experience driving end-to-end reliability improvements, performance optimization initiatives, or large-scale operational enhancements.
• Strong debugging, analytical, and problem-solving skills.
• Experience working in highly available production environments with operational ownership responsibilities.

Preferred Qualifications

• Experience with Kubernetes, container orchestration platforms, and cloud-native infrastructure.
• Strong understanding of distributed filesystem concepts including metadata management, caching, consistency, failover, recovery, and high availability.
• Hands-on experience with one or more enterprise storage technologies such as:
  • Lustre
  • ZFS / OpenZFS
  • VAST Data
  • Pure Storage
  • Hammerspace
  • Similar large-scale distributed storage platforms
• Experience supporting AI, Machine Learning, Data Engineering, or HPC infrastructures.
• Understanding of high-throughput data ingestion, checkpoint management, data staging, and accelerator-cluster storage requirements.
• Experience with Infrastructure as Code (IaC) and automation tools including:
  • Terraform
  • Helm
  • Kubernetes Operators
• Experience implementing observability and monitoring solutions using metrics, logging, tracing, benchmarking, and capacity-planning tools.
• Familiarity with object storage technologies, S3-compatible storage, lifecycle management, tiering, replication, and cross-region data movement.

Preferred Profile

• 5+ years of experience in Storage Engineering, Platform Engineering, Site Reliability Engineering (SRE), Infrastructure Engineering, HPC Operations, or Cloud Infrastructure.
• Experience in enterprise-scale production environments supporting mission-critical workloads.
• Strong communication skills and ability to collaborate with globally distributed engineering teams.
• Self-driven engineer with a passion for automation, reliability, scalability, and operational excellence.

Submission Format:

Candidate Full Name as per passport

Email

Phone Number

Current Location

Work Authorization

Total Experience

Joining Availability

Salary / Rate

Any relatives working in Tech M (If yes, please specify relation)

Ex-Tech M (If yes, year of leaving Tech M)

Skill

Years Of Experience

Linux Administration

Distributed Storage Technologies

Lustre / ZFS / VAST / Pure / Hammerspace

Python Automation

Kubernetes

Cloud Infrastructure

Terraform / Infrastructure as Code

Storage Performance Tuning

Incident Management & RCA

Monitoring & Observability

HPC Environment Experience

AI Infrastructure Experience