Machine Learning Engineer, Ops

Cantina Labs is a social AI company, developing advanced real-time models that enhance storytelling and creativity. They are seeking an MLOps Engineer to build and scale the inference infrastructure for generative audio models, focusing on high-performance systems for model serving.

Responsibilities:

• Design and maintain inference infrastructure for generative audio model architectures
• Implement and manage high-performance inference engines
• Orchestrate service deployments using Kubernetes (K8S), implementing advanced autoscaling paradigms to handle varying traffic loads efficiently
• Develop and automate robust CI/CD pipelines to streamline the testing and deployment of model artifacts and inference configurations
• Monitor production systems, establishing observability practices to track latency, resource utilization, and overall model performance
• Collaborate closely with research teams to optimize model serving paths and evaluate various inference strategies
• Optimize inference performance for both streaming and batch applications

Requirements:

• Deep understanding of modern audio model architectures (e.g., TTS, ASR) and their specific inference requirements
• Strong hands-on experience with Kubernetes (K8S), container orchestration, and implementing autoscaling strategies for production workloads
• Solid background in MLOps, including CI/CD automation and managing scalable cloud infrastructure
• Proficiency in software engineering principles and experience with Python or Go for infrastructure tooling and backend services
• Experience with GPU-accelerated inference and performance profiling techniques
• Familiarity with high-performance inference engines (e.g., Triton Inference Server, vLLM-Omni) is a plus