Machine Learning Engineer - Voice Conversion

Cantina Labs is a new social platform founded by Sean Parker focused on advanced AI character creation. They are seeking a Machine Learning Engineer to join their Speech Team, responsible for building state-of-the-art speech systems and collaborating across research, data, and infrastructure to deliver reliable AI models.

Responsibilities:

• Model Building: Architect, implement, pre-train, fine-tune, and post-train/alignment (e.g., GRPO/DPO) for large-scale speech models
• Experimental Design: Design, run, and analyze scientific experiments to advance our understanding of the models
• Tool Development: Develop and improve dev tooling to enhance team productivity
• Full-Stack Contribution: Contribute to the entire stack, from low-level optimizations to high-level model design
• Data Ownership: Define data requirements and collaborate on acquisition, curation, augmentation, labeling quality, and synthetic data strategies
• Rigorous Evaluation: Design automated objective/subjective evaluations—listening tests, SV/WER/ASR-based metrics, robustness & bias checks, and red-team studies
• Pipeline Delivery: Harden the training → evaluation → inference pipeline; profile latency, memory, and cost; and meet production SLAs with robust monitoring and rollback
• Safety & Responsibility: Contribute to safety/consent guardrails and to misuse/abuse mitigation for responsible speech technology

Requirements:

• Exceptional research/development experience with large-scale audio models (>8B parameters, >500k hours of data)
• Deep hands-on experience with diffusion and/or flow-matching transformers, including practical knowledge of samplers, schedules, conditioning mechanisms, and distillation
• Deep hands-on experience training audio VAEs, neural audio codecs, and vocoders latent/tokenizer design, reconstruction and perceptual objectives, adversarial training
• Strong experience with multi-node, multi-GPU distributed training (FSDP/DeepSpeed or equivalent)
• Strong software engineering skills with a proven track record of building complex systems
• Strong with PyTorch and performance work (profiling, CUDA/Triton/C++ as needed) and writing reliable production-quality code
• Shipped large-scale speech/audio or multimodal generative models to production
• Background in working with large-scale ML data, and the ability to iterate on data and triangulate quality using both subjective and objective signals
• Experience with voice cloning, speech control/steerability, or expressive speech generation
• Notable publications and/or open-source contributions in speech/audio/ML