Machine Learning Infrastructure Engineer

An unnamed software company is hiring a Machine Learning Infrastructure Engineer to improve production machine learning serving speed and efficiency. The role focuses on optimizing inference across multiple model types, building performance tooling, and partnering with research and infrastructure colleagues to launch models with efficient serving.

Responsibilities:

• Lead inference optimization from start to finish across language, audio and image-generation models
• Implement and tune memory management strategies for long-running sessions, including eviction, compression and efficient attention
• Evaluate, deploy and extend open-source model serving frameworks for specific workloads
• Profile and benchmark latency and throughput, then systematically eliminate bottlenecks
• Build internal tooling such as profiling viewers and end-to-end test harnesses to make optimization faster and more rigorous
• Speed up generative image model inference using techniques like distillation, caching and custom kernel work
• Apply and develop quantization methods to reduce memory use and raise throughput while preserving output quality
• Work with research and infrastructure colleagues so new models ship with optimized serving from the start