Palantir Technologies builds the world’s leading software for data-driven decisions and operations. They are seeking a Software Engineer to enable machine learning models in production, focusing on building high-performance infrastructure and deployment pipelines for AI capabilities.
Responsibilities:
- Building high-performance model serving infrastructure that integrates with security models, hardware constraints, and different inference engines
- Designing intelligent request handling including authentication, rate limiting, concurrency control, and audit logging for multi-tenant model access
- Building and maintaining packaging and deployment pipelines enabling fast, secure, and reliable model rollouts across on-premises and air-gapped environments
- Developing observability for production AI systems to enable easy service monitoring and fast incident triage and resolution
- Debugging complex issues and performance problems throughout the stack, including open source inference engines, container runtimes, and GPU drivers, in environments you cannot always access directly
- Designing and running testing and benchmarking infrastructure that validates model deployments across varying GPU hardware before they reach production
- Working with product teams and customers to understand requirements, debug production issues, and deliver the models and capabilities they need
- Integrating hosted model infrastructure with Palantir's deployment, configuration, and identity systems