Software Engineer - Hosted Model Infrastructure

Palantir Technologies builds the world’s leading software for data-driven decisions and operations. They are seeking a Software Engineer to enable machine learning models in production, focusing on building high-performance infrastructure and deployment pipelines for AI capabilities.

Responsibilities:

• Building high-performance model serving infrastructure that integrates with security models, hardware constraints, and different inference engines
• Designing intelligent request handling including authentication, rate limiting, concurrency control, and audit logging for multi-tenant model access
• Building and maintaining packaging and deployment pipelines enabling fast, secure, and reliable model rollouts across on-premises and air-gapped environments
• Developing observability for production AI systems to enable easy service monitoring and fast incident triage and resolution
• Debugging complex issues and performance problems throughout the stack, including open source inference engines, container runtimes, and GPU drivers, in environments you cannot always access directly
• Designing and running testing and benchmarking infrastructure that validates model deployments across varying GPU hardware before they reach production
• Working with product teams and customers to understand requirements, debug production issues, and deliver the models and capabilities they need
• Integrating hosted model infrastructure with Palantir's deployment, configuration, and identity systems