Sr. Manager, Incident Management and Site Reliability Engineering

Peloton Interactive is seeking a Sr. Manager in Incident Management and Site Reliability Engineering. The role involves architecting AI solutions and implementing integration layers to enhance Peloton’s enterprise infrastructure while ensuring compliance and security.

Responsibilities:

• Ship End-to-End AI Products: Take ambiguous problems and turn them into functional, high-performing AI features and standalone tools
• Peloton Enterprise Connectors: Implement standardized integration layers connecting agents to core enterprise systems using secure protocols like MCP
• Persistent & Compliant Memory: Solve the "State Problem" for AI by building memory architectures and vector stores (Pinecone, Pgvector) that maintain context while strictly adhering to data retention and deletion policies
• Zero-Trust & Identity Propagation: Partner with Security to implement Identity Propagation (e.g., OAuth On-Behalf-Of flows) and RBAC. Ensure agents execute tasks strictly using the user’s specific permissions, eliminating privilege escalation risks
• Data Privacy & PII Pipelines: Build "Data Clean Rooms" and automated PII masking/redaction pipelines to guarantee that sensitive member, employee, or proprietary financial data is never exposed to external model providers
• Compliance & Governance: Ensure all AI workflows comply with enterprise regulatory standards and internal data governance policies
• EvalOps & Safety Guardrails: Deploy continuous evaluation pipelines to test models for hallucinations, prompt injections, data leakage, and performance regressions before code hits production
• Cost & Latency Optimization: Implement Semantic Caching and smart routing logic (directing routine queries to smaller, cheaper models) to maintain low latency and efficient inference costs
• "Guide vs. Control" Guardrails: Participate in creating templates, SDKs, and libraries that enable non-engineers and analysts to "vibe code" and prototype safely without bypassing compliance or security controls

Requirements:

• 5–7+ Years of General Software Engineering: Strong foundational software engineering skills (distributed systems, async patterns, API design, microservices, and databases)
• Language-Agnostic Mindset: Deep fluency in Python or Go, but comfortable pivoting to whatever language or framework gets the job done fastest
• Production AI Execution: Hands-on experience deploying RAG (Retrieval-Augmented Generation) and Agentic Workflows into production using LLM APIs, vector databases, and orchestration frameworks