Software Engineer III, Copilot Models Inference

GitHub is a leading AI-powered developer platform used by developers and organizations worldwide. The Copilot Models Inference team is seeking a Software Engineer III to design, implement, deploy, and operate backend services and APIs that connect Copilot experiences to AI model providers, while improving reliability, performance, and safe model delivery.

Responsibilities:

• Design, implement, test, deploy, and maintain backend services and APIs that connect GitHub Copilot experiences to model providers
• Own scoped features end to end, from technical design and implementation through operational readiness, safe rollout, measurement, and follow-up
• Integrate new models and provider capabilities while accounting for availability, compatibility, capacity, failure modes, and safe fallback behavior
• Improve service latency, throughput, capacity efficiency, and high-availability behavior using measurable production outcomes
• Add and maintain automated tests, dashboards, alerts, runbooks, and diagnostics that make service and model health understandable and actionable
• Keep teammates and partners informed through clear discussions, written updates, and documentation, especially when decisions, risks, or incidents affect others
• Participate in code reviews and design discussions, incorporating feedback and helping teammates ship maintainable changes
• Participate in on-call and first-responder rotations, investigate incidents, communicate impact, restore service, and complete appropriate repair and automation work
• Use progressive delivery practices and production evidence to reduce risk when launching models, changing routing behavior, or updating provider integrations
• Collaborate with Models Lifecycle, product, safety, infrastructure, and provider partners to deliver model availability and operational improvements across team boundaries

Requirements:

• 4+ years experience in Software Engineering, Computer Science, or related technical discipline with proven experience maintaining and delivering production software languages including, but not limited to, C, C++, C#, JavaScript, Go, Ruby, Rust, or Python
• OR Associate's Degree in Computer Science, Electrical Engineering, Electronics Engineering, Math, Physics, Computer Engineering, Computer Science, or related field AND 3+ years experience
• OR Bachelor's Degree in Computer Science, Electrical Engineering, Electronics Engineering, Math, Physics, Computer Engineering, Computer Science, or related field AND 2+ years experience in Computer Science, or related technical discipline with proven experience coding in languages including, but not limited to, C, C++, C#, JavaScript, Go, Ruby, Rust, or Python
• OR Master's Degree in Computer Science, Electrical Engineering, Electronics Engineering, Math, Physics, Computer Engineering, Computer Science, or related field
• OR equivalent experience
• Experience applying software design principles such as clear interfaces, useful interfaces, useful abstractions, and composition to build maintainable and testable systems
• Experience working with both relational databases and NoSQL data stores, including making appropriate data-modeling and access-pattern tradeoffs
• Experience building or operating cloud-native services, including containers, Kubernetes, or managed cloud infrastructure
• Experience with AI or large language model inference systems, model APIs, or model-provider integrations
• Experience with observability and reliability engineering, including improving latency, capacity, resilience, or incident response for production services
• Experience with canary releases, feature flags, progressive delivery, or other safe rollout practices